Вы находитесь на странице: 1из 104

Chuck Laing

Senior Technical Staff Member


IBM GTS Storage

IBM SVC Zoning -


Best Practices 101

© Copyright IBM Corporation 2014


Technical University/Symposia materials may not be reproduced
in whole or in part without the prior written permission of IBM. 9.0
Please fill out an evaluation for
pSN561
• ibmtechu.com

Some great prizes


to be won!

© Copyright IBM Corporation 2014


Session objectives
By the end of this session you should know:
– Zoning is one of the most important configuration for proper:
• Failover
• Redundancy
• Resiliency
• Performance
– Understand best practice zoning for 4 port SVC models
– Understand best practice zoning for 8 port SVC models – CG8
– Understand best practice zoning for 12 port SVC models –DH8
– Understand best practice zoning for Dual Core Fabric to SVC
– Hopefully enjoyed learning Right and Wrong Zoning Scenarios by
playing along
– Had “Fun”

© Copyright IBM Corporation 2014


SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101


A Closer Look

© Copyright IBM Corporation 2014 4


Starting with ≤ CF8 Models
Physical to Logical evolution

© Copyright IBM Corporation 2014


SVC CG8 with extra FC HBA
Physical to Logical evolution

© Copyright IBM Corporation 2014


SVC CG8- Port Positioning
Physical to Logical evolution
Logical port #’s reflecting the embedded WWPN

Blue ports go to one Fabric


Red Ports go to the other Fabric

Logical port with wwpn # embedded

Physical port number

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


SVC DH8 – 12 Ports
Physical to Logical evolution
DH8 - Physical to Logical Port numbering for “existing” clusters

© Copyright IBM Corporation 2014


SVC DH8 – 12 Ports
Physical to Logical evolution
Understand the physical slot and port positioning
• DH8 - Physical to Logical port numbering for “new builds”.

© Copyright IBM Corporation 2014


Evolution and Types of Zones

IBM SVC Zoning - Best Practices 101


Backend/Storage to SVC Zoning
A Closer Look

© Copyright IBM Corporation 2014 10


SVC CG8 – 8 Ports / DH8 12 Ports
Evolution and Types of Zones
• Think of 4 types of zones per fabric (Use case - existing environment
– FA = Fabric A; FB = Fabric B
– SPx = Storage Port
– NxPx = SVC Node x Port x
– … = include multiple remaining node ports the same way

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


Storage to SVC Zones – 4 Port Node
Evolution and Types of Zones
• Example Back-end Storage to SVC Zoning
DS8K Left I/O Enclosures DS8K Right I/O Enclosures
Bay 0 Bay 2 Bay 4 Bay 6 Bay 1 Bay 3 Bay 5 Bay 7
C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1
L2 L4 L3 L7 L4 L8 L3 L7 R2 R4 R3 R7 R4 R8 R3 R7
0 0 0 0 0 0 0 0 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 6 66 6 6 6 6 6 1 1 1 1 1 1 1 1 3 3 3 3 3 3 3 3 5555 5555 7 7 7 7 7 7 7 7
3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 33 3 0 0 0 0 0 0 0 0 3 3 3 3 0000 3333 0 0 0 0 3 3 3 3
0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0123 0123 0 1 2 3 0 1 2 3

DIR1 SAN Fabric DIR2 SAN Fabric

Supported DS8K to SVC Zoning


Either ports 1&3 and 2&4 should be
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 zoned to a fabric or ports as shown on the
Node 1 Node 2 Node 3 Node 4 next page, both configs are supported
iogrp 0 iogrp 1
2048 LUNs max 2048 LUNs max
Newer SVC nodes may contain
wwpn 4 Node SVC MAX Vdisk 4096
5005076801 one HBA card with 4 ports
port1=11052ca port1=11052b7 port1=110529e port1=110528b
port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b
port4=14052ca port4=14052b7 port4=140529e port4=140528b

© Copyright IBM Corporation 2014


Storage to SVC Zones – 8 Port Node
Evolution and Types of Zones
• Example excluding heartbeat and MM/GM ports

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Storage to SVC Zones - 12 Port Node
Evolution and Types of Zones
• Example excluding heartbeat and MM/GM ports

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Storage to SVC Zones – 12 Port Node
Write Data Rate > 3GB/sec
• Example Back-end Storage to SVC Zoning

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


XIV Storage to SVC Zone Example
Storage Zone Type
• Example XIV Storage to SVC Zoning Ports 1 and 3 only

© Copyright IBM Corporation 2014


Flash Systems to SVC Zones
Storage Zone Type
Flash Systems
HBA1 HBA2
P P P P
1 2 1 2

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


V7000 Storage-SVC - 12 Ports
Storage Zone Type
CAUTION 1 2 1 2 3 4 1 2 3 4 CAUTION
3 4 1 2

1 2 1 2
3 4 3 4
Disconnect all Disconnect all
supply power for supply power for
complete isolation 1 2 complete isolation

1 2

2 1

2 1
complete isolation complete isolation
supply power for supply power for
Disconnect all Disconnect all
4 3 4 3
2 1 2 1
2 1 4 3
4 3 2 1 4 3 2 1 2 1
CAUTION CAUTION

SAN Fabric 1 SAN Fabric 2

Slide provided by Tron Pryor


18
© Copyright IBM Corporation 2014
EMC Symmetrix Storage
to SVC zoning with 12 ports
IBM, EMC Clariion, HP, etc. use one WWNN per subsystem; each appears as a single
controller with multiple ports/WWPNs
• Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024
Rule of 17

A A A A A A A A A A A A A A A A
S S S S S S S S
D D D D D D D D l l l l l l l l D
D D D D D D D
o o o o o o o o i i i i i i i
i i i i i i i i i
r r r r r r r
r r r r r r r r t t t t t t t t r
1 1 1 1 1 1 1
1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6
B B B B B B B B
1 1 1 1 1 1 1 1 B B B B B B B B
0 1 2 3 4 5 6 7
B B F F F F F F F F F F F F B B
E E E E E E E E E E E E E E E E

SAN Fabric 1 SAN Fabric 2

Slide provided by Tron Pryor 19


© Copyright IBM Corporation 2014
HDS USP Storage to SVC Zones
with 12 ports
HDS uses one WWNN per subsystem; each appears as a single controller with multiple
ports/WWPNs
• Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024
CL 1 - FRONT CHAs CL 2 - REAR CHAs
P/K CHPxx Port-Label P/K CHPxx Port-Label
LOC MPID HardID Port-Label LOC MPID HardID Port-Label
00 00 1A / 5A 40 40 2A / 6A
01 01 3A / 7A 41 41 4A / 8A
02 02 1B / 5B 42 42 2B / 6B
03 03 3B / 7B 43 43 4B / 8B

2Q
1E

04 04 1C / 5C 44 44 2C / 6C
05 05 3C / 7C 45 45 4C / 8C
06 06 1D / 5D 46 46 2D / 6D
07 07 3D / 7D 47 47 4D / 8D

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014 Slide provided by Tron Pryor


Storage Zones Hitachi Example
Storage Zone Type
HDS uses one WWNN per subsystem; each appears as a single controller with multiple
ports/WWPNs
• Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of 1024
Hitachi 9980 connected to SVC
Left Controller(1) Right Controller(2)
CHA P CHA P CHA P CHA P
1P 1Q 1R 1S 2V 2W 2X 2Y
E J K L M N P Q R
A B C D
C
E F G H
H J KL
L M N P Q R A B C D
B F G H
F K P

01 02 03 04 05 06 07 08 01 02 03 04 05 06 07 08

Even SAN Fabric ODD SAN Fabric


09 10 11 12 13 14 15 16 09 10 11 12 13 14 15 16

P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2

SVC Node 1 Node 2 Node 3 Node 4


© Copyright IBM Corporation 2014
Storage to SVC Zones DS4k/5K
Storage Zone Type

Server 1 Server 2
SVC Cabling
Best Practice A B A B
and Zoning

SAN Fabric A SAN Fabric B

HOST ZONING SVC ZONING


Create a SVC/Host zone Create one zone in the
for each server that RED fabric with all the
receives storage from SVC node ports cabled
the SVC cluster. to Fabric A and create
one zone in the BLUE
Example: I/O G-0 I/O G-1 fabric with all the SVC
node ports cabled to
Zone Server 1 port A (RED) 1 2 1 2 Fabric B.
with all SVC node port 3's.
Node 1 Node 3 Example:
Zone Server 1 port B (BLUE)
with all SVC node port 2's. 3 4 3 4 All odd (RED) SVC node
ports in one zone and all
Zone Server 2 port A (RED) even (BLUE) SVC node ports
with all SVC node port 1's. 1 2 1 2 in one zone.
Zone Server 2 port B (BLUE) Node 2 Node 4 Note: For a cluster to be
with all SVC node port 4's. created and to operate
3 4 3 4 correctly all node ports must
*** NOTE ***
SVC be zoned together.
SVC supports a maximum of Cluster
256 host objects per I/O STORAGE
group thus a maximum of
1024 per cluster. The above ZONING
host zoning results in each
server being seen by every MDisk Group 1 / DS5K_1 Create a SVC/Storage
I/O group and the default zone for each storage
host object creation VDisk 1 subsystem virtualized
behavior results in each by the SVC cluster.
Cntrl A
MDisk10 / Array10

MDisk11 / Array11

MDisk12 / Array12

MDisk13 / Array13
host object counting as one
MDisk1 / Array1

MDisk2 / Array2

MDisk3 / Array3

MDisk4 / Array4

MDisk5 / Array5

MDisk6 / Array6

MDisk7 / Array7

MDisk8 / Array8

MDisk9 / Array9

towards this 256 maximum. VDisk 3 Example:


Channels 2
To create more then 256
host objects in the cluster and 4 Cntrl B Zone DS5K_1 controller A
and B daughter card
you must zone a host to a Channels 1 channel ports 1 and 3 with
subset of the I/O groups, Channels 1 all SVC node ports 1 and 3
you must assign the host and 3 in the RED fabric.
object at host creation time and 3
to that same subset of I/O Channels 2 Zone DS5K_1 controller A
groups and then you must VDisk 2 and B daughter card
assign that host’s VDisks to and 4 channel ports 2 and 4 with
one of those I/O groups in all SVC node ports 2 and 4
that same subset. VDisk 4 in the BLUE fabric.

22 © Copyright IBM Corporation 2014


Types of Zones
Host to SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101


Host Zones a Closer Look

© Copyright IBM Corporation 2014 23


Host Zones Evolution – 4 Port Node

© Copyright IBM Corporation 2014


Host Zones evolution – 8 Port Node

© Copyright IBM Corporation 2014


Host Zones Evolution – 12 Port Node
• SVC - DH8
Correct
Host
A1 B1

Fabric1 Core1 Fabric2 Core1

© Copyright IBM Corporation 2014


Zoning Multi HBA hosts for
Resiliency
• Right way to zone a multiport Host
– Make four zones (Red, Blue, Orange and Green zones)
• Spread IO across multiple SVC Ports
Multi Port Host
HBA1 HBA2
P P P P P
1 2 1 2 1 2

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Zoning Multi HBA hosts for
Redundancy and Resiliency
• Sys Admins – provide PCI slot to Port WWPN identity to Storage Admins
• Storage Admins – define the SVC host definitions to match
– Avoid single points of hardware failure at the Host HBA, Fabric and SVC
– Make four zones, one for each sudo host per fabric(Red, Blue, Orange and Green zones)

SVC defined Sudo Host1 Physical Host SVC defined Sudo Host2
HBA1 HBA2 HBA1 HBA2 HBA1 HBA2
P P P P P P P P P P
1 1 2 1 2 1 2 1 2 1

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Zoning Multi HBA hosts for
Resiliency
• Sys Admins – provide PCI slot to Port WWPN identity to Storage Admins
• Storage Admins – define the SVC host definitions to match
– Avoid single points of hardware failure at the Host HBA, Fabric and SVC
– Make four zones, one for each sudo host per fabric(Red, Blue, Orange and Green zones)

SVC defined Sudo Host1 Physical Host SVC defined Sudo Host2
HBA1 HBA2 HBA1 HBA2 HBA1 HBA2
P P P P P P P P P P
1 1 2 1 2 1 2 1 2 1

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Types of Zones - Host to SVC
AIX VIOS to SVC Zoning Concepts

IBM SVC Zoning - Best Practices 101


A Closer Look

© Copyright IBM Corporation 2014 30


VIOS Simplistic Dual Path Overview

© Copyright IBM Corporation 2014


N_Port ID Virtualization

c0:50:76:07:0f:5f:00:20 c0:50:76:07:0f:5f:00:30

© Copyright IBM Corporation 2014


Host Zones
Best Practices Concepts
• Planning and design for AIX Live Partition Mobility (LPM)
– Create two separate and isolated zones on each fabric for each LPAR
• Do not put both the active and inactive LPAR wwpns in either the same zone or
same SVC host definition
– Create Sudo host SVC host definitions containing only two virtual
wwpns, one from each fabric
• Follow the rules shown in the next few slides
– Map LUNs to the virtual host FC HBA port wwpns not he physical host
FCA Adapter wwpn
• Exception to map only boot from SAN LUNs to the physical host FC HBA wwpn
• Share the Vdisks to both LPARs (both the active and inactive) to enable LPM to
see the LUNs in both LPARs

– Note: Following these rules will result in not having a degraded path error on the
SVC, and keep the 4 paths per Vdisk rule

© Copyright IBM Corporation 2014


Dual VIOS to Single LPAR Example

VIO Server1 VIO Server2


4
H P1 P3 H
B B
A A
3
1 P2 P4 2

VP1 VP3 1
VHBA1 VHBA2

VP2 VP4
2

SVC

© Copyright IBM Corporation 2014


Dual VIOS to Multiple LPARs
Zoning for Resiliency

VP1.1
VIO Server1 VIO Server2
Sudo 1
P1 P3 Client
HBA1 SAN HBA2 VP4.1 Logical
VP3.1 Partition
P2 P4 (LPAR1)
Sudo 2
VP1.1 VP3.1
VP2.1
VP1.2 VP3.2
VHBA1 VHBA2
VP2.1 VP4.1 VP1.2
SVC
VP2.2 VP4.2 Sudo1
P3 … P8 P3 … P8 Client
VP4.2
Logical
VP3.2 Partition
(LPAR2)
Sudo2

VP2.2

Hypervisor

© Copyright IBM Corporation 2014


Dual VIOS to Multiple LPARs
One VIOS Failure

VIO Server1 VIO Server2 x


VP1.1
Sudo 1
P1 P3 VP4.1 Client
HBA1 HBA2 Logical
SAN Partition
VP3.1
P2 P4 (LPAR1)
Sudo 2

VP1.1 VP3.1
VP1.2 VP3.2
x
VP2.1

VHBA1 VHBA2
VP2.1 VP4.1 VP1.2
VP2.2
SVC
VP4.2 xSudo 1
P3 … P8 P3 … P8 Client
VP4.2
Logical
VP3.2 Partition
(LPAR2)
Sudo 2

x
VP2.2

Hypervisor

© Copyright IBM Corporation 2014


Dual VIOS to Multiple LPARs
One SAN Fabric Failure

VP1.1
VIO Server1 VIO Server2
Sudo 1

HBA1
P1 P3
HBA2
x
VP4.1 Client
Logical
SAN Partition
xP2 x
P4 VP3.1
Sudo 2
(LPAR1)

VP1.1 VP3.1
VP1.2 VP3.2
x
VP2.1

VHBA1 VHBA2
VP2.1 VP4.1
x
VP2.2
SVC x
VP4.2
VP1.2
Sudo 1
P3 … P8 P3 … P8
x
VP4.2 Client
Logical
VP3.2 Partition
(LPAR2)
Sudo 2

x
VP2.2

Hypervisor

© Copyright IBM Corporation 2014


Types of Zones
Host ESX to SVC Zones

© Copyright IBM Corporation 2014


Types of Zones
Host ESX to SVC Zones

2+2 =4 Paths per LUN

© Copyright IBM Corporation 2014


Types of Zones
SVC Node to Node / Heartbeat Zones

IBM SVC Zoning - Best Practices 101


A Closer Look

© Copyright IBM Corporation 2014 40


Node to Node / Heartbeat Zones
The Intra-cluster/Heartbeat/node to node communication zone
– Make the local_fc_port_mask (Intra-cluster) = 000011000000 for 12 ports
– Only make the local_fc_port_mask (Intra-cluster) = 110011000000
• if Write Data Rate > 3GB/sec per Iogrp
– In dual core fabrics, all dedicated heartbeat ports should reside on only one core

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


Types of Zones
Metro Mirror / Global Mirror Zones

IBM SVC Zoning - Best Practices 101


MM/GM - A Closer Look

© Copyright IBM Corporation 2014 42


Mirror Zones Evolution
MM/GM and Heartbeat – 4 Port Nodes
• One SVC port from each node on each fabric should be zoned for GM traffic, taking care not to include both ports that a
server might use. This means ports that would also be used for replication would be either ports 1 & 2, or ports 3 & 4
• For each node in a cluster, exactly two fibre channel ports should be zoned to exactly two fibre channel ports from each node
in the partner cluster.
• If dual-redundant ISLs are available, then the two ports from each node should be split evenly between the two ISLs, i.e.
exactly one port from each node should be zoned across each ISL.
• Local cluster zoning should continue to follow the standard requirement for all ports on all nodes in a cluster to be zoned to
one another.
• This is discussed more verbosely on the Flash published for this issue on the IBM website:
– http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003634 and https://www-304.ibm.com/support/docview.wss?uid=ssg1S1003634

© Copyright IBM Corporation 2014


Mirror Zones Evolution
SVC GG8 – 8 Port Node
• Make the local_fc_port_mask (Node to Node/heartbeat) = 11000000
• Make the partner_fc_port_mask (MM/GM) = 00110000

WAN direct connection to the WAN direct connection to the


FCiP connection to the SVC ports FCiP connection to the SVC ports
GM/MM zone A GM/MM zone B

© Copyright IBM Corporation 2014


Mirror Zones Evolution
SVC DH8 – 12 Port Node
• Make the local_fc_port_mask(Intra-cluster)=000011000000 for 12 ports
• Make the local_fc_port_mask(Intra-cluster)=110011000000 - Write Data Rate > 3GB/sec per
IOgrp
• Make the partner_fc_port_mask (MM/GM) = 000000110000

WAN direct connection to the WAN direct connection to the


FCiP connection to the SVC ports FCiP connection to the SVC ports
GM/MM zone A GM/MM zone B

© Copyright IBM Corporation 2014


Planning Design example for correct
“Dual Core” fabric zoning

IBM SVC Zoning - Best Practices 101


Dual Core Design - A Closer Look

© Copyright IBM Corporation 2014 46


Dual Core Fabric
Storage Back-end Problem Scenario
• Problem: A single back-end zone had ports p1 and p2 from SVC 7 and port a3 from XIV (example). SVC 7 will form paths
from p1 to a3 (purple) and from p3 to a3 (green). Purple paths traverse 2 cores and 1 edge switch – interfering with host
edge-to-core traffic. This issue is pervasive. Also, eliminate (unnecessary) separate heartbeat zone.
• Fix: Split zones so that only common-core switch (green paths) can occur. Reduce paths, contention, latency. Also
group XIVn array ports attaching common core switch in common zone.

Slide provided by Kirby Dahman


© Copyright IBM Corporation 2014
Dual Core Fabric
Host Front-end Problem Scenario
• Problem: Many host HBA ports are in a single zone with SVC ports cabled to different core switches. This
does not hurt the host directly, but as a side-effect it permits SVC inter-node cache updates (after each host
write) from preferred to non-preferred node in an IOgroup to cross from core-switch-to-edge switch-to-core
switch (see purple path).
• Fix: Split zones so that only common-core switch (green paths) can occur for inter-node cache updates.

© Copyright IBM Corporation 2014


Incorrect “Dual Core” fabric zoning
example for a 2 node SVC Cluster

© Copyright IBM Corporation 2014 Slide provided by John Locke


Correct “Dual Core” fabric zoning
example for a 2 node SVC Cluster

© Copyright IBM Corporation 2014 Slide provided by John Locke


SVC Cluster zone for a
“Single vs. Dual Core” per fabric

• A Single core cluster “zone” Brocade • A Dual Core cluster “zone” Brocade
Example Example
– CLI View of one Fabric /8 node SVC – CLI View of one Fabric /8 node SVC “Fabric 1
• zone: lsan_allsvc_zone_a Core1”
• 50:05:07:68:01:10:54:28 • zone: lsan_allsvc_zone_a
• 50:05:07:68:01:40:54:28 • 50:05:07:68:01:10:54:28
• 50:05:07:68:01:10:54:62 • 50:05:07:68:01:10:54:62
• 50:05:07:68:01:40:54:62 • 50:05:07:68:01:10:51:07
• 50:05:07:68:01:10:51:07 • 50:05:07:68:01:10:54:73
• 50:05:07:68:01:40:51:07 • 50:05:07:68:01:10:4e:c3
• 50:05:07:68:01:10:54:73 • 50:05:07:68:01:10:45:64
• 50:05:07:68:01:40:54:73 • 50:05:07:68:01:10:4f:67
• 50:05:07:68:01:10:4e:c3 • 50:05:07:68:01:10:4f:66
• 50:05:07:68:01:40:4e:c3
• 50:05:07:68:01:10:45:64
• 50:05:07:68:01:40:45:64
– CLI View of one Fabric /8 node SVC “Fabric
• 50:05:07:68:01:10:4f:67 Core2”
• 50:05:07:68:01:40:4f:67 • 50:05:07:68:01:40:54:28
• 50:05:07:68:01:10:4f:66 • 50:05:07:68:01:40:54:28
• 50:05:07:68:01:40:4f:66 • 50:05:07:68:01:40:54:62
• 50:05:07:68:01:40:51:07
• 50:05:07:68:01:40:54:73
• 50:05:07:68:01:40:4e:c3
• 50:05:07:68:01:40:45:64
• 50:05:07:68:01:40:4f:67
• 50:05:07:68:01:40:4f:66

 Slide provided by Chuck Laing

51
Incorrect vs. correct Dual core fabric
Host to SVC port zoning
Right or Wrong?
Host
A1 B1

Fabric1 Fabric 1 Fabric2 Fabric 2


Core1 Core2 Core1 Core2

52
Incorrect vs. correct Dual core fabric
Host to SVC port zoning
Incorrect Correct
Host Host
A1 B1 A1 B1

Fabric1 Fabric 1 Fabric2 Fabric 2 Fabric1 Fabric 1 Fabric2 Fabric 2


Core1 Core2 Core1 Core2 Core1 Core2 Core1 Core2

53
Correct Dual core fabric / Host to
SVC port zoning
Right or Wrong?
Red and Blue cores have ISLs between cores respectively?
Host1 Host2

A1 B1 A1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

43 12 56 78 43 12 56 78 43 12 56 78 43 12 56 78
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

12 34 56 78 12 34 56 78 12 34 56 78 12 34 56 78
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2
Node 1 Node 2 Node 1 Node 2
iogrp 0 iogrp 1
2048 LUNs max 2048 LUNs max

4 Node CG8 SVC

© Copyright IBM Corporation 2014


Correct Dual core fabric / Host to
SVC port zoning

Correct Correct
Host1 Host2

A1 B1 A1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

43 12 56 78 43 12 56 78 43 12 56 78 43 12 56 78
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

12 34 56 78 12 34 56 78 12 34 56 78 12 34 56 78
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2
Node 1 Node 2 Node 1 Node 2
iogrp 0 iogrp 1
2048 LUNs max 2048 LUNs max

4 Node CG8 SVC

© Copyright IBM Corporation 2014


Dual core fabric / Multiple Host - ESX
Cluster to SVC port zoning

Host_1a Host_1b Host_


Host_2a Host_2b
A1
A1 B1 A1
A1 B1 A1 B1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

 Slide provided by Chuck Laing


56
Dual core fabric / Multiple Host - ESX
Cluster to SVC port zoning

Host_1a Host_1b Host_


Host_2a Host_2b
A1 B1 A1
A1 B1 A1 B1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

 Slide provided by Chuck Laing


57
Dual core fabric / Multiple Host - ESX
Cluster to SVC port zoning

Host_1a Host_1b Host_


Host_2a Host_2b
A1 B1 A1
A1 B1 A1 B1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

 Slide provided by Chuck Laing


58
Dual core fabric / Multiple Host - ESX
Cluster to SVC port zoning

Host_1a Host_1b Host_


Host_2a Host_2b
A1 B1 A1
A1 B1 A1 B1 B1

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

 Slide provided by Chuck Laing


59
Dual core fabric / Multiple Host - ESX
Cluster to SVC port zoning
• Zone a single HA/ESX cluster to the same core to avoid ISL IO traversing
• Spread Multiple Host clusters evenly across Cores
• Create separate zones for each Host node in the ESX cluster or AIX HA cluster

© Copyright IBM Corporation 2014


Correct Dual Core Zoning to Multiple
Storage devices
• Zone each device to only one core to avoid extra “ISL IO traversing” cutting
down ISL hops
• Spread Multiple Storage devices as evenly possible across Cores
• Create separate zones for each Storage device

DS8K_1 DS8K_2 XIV V7000

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

© Copyright IBM Corporation 2014


Correct Dual Core Zoning to Multiple
Storage devices
• Zone each device to only one core to avoid extra “ISL IO traversing” cutting
down ISL hops
• Spread Multiple Storage devices as evenly possible across Cores
• Create separate zones for each Storage device

DS8K_1 DS8K_2 XIV V7000

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

© Copyright IBM Corporation 2014


Correct Dual Core Zoning to Multiple
Storage devices
• Zone each device to only one core to avoid extra “ISL IO traversing” cutting
down ISL hops
• Spread Multiple Storage devices as evenly possible across Cores
• Create separate zones for each Storage device

DS8K_1 DS8K_2 XIV V7000

Fabric1 Core1 Fabric 1 Core2 Fabric2 Core1 Fabric 2 Core2

© Copyright IBM Corporation 2014


SVC DH8- Mirror Zones
• Make the local_fc_port_mask (Node to Node/heartbeat)=000010001000
• Make the partner_fc_port_mask (MM/GM = 000001000100

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


Right or Wrong way Scenarios

IBM SVC Zoning - Best Practices 101


Lets Have Some Fun, with Right or Wrong

© Copyright IBM Corporation 2014 65


Zoning Multi HBA hosts for
Resiliency
• Right or Wrong?

Multiport Host
HBA1 HBA2
P P P P P
1 2 1 2 1 2

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Zoning Multi HBA hosts for
Resiliency
• Is this better?

Multiport Host
HBA1 HBA2
P P P P P
1 2 1 2 1 2

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Zoning Multi HBA hosts for
Resiliency
• Right way to zone a multiport Host
– Make four zones (Red, Blue, Orange and Green zones)
• Spread IO across multiple SVC Ports
Multi Port Host
HBA1 HBA2
P P P P P
1 2 1 2 1 2

SAN Fabric 1 SAN Fabric 2

© Copyright IBM Corporation 2014


Right or Wrong Game/Storage - SVC
• Which example is Right?

1 2

DS8K Left I/O Enclosures DS8K Right I/O Enclosures DS8K Left I/O Enclosures DS8K Right I/O Enclosures
Bay 0 Bay 2 Bay 4 Bay 6 Bay 1 Bay 3 Bay 5 Bay 7 Bay 0 Bay 2 Bay 4 Bay 6 Bay 1 Bay 3 Bay 5 Bay 7
C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1
L2 L4 L3 L7 L4 L8 L3 L7 R2 R4 R3 R7 R4 R8 R3 R7 L2 L4 L3 L7 L4 L8 L3 L7 R2 R4 R3 R7 R4 R8 R3 R7
0 0 0 0 0 0 0 0 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 6 66 6 6 6 6 6 1 1 1 1 1 1 1 1 3 3 33 3 3 3 3 5555 5 555 7 7 7 7 7 77 7 0 0 0 0 0 0 0 0 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 6 66 6 6 6 6 6 1 1 1 1 1 1 1 1 3 3 33 3 3 3 3 5555 5 555 7 7 7 7 7 77 7
3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 33 3 0 0 0 0 0 0 00 3 3 3 3 0000 3 333 0 0 0 0 3 33 3 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 33 3 0 0 0 0 0 0 00 3 3 3 3 0000 3 333 0 0 0 0 3 33 3
0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 23 0 1 2 3 0123 0 123 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 23 0 1 2 3 0123 0 123 0 1 2 3 0 12 3

DIR1 SAN Fabric DIR2 SAN Fabric DIR1 SAN Fabric DIR2 SAN Fabric

A2 B2 A2 B2
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 Application Host Server HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 Application Host Server

Node 1 Node 2 Node 3 Node 4 The same native Node 1 Node 2 Node 3 Node 4 The same native
iogrp 0 iogrp 1 backend ports should not be iogrp 0 iogrp 1 backend ports should not be
2048 LUNs max 2048 LUNs max 2048 LUNs max 2048 LUNs max
shared for both direct host connectivity shared for both direct host connectivity
wwpn 4 Node SVC MAX Vdisk 4096 wwpn 4 Node SVC MAX Vdisk 4096
5005076801 and SVC connectivity as shown here. 5005076801 and SVC connectivity as shown here.
port1=11052ca port1=11052b7 port1=110529e port1=110528b The correct way to bypass SVC is to use port1=11052ca port1=11052b7 port1=110529e port1=110528b The correct way to bypass SVC is to use
port2=12052ca port2=12052b7 port2=120529e port2=120528b port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b other backend ports not connected to the port3=13052ca port3=13052b7 port3=130529e port3=130528b other backend ports not connected to the
port4=14052ca port4=14052b7 port4=140529e port4=140528b port4=14052ca port4=14052b7 port4=140529e port4=140528b
SVC SVC

© Copyright IBM Corporation 2014


Right or Wrong Game/Storage - SVC
SVC Correct Example SVC Incorrect Example

The same port on the back-end is being used for host and SVC

DS8K Left I/O Enclosures DS8K Right I/O Enclosures DS8K Left I/O Enclosures DS8K Right I/O Enclosures
Bay 0 Bay 2 Bay 4 Bay 6 Bay 1 Bay 3 Bay 5 Bay 7 Bay 0 Bay 2 Bay 4 Bay 6 Bay 1 Bay 3 Bay 5 Bay 7
C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1 C0 C1
L2 L4 L3 L7 L4 L8 L3 L7 R2 R4 R3 R7 R4 R8 R3 R7 L2 L4 L3 L7 L4 L8 L3 L7 R2 R4 R3 R7 R4 R8 R3 R7
0 0 0 0 0 0 0 0 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 6 66 6 6 6 6 6 1 1 1 1 1 1 1 1 3 3 33 3 3 3 3 5555 5 555 7 7 7 7 7 77 7 0 0 0 0 0 0 0 0 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 6 66 6 6 6 6 6 1 1 1 1 1 1 1 1 3 3 33 3 3 3 3 5555 5 555 7 7 7 7 7 77 7
3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 33 3 0 0 0 0 0 0 00 3 3 3 3 0000 3 333 0 0 0 0 3 33 3 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3 0 0 0 0 3 33 3 0 0 0 0 0 0 00 3 3 3 3 0000 3 333 0 0 0 0 3 33 3
0 0 0 0 3 3 3 3 0 0 0 0 3 3 3 3
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 23 0 1 2 3 0123 0 123 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 12 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 23 0 1 2 3 0123 0 123 0 1 2 3 0 12 3

DIR1 SAN Fabric DIR2 SAN Fabric DIR1 SAN Fabric DIR2 SAN Fabric

A2 B2 A2 B2
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 Application Host Server HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 Application Host Server

Node 1 Node 2 Node 3 Node 4 The same native Node 1 Node 2 Node 3 Node 4 The same native
iogrp 0 iogrp 1 backend ports should not be iogrp 0 iogrp 1 backend ports should not be
2048 LUNs max 2048 LUNs max 2048 LUNs max 2048 LUNs max
shared for both direct host connectivity shared for both direct host connectivity
wwpn 4 Node SVC MAX Vdisk 4096 wwpn 4 Node SVC MAX Vdisk 4096
5005076801 and SVC connectivity as shown here. 5005076801 and SVC connectivity as shown here.
port1=11052ca port1=11052b7 port1=110529e port1=110528b The correct way to bypass SVC is to use port1=11052ca port1=11052b7 port1=110529e port1=110528b The correct way to bypass SVC is to use
port2=12052ca port2=12052b7 port2=120529e port2=120528b port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b other backend ports not connected to the port3=13052ca port3=13052b7 port3=130529e port3=130528b other backend ports not connected to the
port4=14052ca port4=14052b7 port4=140529e port4=140528b port4=14052ca port4=14052b7 port4=140529e port4=140528b
SVC SVC

© Copyright IBM Corporation 2014


3 factors determine right/wrong
number of paths per vdisk device
• Example of correct/incorrect number of datapaths with Host to SVC
zoning
• Which Side is right?

© Copyright IBM Corporation 2014


Summary - 3 factors determine
right/wrong paths
• Example of correct/incorrect number of datapaths with Host to SVC zoning
• Correct datapathing has 3 factors
• Proper zoning
• Proper SVC Host definitions (SVC logical config of the host def)
• Proper redundancy for the SVC preferred /non preferred pathing

© Copyright IBM Corporation 2014


Right or Wrong ?
fscsi5=10000000C9C0DC7F
fscsi0=10000000C9C0A984 fscsi7=10000000C9C0B3DB
fscsi2=10000000C9C0E0E0
Zone for p770_1_vio1a_d3_SVC Zone for p770_1_vio1a_d4_SVC
Zone for p770_1_vio1a_d1 Zone for p770_1_vio1a_d2
b03vio101
NRPOKVIO1A 10000000C9C0DC7F 10000000C9C0B3DB
10000000c9779a4a 10000000C9C0E0E0
500507680120B374 500507680120B374
500507680110B374 500507680110B374 d1 d4
d2 d3 500507680140B374 500507680140B374
500507680130B374 500507680130B374 500507680120B363
500507680120B363
500507680110B363 500507680110B363 500507680140B363
500507680140B363
500507680130B363 500507680130B363 500507680120B371
500507680120B371
500507680110B371 500507680110B371 500507680140B371
500507680140B371
500507680130B371 500507680130B371
500507680120B335 500507680120B335
500507680110B335 500507680110B335 500507680140B335
500507680140B335
500507680130B335 500507680130B335

DIR1 SAN Fabric DIR2 SAN Fabric

Over subscribed SVC to Host


HBA Zoning causing to many
datapaths
DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: Load
Balance
SERIAL: 600507680181059BA000000000000005
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
==========================================================
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 SVC Host DefinitionsPath# Adapter/Path Name State Mode Select Errors
id:1 0 fscsi0/path0 OPEN NORMAL 558254 0
Node 1 Node 2 Node 3 Node 4 name:P770_1_vio1A
1*
2*
fscsi0/path1
fscsi0/path2
OPEN NORMAL
OPEN NORMAL
197
197
0
0
10000000C9C0B3DB
iogrp 0 iogrp 1 10000000C9C0DC7F
3
4
fscsi0/path3
fscsi2/path4
OPEN NORMAL
OPEN NORMAL
493559
493330
0
0
2048 LUNs max 2048 LUNs max 10000000C9C0E0E0 5* fscsi2/path5 OPEN NORMAL 197 0
10000000C9C0A984 6* fscsi2/path6 OPEN NORMAL 197 0
wwpn 4 Node SVC MAX Vdisk 4096 CF8
7
8
fscsi2/path7
fscsi5/path8
OPEN NORMAL
OPEN NORMAL
493451
492225
0
0
5005076801 9* fscsi5/path9 OPEN NORMAL 197 0
10* fscsi5/path10 OPEN NORMAL 197 0
11 fscsi5/path11 OPEN NORMAL 492660 0
port1=10B374 port1=10B363 port1=10B371 port1=10B335 12 fscsi7/path12 OPEN NORMAL 491988 0
port2=20B374 port2=20B363 port2=20B371 port2=20B335 13* fscsi7/path13 OPEN NORMAL 197 0
port3=30B374 port3=30B363 port3=30B371 port3=30B335 14* fscsi7/path14 OPEN NORMAL 197 0
port4=40B374 port4=40B363 port4=40B371 port4=40B335 15 fscsi7/path15 OPEN NORMAL 492943 0

© Copyright IBM Corporation 2014


Wrong - Causes Over Subscription
fscsi5=10000000C9C0DC7F
fscsi0=10000000C9C0A984 fscsi7=10000000C9C0B3DB
fscsi2=10000000C9C0E0E0
Zone for p770_1_vio1a_d3_SVC Zone for p770_1_vio1a_d4_SVC
Zone for p770_1_vio1a_d1 Zone for p770_1_vio1a_d2
b03vio101
NRPOKVIO1A 10000000C9C0DC7F 10000000C9C0B3DB
10000000c9779a4a 10000000C9C0E0E0
500507680120B374 500507680120B374
500507680110B374 500507680110B374 d1 d4
d2 d3 500507680140B374 500507680140B374
500507680130B374 500507680130B374 500507680120B363
500507680120B363
500507680110B363 500507680110B363 500507680140B363
500507680140B363
500507680130B363 500507680130B363 500507680120B371
500507680120B371
500507680110B371 500507680110B371 500507680140B371
500507680140B371
500507680130B371 500507680130B371
500507680120B335 500507680120B335
500507680110B335 500507680110B335 500507680140B335
500507680140B335
500507680130B335 500507680130B335

DIR1 SAN Fabric DIR2 SAN Fabric

Over subscribed SVC to Host


HBA Zoning causing to many
datapaths
DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: Load
Balance
SERIAL: 600507680181059BA000000000000005
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
==========================================================
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 SVC Host DefinitionsPath# Adapter/Path Name State Mode Select Errors
id:1 0 fscsi0/path0 OPEN NORMAL 558254 0
Node 1 Node 2 Node 3 Node 4 name:P770_1_vio1A
1*
2*
fscsi0/path1
fscsi0/path2
OPEN NORMAL
OPEN NORMAL
197
197
0
0
10000000C9C0B3DB
iogrp 0 iogrp 1 10000000C9C0DC7F
3
4
fscsi0/path3
fscsi2/path4
OPEN NORMAL
OPEN NORMAL
493559
493330
0
0
2048 LUNs max 2048 LUNs max 10000000C9C0E0E0 5* fscsi2/path5 OPEN NORMAL 197 0
10000000C9C0A984 6* fscsi2/path6 OPEN NORMAL 197 0
wwpn 4 Node SVC MAX Vdisk 4096 CF8
7
8
fscsi2/path7
fscsi5/path8
OPEN NORMAL
OPEN NORMAL
493451
492225
0
0
5005076801 9* fscsi5/path9 OPEN NORMAL 197 0
10* fscsi5/path10 OPEN NORMAL 197 0
11 fscsi5/path11 OPEN NORMAL 492660 0
port1=10B374 port1=10B363 port1=10B371 port1=10B335 12 fscsi7/path12 OPEN NORMAL 491988 0
port2=20B374 port2=20B363 port2=20B371 port2=20B335 13* fscsi7/path13 OPEN NORMAL 197 0
port3=30B374 port3=30B363 port3=30B371 port3=30B335 14* fscsi7/path14 OPEN NORMAL 197 0
port4=40B374 port4=40B363 port4=40B371 port4=40B335 15 fscsi7/path15 OPEN NORMAL 492943 0

© Copyright IBM Corporation 2014


Right or Wrong?

© Copyright IBM Corporation 2014


Right or Wrong?

© Copyright IBM Corporation 2014


Right or Wrong?

© Copyright IBM Corporation 2014


Wrong, Shows as Degraded Paths

© Copyright IBM Corporation 2014


What’s wrong?
Fabric_A Zone Definitions
Fabric_B Zone Definitions
Zone 101_a Zone_101_d
b03vio101
b03vio101 Zone 101_c
A1=10000000c97549bc B2=10000000c9771aae
B1=10000000c9771aaf
P1=50050768011052ca P1=50050768011052ca
P1=50050768014052ca
P1=50050768011052b7 P1=50050768011052b7 A1 A2 A3 B1
P1=50050768014052b7
P1=500507680110529e P1=500507680110529e
P1=500507680140529e
P1=500507680110528b P1=500507680110528b
P1=500507680140528b
Zone_101_b
A2=10000000c97549bd
P1=50050768013052ca
P1=50050768013052b7
P1=500507680130529e
P1=500507680130528b

DIR1 SAN Fabric DIR2 SAN Fabric

Port to Fabrics
not evenly distributed
SVC Host Definitions
id:2 In this example we show
name:b03vio100_apvg
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 10000000C97549BD 2 SVC Host definitions, for 1 host,
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 10000000C97549BC with uneven port to fabric distribution
Node 1 Node 2 and 4 zones
Node 3 Node 4 id:3
name:b03vio100_dbvg As you can see...A1 will failover to A2,
iogrp 0 iogrp 1 10000000C9771AAF on the same fabric..this is a Single point
2048 LUNs max 10000000C9771AAE
2048 LUNs max of Falure (SPoF)
wwpn 4 Node SVC MAX Vdisk 4096
5005076801 See the next page for correct zoning
port1=11052ca port1=11052b7 port1=110529e port1=110528b where there are multiple Host HBAs,
port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b
zoned to the SVC
port4=14052ca port4=14052b7 port4=140529e port4=140528b

© Copyright IBM Corporation 2014


Wrong
Fabric_A Zone Definitions
Fabric_B Zone Definitions
Zone 101_a Zone_101_d
b03vio101
b03vio101 Zone 101_c
A1=10000000c97549bc B2=10000000c9771aae
B1=10000000c9771aaf
P1=50050768011052ca P1=50050768011052ca
P1=50050768014052ca
P1=50050768011052b7 P1=50050768011052b7 A1 A2 A3 B1
P1=50050768014052b7
P1=500507680110529e P1=500507680110529e
P1=500507680140529e
P1=500507680110528b P1=500507680110528b
P1=500507680140528b
Zone_101_b
A2=10000000c97549bd
P1=50050768013052ca
P1=50050768013052b7
P1=500507680130529e
P1=500507680130528b

DIR1 SAN Fabric DIR2 SAN Fabric

Port to Fabrics
not evenly distributed
SVC Host Definitions
id:2 In this example we show
name:b03vio100_apvg
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 10000000C97549BD 2 SVC Host definitions, for 1 host,
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 10000000C97549BC with uneven port to fabric distribution
Node 1 Node 2 and 4 zones
Node 3 Node 4 id:3
name:b03vio100_dbvg As you can see...A1 will failover to A2,
iogrp 0 iogrp 1 10000000C9771AAF on the same fabric..this is a Single point
2048 LUNs max 10000000C9771AAE
2048 LUNs max of Falure (SPoF)
wwpn 4 Node SVC MAX Vdisk 4096
5005076801 See the next page for correct zoning
port1=11052ca port1=11052b7 port1=110529e port1=110528b where there are multiple Host HBAs,
port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b
zoned to the SVC
port4=14052ca port4=14052b7 port4=140529e port4=140528b

© Copyright IBM Corporation 2014


Right
Fabric_A Zone Definitions Fabric_B Zone Definitions
Zone 101_a Zone 101_a
A1=10000000c97549bc B1=10000000c9771aaf
b03vio101_a b03vio101_b
P1=50050768011052ca P1=50050768014052ca
P1=50050768011052b7 P1=50050768014052b7
P1=500507680110529e A1 B1 A2 B2 P1=500507680140529e
P1=500507680110528b P1=500507680140528b

Zone_101_b Zone_101_b
A2=10000000c97549bd B2=10000000c9771aae
P1=50050768013052ca P1=50050768012052ca
P1=50050768013052b7 P1=50050768012052b7
P1=500507680130529e P1=500507680120529e
P1=500507680130528b P1=500507680120528b

DIR1 SAN Fabric DIR2 SAN Fabric

The host definitions in the SVC


should be defined as 2 hosts
and 4 zones in the Fabrics

In this example we show


P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2
SVC Host Definitions 2 seperate SVC Host definitions and
id:2 2 Fabric zones, per host per Fabric
name:b03vio101_a
Node 1 Node 2 Node 3 Node 4 10000000C97549BC Totaling 4 zones
iogrp 0 iogrp 1 10000000C9771AAF

2048 LUNs max 2048 LUNs max id:3


This allows proper failover from Primary
name:b03vio101_b to Alternate nodes within the SVC iogrp.
wwpn 4 Node SVC MAX Vdisk 4096 10000000C97549BD
500507680 10000000C9771AAE

port1=11052ca port1=11052b7 port1=110529e port1=110528b


port2=12052ca port2=12052b7 port2=120529e port2=120528b
port3=13052ca port3=13052b7 port3=130529e port3=130528b
port4=14052ca port4=14052b7 port4=140529e port4=140528b

© Copyright IBM Corporation 2014


Right or Wrong?

Fabric_A Zone Definitions Fabric_B Zone Definitions


A1=10:00:00:00:c9:3f:75:37 b03vio210 B1=10:00:00:00:c9:3f:74:16
A2=10:00:00:00:c9:3f:80:54 A1 A2 A3 A4 B1 B2 B3 B4 B2=10:00:00:00:c9:3f:f7:2f
A3=10:00:00:00:c9:42:89:ac B3=10:00:00:00:c9:42:b4:4b
A4=10:00:00:00:c9:42:80:16 B4=10:00:00:00:c9:42:b6:18

DIR1 SAN Fabric DIR2 SAN Fabric

In this example we show


SVC Host Definitions 1 SVC Host definitions and
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 id:8
name:b03vio210_allvg
2 Fabric zones, per host per Fabric.
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2
10000000C942B618
Node 1 Node 2 Node 3 Node 4 10000000C942B44B This does not allow proper failover from
10000000C9428016
iogrp 0 iogrp 1 10000000C94289AC Primary to Alternate nodes within the
2048 LUNs max 2048 LUNs max
10000000C93FF72F SVC iogrp.
10000000C93F7416
wwpn 4 Node SVC MAX Vdisk 4096
10000000C93F8054
10000000C93F7537 See the next page for proper zoning
5005076801

© Copyright IBM Corporation 2014


Right

b03vio210_1 b03vio210_2 b03vio210_3 b03vio210_4

A1 B1 A2 B2 A3 B3 A4 B4
Fabric_A Zone Definitions Fabric_A Zone Definitions
Fabric_B Zone Definitions A4=10:00:00:00:c9:42:80:16
Fabric_B Zone Definitions A3=10:00:00:00:c9:42:89:ac
B1=10:00:00:00:c9:3f:74:16
Fabric_A Zone Definitions B2=10:00:00:00:c9:3f:f7:2f
A1=10:00:00:00:c9:3f:75:37
Fabric_B Zone Definitions
Fabric_B Zone Definitions
Fabric_A Zone Definitions B4=10:00:00:00:c9:42:b6:18
B3=10:00:00:00:c9:42:b4:4b
A2=10:00:00:00:c9:3f:80:54

DIR1 SAN Fabric DIR2 SAN Fabric

SVC Host Definitions


id:1 name:b03vio210_1 In this example we show
10000000C93F7537 4 seperate SVC Host definitions and
P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 P1 P2 P3 P4 10000000C9428016
2 seperate Fabric zones, per host
HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2 HBA 1 HBA 2
id:2 name:b03vio210_2 per Fabric
10000000C93F8054
Node 1 Node 2 Node 3 Node 4 10000000C93FF72F
This does allow proper failover from
iogrp 0 iogrp 1 id:3 name:b03vio210_3 Primary to Alternate nodes within the
2048 LUNs max 2048 LUNs max 10000000C94289AC
10000000C942B44B SVC iogrp.
wwpn 4 Node SVC MAX Vdisk 4096
5005076801 id:4 name:b03vio210_4
10000000C93F7416
10000000C942B618

© Copyright IBM Corporation 2014


Session summary
• You should know:
– Zoning is one of the most important configuration to proper:
• Failover
• Redundancy
• Resiliency
• Performance
• Understand best practice zoning for 4 port SVC models
• Understand best practice zoning for 8 port SVC models – CG8
• Understand best practice zoning for 12 port SVC models –DH8
• Understand best practice zoning for Dual Core Fabric to SVC
• Hopefully enjoyed learning Right and Wrong Zoning Scenarios
by playing along
• Had “Fun”

© Copyright IBM Corporation 2014


Thank you!
For you interest
and attendance

© Copyright IBM Corporation 2014


SVC Zoning Evolution

IBM SVC Zoning - Best Practices 101

Backup Slides

© Copyright IBM Corporation 2014 86


Agenda
• SVC Zoning, Volume Pathing and Multipathing testing
• Most common questions
• Understanding the physical and logical
• General SVC Best Practice Zoning Concepts
• SVC Zone Concepts
– Zoning evolution with SVC Model evolution
– Storage to SVC Zoning
– Host HBA to SVC zoning (preferred/non preferred pathing)
• Dual Host HBAs
• Multiple Host HBAs
• Host HBA multipathing settings and testing
• Zone Types for 8 port Nodes
– MM/GM and heartbeat zones for 8 port nodes
• SVC Cluster zoning for Dual Core Fabrics examples
– Cluster Zone 4 port vs. 8 port nodes
© Copyright IBM Corporation 2014
Most common zoning questions - BP
recommendations
• What is the max number of WWNN/WWPNs devices?
– Always check the Max Limit configuration URL for the most current
updates
• Search for the term "configuration limits and restrictions" in the Search
support and downloads search box at the following website:
– www.ibm.com/storage/support/2145
– Currently 1024 WWNN per cluster and 1024 WWPNs per cluster
• Recommendation - The more wwpns per wwnn, the more throughput up
to 16
• How many IO connections/zones per storage device to the
SVC should I zone?
– 16 from any “one” storage device unit zoned with all SVC node ports
• All SVC nodes must see same set of LUNs from disk controller
– Otherwise degraded mode on controller and/or MDisks

© Copyright IBM Corporation 2014


Most common zoning questions - BP
recommendations
• Should I mask host connections through zoning or SVC
configurations?
– Best practice is to use the default value of …..111111111111 (all ports
enabled) and control masking through zoning
– Use host type Target Port Group (tpgs) for Solaris hosts, (hpux) for
HP/UX, (openvms) for HP AlphaServer/HP Intergrity and (generic) for
everything else
– Separate disk and tape IO on host HBAs
• What works better, 4 or 8 paths per Vdisk?
– Recommendation is 4 paths per Vdisk

© Copyright IBM Corporation 2014


Most common zoning questions - BP
recommendations
• How many Iogrps should I map to a host? 4?
– Recommendation is to size per throughput and number of hosts per
cluster
– May need to isolate/sement host activity/chatter from one Iogrp to
another
– For more horsepower – spread IO across more LUNs from more
Iogrps
• Can a host have only one connection to the SVC ?
– Dual host HBA connections

© Copyright IBM Corporation 2014


Three (3) types of Port Masking
• There are three(3) types of Port masking to be aware of with the new code and DH8
• Host port masking
– Default port masking, with 32 bits, used to be "1111" now it is "...111111111111". reading from
right to left...1 for each of the 12 ports. Leaving the default of 1 enables the flow of IO to pass
through the node port, "0" means to block IO on the port.
• Benefits
– zoning changes
– Less complexity for Vdisk IOgrp moves,
– Changes introduced in the SAN, or the hosts, with less complexity
• The MM/GM port mask
– Allows only MM/GM replication traffic to flow through the enable ports,
• Note: Other traffic can flow here from other connected devices, but defeats the purpose of dedicating
MM/GM traffic . We dedicate and use port 5&6 for MM/GM (partner_fc_port_mask) so the mask would be
000000110000
• The Intra-cluster communication port mask
– Also dedicated at 0000110000 for 12 ports meaning ports 7 & 8 are enabled with 1's and the rest
are blocked.
– Bottom-line: We have so much field experience behind these best practices, that this is what we recommend. The most
common reoccurring issue is when LUNs are moved around between IOgrps or shared to clusters, the zoning gets
redone, but most SME's forget about or don't understand the port mask and so don't change it and then the redundancy is
lost when change is made, resulting in IO blocked and outages...this we made it a best practice to reduce the complexity
of managing the LUN to host mapping.

© Copyright IBM Corporation 2014


Disk Controller Best Practices
• Maximum of 1024 WWNNs
– EMC DMX/SYMM, All HDS and SUN/HP HDS clones use one WWNN
per port; each appears as a separate controller to SVC
• Map LUNs through up to 16 FA ports
– Results in 16 WWNNs/WWPNs used out of the max of 1024
– IBM, EMC Clariion, HP, etc. use one WWNN per subsystem; each
appears as a single controller with multiple ports/WWPNs
• Maximum of 16 ports/WWPNs per WWNN using 1 out of the max of
1024
• Greater Qdepth is archived with fewer Mdisks

© Copyright IBM Corporation 2014 92


Disk Controller Best Practices
• DS4K/5K – EMC Clariion/CX
– Both have preferred controller architecture
• SVC honors this configuration
– Use minimum of 4 and preferably 8 ports or more up to maximum of
16
– More ports equate to more concurrent I/O driven by SVC
– Support for mapping controller A ports to Fabric A and controller B
ports to Fabric B or cross connecting ports to both fabrics from both
controllers
• IBM® DS4000® does not support a storage system A and storage
system B in the same zone
• Later is preferred to avoid AVT/Trespass occurring if a fabric or all
paths to a fabric fail
– SVC supports SVC queue depth change for CX models
• Drives more I/O per port per Mdisk

93
Storage Zones - General SVC Best
Practice Zoning Concepts
• Create two cluster zones (do not include MM/GM and intra-
cluster traffic ports)
• Never span zones to include more than one Backend storage
device!
• Never put Host OS ports, SVC ports and Backend Storage
ports together in the same zone
– Instead
• Create zones with Host ports and SVC ports
• Create zones with Backend and SVC ports
– Never use the same DS8K ports or any native back-end port for
connectivity to SVC and an attached host
• If SVC is attached to the DS8K or other native back-end devices and the DS8K
or other back-end device is using native GM (not SVC GM) then dedicate
appropriate back-end ports specifically for GM, not to be used for attaching any
other device, whether Host Server, SVC or other connectivity relationships.

© Copyright IBM Corporation 2014


Host Zones
Best Practices Concepts
• Too many paths to a Vdisk
– If the recommended number of paths to a Vdisk are exceeded, path
failures may not be recovered in the required amount of time
• Causes excessive I/O waits, resulting in application failures
• Under certain circumstances, it can reduce performance
– Note: 8 paths are supported but 4 are optimum for SDD/SDDDSM/SDDPCM
• SVC host zones
– There must be a single zone for each host port. This zone must
contain the host port, and one port from each SVC node that the host
will need to access. While there are two ports from each node per
SAN fabric in a usual dual-fabric configuration, make sure that the
host only accesses one of them.
• Check with the Host System Administrators to receive output from the SDD or
PCM path query device commands on each host to verify proper pathing
between the SVC nodes and host systems
– Note: It is a supported configuration to have eight paths to each VDisk, but this design
provides no performance benefit, and it does not improve reliability or availability by any
significant degree
© Copyright IBM Corporation 2014
Host Zones
SVC Preferred Node Scheme
• Hosts with four (or more) Host Bus Adapters (HBAs)
– Takes a little more planning.
– Because eight paths are not an optimum number, you must instead
configure your SVC Host Definitions (and zoning) as though the single
host is two or more separate hosts
– During Vdisk assignment, alternate which Vdisk is assigned to one of
the “pseudo-hosts”, in a round robin fashion (a pseudo-host is nothing
more than another regular host definition in the SVC host config. Each pseudo-
host will contain 2 unique host WWPNs, 1 WWPN mapped to each fabric)
– Note: A pseudo-host, is not a defined function or feature of the SVC. If you
need to define a pseudo-host, you are simply adding another host id to the
SVC host config. Instead of creating one host id with 4 WWPNs, you would
define 2 hosts with 2 WWPNs. This is now the reference for the term pseudo-
host.
– Note: Be careful not to share the Vdisk to more than two adapters per host,
so as to not oversubscribe the number of datapaths per vdisk per host

© Copyright IBM Corporation 2014


General SVC Best Practice Zoning
Concepts
• Dual Core Fabrics require a breakout in cluster zoning to
separate SVC node ports on core1 from core2
– Prevents/Minimizes IO from spanning/routing across ISLs and Edge
switches in a Core/Edge design
• SVC node ports plugged into dir1 core1 should reside in a separate zone
from SVC node ports plugged into dir1 core2.
• For MM/GM zones, take advantage of spreading the IO across both
cores through careful zoning
• Ensure the Intra-cluster/heartbeat zone is also isolated properly between
each fabrics dual cores (don’t span heartbeat across cores in one fabric)
– Note: - Never make zoning changes on redundant Fabrics at the same time
• Make changes on one fabric and wait 30 min in-between
• Please see the following link for zoning BP’s
– Cisco:
http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=5EB2DDF9-7568-368A-7F99-1BCFFF85E472
– Brocade:
http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=57A9649D-A04B-5931-05DA-F389F697D4D3

© Copyright IBM Corporation 2014


New Build- Positioning for HBA
redundancy
Logical port #’s reflecting the embedded WWPN

Blue ports go to one Fabric


Red Ports go to the other Fabric

Logical port with wwpn # embedded

Physical port number

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


New Build - Physical to Logical Port
numbering for 12 port nodes
• When creating a new cluster on DH8 etc then you will get a slot and port encoding
for all cards with all ports logically left to right.
• Logical port #’s reflecting the embedded WWPN

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


New Build - Physical to Logical Port
numbering
• Logical port #’s reflecting the embedded WWPN
• When Write Data Rate > 3GB/sec per IOgrp

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


New Build - Physical to Logical Port
numbering for 12 port nodes

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


New Build Numbering – When Write Data
Rate > 3GB/sec per IOgrp

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


New CG8 Dual Core Zones
• Make the local_fc_port_mask (Node to Node/heartbeat) = 10001000
• Make the partner_fc_port_mask (MM/GM = 01000100

© Copyright IBM Corporation 2014 Port Layout provided Keith Williams


Correct way to make MM/GM Zone,
new implementation 7.1.x
 Make the local_fc_port_mask (Node to Node/heartbeat) = 000010001000
 Make the partner_fc_port_mask (MM/GM = 000001000100

104

Вам также может понравиться