10 Map Multipath SAN Disks to Hardware Path

Solaris Multipathed SAN Disks or any disks for that matter are hidden behind drivers. Those drivers hide the
real physical hardware paths to which they are attached. This is normal and expected behaviour for
multipathed storage. Its the same on HP-UX, AIX, all the big players. As administrators, there are times when
you need to find the physical paths of devices. As an example, you are about to implement Solaris or Oracle
VM Server or Sun LDoms. They name keeps changing You need the physical PCI hardware paths within
Solaris to divide the PCI buses up when creating a separate I/O domain, as an example. The below commands
apply to Solaris 10 and a Sun or Oracle T4-4 server.
Lets look at the disks. We have some HDS storage here, multipathed with the native MPxIO, enabled with the
stmsboot -e command.
# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t5000CCA02532E80Cd0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>
/scsi_vhci/disk@g5000cca02532e80c
1. c0t5000CCA02532E794d0 <SUN300G cyl 46873 alt 2 hd 20 sec
625> solaris
/scsi_vhci/disk@g5000cca02532e794
2. c0t5000CCA02533AC20d0 <SUN300G cyl 46873 alt 2 hd 20 sec
625> solaris
/scsi_vhci/disk@g5000cca02533ac20
3. c0t5000CCA02533ACC8d0 <SUN300G cyl 46873 alt 2 hd 20 sec
625> solaris
/scsi_vhci/disk@g5000cca02533acc8
4. c0t60060E8006FF03000000FF030000150Ad0 <HITACHI-OPEN-V -SUN-
7005-300.00GB>
/scsi_vhci/ssd@g60060e8006ff03000000ff030000150a
5. c0t60060E8006FF03000000FF0300001500d0 <HITACHI-OPEN-V -SUN-
7005-100.00GB>
/scsi_vhci/ssd@g60060e8006ff03000000ff0300001500
6. c0t60060E8006FF03000000FF0300001501d0 <HITACHI-OPEN-V -SUN-
7005-100.00GB>
/scsi_vhci/ssd@g60060e8006ff03000000ff0300001501
7. c0t60060E8006FF03000000FF0300001503d0 <HITACHI-OPEN-V -SUN-
7005-300.00GB>
/scsi_vhci/ssd@g60060e8006ff03000000ff0300001503
8. c0t60060E8006FF03000000FF0300001504d0 <HITACHI-OPEN-V -SUN-
7005-300.00GB>
/scsi_vhci/ssd@g60060e8006ff03000000ff0300001504
We follow the block device and feed that to the mpathadm show command.
# ls -l /dev/dsk/c0t60060E8006FF03000000FF030000150Ad0s0
lrwxrwxrwx 1 root root 63 Nov 29 10:48
/dev/dsk/c0t60060E8006FF03000000FF030000150Ad0s0 ->
../../devices/scsi_vhci/ssd@g60060e8006ff03000000ff030000150a:a
# mpathadm show lu /dev/dsk/c0t60060E8006FF03000000FF030000150Ad0s0
Logical Unit: /dev/rdsk/c0t60060E8006FF03000000FF030000150Ad0s2
mpath-support: libmpscsi_vhci.so
Vendor: HITACHI
Product: OPEN-V -SUN
Revision: 7005
Name Type: unknown type
Name: 60060e8006ff03000000ff030000150a
Asymmetric: no
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: 21000024ff4794b4
Target Port Name: 50060e8006ff0305
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 21000024ff47915a
Target Port Name: 50060e8006ff0305
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 21000024ff4794b5
Target Port Name: 50060e8006ff0315
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 21000024ff47915b
Target Port Name: 50060e8006ff0315
Override Path: NA
Path State: OK
Disabled: no
Target Ports:
Name: 50060e8006ff0305
Relative ID: 0
Name: 50060e8006ff0315
Relative ID: 0
Sorry, the formatting has been lost here. But the important thing here is that we see 4 paths. Lets see what
the Initiator Port Name: points to, that will be the hardware path, down to the actual PCI bus. All hardware is
different and have multiple PCI bus numbers. This is a T4, and has only pci_0 and pci_1.
# mpathadm show initiator-port 21000024ff4794b4
Initiator Port: 21000024ff4794b4
Transport Type: Fibre Channel
OS Device
File: /devices/pci@400/pci@2/pci@0/pci@2/pci@0/pci@2/SUNW,qlc@0/fp@0,0
# mpathadm show initiator-port 21000024ff47915a
Initiator Port: 21000024ff47915a
Transport Type: Fibre Channel
OS Device
File: /devices/pci@500/pci@1/pci@0/pci@1/pci@0/pci@2/SUNW,qlc@0/fp@0,0
# mpathadm show initiator-port 21000024ff4794b5
Initiator Port: 21000024ff4794b5
Transport Type: Fibre Channel
OS Device
File: /devices/pci@400/pci@2/pci@0/pci@2/pci@0/pci@2/SUNW,qlc@0,1/fp@0,0
# mpathadm show initiator-port 21000024ff47915b
Initiator Port: 21000024ff47915b
Transport Type: Fibre Channel
OS Device
File: /devices/pci@500/pci@1/pci@0/pci@1/pci@0/pci@2/SUNW,qlc@0,1/fp@0,0
So the logic based on the T4 main board bus layout diagram is as follows:
/devices/pci@500 maps to PCI_1
/devices/pci@400 maps to PCI_0


t4-motherboard
Serious NetApp Configuration References & Tricks
Also see: LDAP Tech Tips Puppet Master Config Tips
Your comments are welcomed

netapp
Netapp Qtree CIFS and Unix Configuration Examples
Book Review Cutting For Stone by Abraham Verghese
2 Responses to Solaris 10 Map Multipath SAN Disks to Hardware
Path
1.
Ahmed December 4, 2013 at 5:06 am #
Hi
Thanks for the nice article, it was really helpful for me.but i am confused that how to figure out that
which phyiscal hardware path maps to which pci?
as per your example output
/devices/pci@400/pci@2/pci@0/pci@2/pci@0/pci@2/SUNW,qlc@0/fp@0,0
/devices/pci@500/pci@1/pci@0/pci@1/pci@0/pci@2/SUNW,qlc@0/fp@0,0
/devices/pci@500 maps to PCI_1
/devices/pci@400 maps to PCI_0
but my understanding is pci@400 maps to PCI_2 and you mentioned its mapping on PCI_0 ? can you
please elaborate and confirm this?
Thanks
Reply
2.
rocker December 5, 2013 at 7:21 am #
Hi Ahmed
I have to admit that I rushed this article taken from my build notes. Have a look at the newly added
diagram above and correlate that with the link below. PCIe can be interchangeable used with PCI. Its
the logical PCI controller 0 or 1 based on bus address. Be advised that this link is architecture specific,
as in its for the T4.
http://docs.oracle.com/cd/E23411_01/pdf/E23415.pdf
I hope this helps rationalize. I show how to list PCI buses on my LDOM split IO domain article. See
http://seriousbirder.com/blogs/solaris-ldom-split-io-domain-configuration-example/ I show how to
list PCI buses. I dont know if you are setting up LDOMs or not.
I hope this makes more sense. Im on a Linux contract right now and dont have access to any more
Sun gear.
Cheers
Reply