Вы находитесь на странице: 1из 53
IBM GLOBAL SERVICES IMT France / ITD La Gaude Pseries Department VIO server Installation Configuration

IBM GLOBAL SERVICES

IMT France / ITD La Gaude

Pseries Department

VIO server

Installation

Configuration

Standardization

Author : Alain Lentini

© IBM France ITD La Gaude Unix team 2010

Author : Alain Lentini

Changes history

Document Management

Version

Date

author

 

Changes

1.0

May 2008

A.Lentini

Initial version – DRAFT

2.0

February 2010

A.Lentini

- all types of SEA detailed with figures and build steps

- command line mode for all the SEA configuration

- rules and naming convention for SEA build

- SAN disks config detailed with 3 typical configuration

- command line mode for all the SAN disk configuratio

- rules and naming convention for SAN build

- design and build recommendations for ITD/SSO lge

Reviewers list

Name

Date

 

Ling Wai Hung

February 2010

Stéphane Saleur

February 2010

Guy Aluno

February 2010

Sylvain Desnoes

February 2010

Nicolas Doz

February 2010

© IBM France ITD La Gaude Unix team 2010

- 2 -

1.

INTRODUCTION

7

2. STANDARD DESIGN EXTRACTS

7

3. INSTALL OS VIO

9

3.1. DOWNLOAD FIXPACK AND VIOS

9

3.2. UPGRADE VIOS FIXPACK

9

3.3. VIOS BACKUP/RESTORE

9

4. SHARED ETHERNET ADAPTER CONFIGURATION

9

4.1.

BASIC SEA WITHOUT VLAN TAGGING

9

4.1.1. BASIC SEA LOGICAL VIEW

9

4.1.2. CREATE SEA ON VIO1

10

4.1.2.1.

Create virtual Ethernet adapter on VIO1

10

4.1.2.1.1.

command line mode

11

4.1.2.2.

create control channel on VIO1

11

4.1.2.2.1. command line mode

12

4.1.2.2.2. rules and naming convention

12

4.1.2.3.

sea creation

13

4.1.3.

CREATE SEA ON VIO2

13

4.1.3.1.

Create virtual Ethernet adapter on VIO2

13

4.1.3.1.1.

command line mode

14

4.1.3.2.

create control channel on VIO2

14

4.1.3.2.1.

command line mode

15

4.1.3.3.

sea creation

15

4.1.4.

CLIENT PARTITION CONFIGURATION

15

4.1.4.1. Create virtual Ethernet adapter on client partition

16

4.1.4.2. IP configuration on client partition

16

4.2. ETHERCHANNEL

16

4.3. SEA WITH VLAN TAGGING ON THE SAME DEVICE

16

4.3.1. IMPORTANT : AVOID THIS SOLUTION FOR OUR ARCHITECTURES

16

4.3.2. SEA WITH VLAN TAGGING ON SAME INTERFACE LOGICAL VIEW

16

4.3.3. CREATE SEA ON VIO1

17

4.3.4. CREATE ETHERCHANNEL ON VIO1

17

4.3.4.1.

Create virtual Ethernet adapter on VIO1

17

4.3.4.1.1.

command line mode

18

4.3.4.2.

create control channel on VIO1

18

4.3.4.2.1.

rules and naming convention

19

4.3.4.3.

sea creation on VIO1

20

4.3.5.

CREATE SEA ON VIO2

20

© IBM France ITD La Gaude Unix team 2010

- 3 -

4.3.6.

CREATE ETHERCHANNEL ON VIO2

20

4.3.6.1.

Create virtual Ethernet adapter on VIO2

20

4.3.6.1.1.

command line mode

21

4.3.6.2. create control channel on VIO2

21

4.3.6.3. sea creation on VIO2

22

4.3.7.

CLIENT PARTITION CONFIGURATION

22

4.3.7.1. Create virtual Ethernet adapter on client partition

22

4.3.7.2. IP configuration on client partition

23

4.3.8.

ADD A NEW VLAN

23

4.4.

SEA WITH VLAN TAGGING – RECOMMENDED SOLUTION

24

4.4.1. SEA WITH VLAN TAGGING LOGICAL VIEW

24

4.4.2. WHY THIS SOLUTION IS RECOMMENDED

25

4.4.3. CREATE SEA ON VIO1

25

4.4.4. CREATE ETHERCHANNEL ON VIO1

25

4.4.4.1.

Create virtual Ethernet adapters on VIO1

25

4.4.4.1.1.

command line mode

26

4.4.4.2.

create control channel on VIO1

27

4.4.4.2.1. command line mode

27

4.4.4.2.2. rules and naming convention

28

4.4.4.3.

sea creation on VIO1

28

4.4.5. CREATE SEA ON VIO2

28

4.4.6. CREATE ETHERCHANNEL ON VIO2

28

4.4.6.1.

Create virtual Ethernet adapter on VIO2

28

4.4.6.1.1.

command line mode

29

4.4.6.2.

create control channel on VIO2

30

4.4.6.2.1.

command line mode

30

4.4.6.3.

sea creation on VIO2

31

4.4.7.

CLIENT PARTITION CONFIGURATION

31

4.4.7.1. Create virtual Ethernet adapter on client partition

31

4.4.7.2. IP configuration on client partition

32

4.4.8.

ADD A NEW VLAN

32

4.4.8.1.

Create virtual Ethernet adapters on VIO1

32

4.4.8.1.1.

command line mode

33

4.4.8.2. Update SEA on VIO1

34

4.4.8.3. Create virtual Ethernet adapters on VIO2

34

4.4.8.3.1.

command line mode

35

4.4.8.4. Update SEA on VIO2

35

4.4.8.5. Client partition configuration

36

4.4.8.5.1. Create virtual Ethernet on client partition

36

4.4.8.5.2. IP configuration on client partition

36

5. SAN DISKS CONFIGURATION

5.1. RULES AND NAMING CONVENTION

© IBM France ITD La Gaude Unix team 2010

- 4 -

37

37

5.1.1.

VTD NAMING CONVENTION

37

5.1.2.

SLOT NUMBERS AND VSCSI SERVER

37

5.1.2.1. rules with LPM

37

5.1.2.2. Rules without lpm

38

5.2. REDBOOK REFERENCES FOR RULES AND NAMING CONVENTION

38

5.3. VIRTUAL DEVICES ADAPTER VS QUEUE_DEPTH

38

5.4. IMPLEMENTATION IN TYPICAL CONFIGURATIONS

39

5.4.1. CONFIGURATION 1

39

5.4.2. CONFIGURATION 2

40

5.4.3. CONFIGURATION 3

40

5.5. FIBER CHANNEL AND HDISK DEVICES SETTINGS

41

5.6. CONFIGURATION 2 DETAILED IMPLEMENTATION

41

5.6.1.

CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE B4

41

5.6.1.1.

command line mode

42

5.6.2.

CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE B4

42

5.6.2.1.

command line mode

43

5.6.3.

CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE B4

43

5.6.3.1.

command line mode

44

5.6.4.

CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE B4

44

5.6.4.1.

command line mode

44

5.6.5.

CREATE VSCSI SERVER ADAPTER ON VIO1 FOR STORAGE G1

44

5.6.5.1.

command line mode

45

5.6.6.

CREATE VSCSI SERVER ADAPTER ON VIO2 FOR STORAGE G1

45

5.6.6.1.

command line mode

46

5.6.7.

CREATE CLIENT PARTITION DEVICE FOR VIO1 AND STORAGE G1

46

5.6.7.1.

command line mode

47

5.6.8.

CREATE CLIENT PARTITION DEVICE FOR VIO2 AND STORAGE G1

47

5.6.8.1.

command line mode

47

5.6.9. CREATE VIRTUAL DEVICES ON VIO1

47

5.6.10. CREATE VIRTUAL DEVICES ON VIO2

48

5.6.11. CLIENT PARTITION DEVICES SETTING

48

5.6.11.1. priority

48

5.6.11.2. hcheck_interval

49

5.6.11.3. queue depth

49

5.6.11.4. vscsi_path_to

49

6. SCSI DISK DEVICES CONFIGURATION FOR ROOTVG CLIENT PARTITIONS

© IBM France ITD La Gaude Unix team 2010

- 5 -

49

6.1.

CREATE THE LOGICAL VOLUMES AND THE VIRTUAL DEVICES

49

7. NTP CONFIGURATION ON VIO AND HMC

52

7.1. NTP CONFIGURATION ON VIO SERVERS

52

7.2. NTP CONFIGURATION ON HMC

52

© IBM France ITD La Gaude Unix team 2010

- 6 -

1.

Introduction

In this document, we assume that we must redundant VIO servers for each type of I/O. This means we can have configuration with 2 VIO servers for LAN and 2 others VIO servers for SAN, or we can have configuration with 2 VIO servers for LAN and SAN. You can use the document to adapt to other types of configurations, but you manage the risks.

2. Standard design extracts

Here are architecture designs recommended in this document.

Here are architecture designs recommended in this document. Basic SEA without vlan tagging © IBM France

Basic SEA without vlan tagging

© IBM France ITD La Gaude Unix team 2010

- 7 -

SEA with vlan tagging – recommended solution Vscsi configuration 1 : 2 SAN VIO -

SEA with vlan tagging – recommended solution

SEA with vlan tagging – recommended solution Vscsi configuration 1 : 2 SAN VIO - 1

Vscsi configuration 1 : 2 SAN VIO - 1 storage subsystem - 1 vscsi server per VIO server

© IBM France ITD La Gaude Unix team 2010

- 8 -

Vscsi configuration 2 : 2 SAN VIO - 2 storage subsystems - 2 vscsi server

Vscsi configuration 2 : 2 SAN VIO - 2 storage subsystems - 2 vscsi server per VIO

3. install OS Vio

3.1. download fixpack and vios

http://www14.software.ibm.com/webapp/set2/sas/f/vios/home.html

3.2. upgrade vios fixpack

updateios -dev /mnt/lpp_source/vios14_fixpack92_lpp -install –accept updateios -dev /mnt/lpp_source/vios15_fixpack101_lpp -install –accept reboot ioslevel

3.3. VIOs BACKUP/RESTORE

backupios -file /mnt ( generate a tar file containing a mksysb and other informations )

The command creates a full backup tar file package including all of the resources that the installios command will need to install a Virtual I/O Server (mksysb, bosinst.data, network bootimage, and SPOT) from an HMC using the installios command. backupios file

/mnt/VIOS1_newP5_B2_220308.mksysb -mksysb ( generate only the mksysb file )

4. Shared Ethernet Adapter configuration

4.1. Basic SEA without vlan tagging

4.1.1. Basic SEA logical view

© IBM France ITD La Gaude Unix team 2010

- 9 -

4.1.2. create SEA on VIO1 Initialize all physical lan interf aces with the good settings

4.1.2. create SEA on VIO1

Initialize all physical lan interfaces with the good settings :

For example : 1000_Full_Duplex on the 2 VIOs

4.1.2.1. Create virtual Ethernet adapter on VIO1

Create the Virtual LAN adapter on the Hmc ( with dlpar and in the Vio partition profile ) :

trunk priority = 1 on VIO1

A numerically lower priority receives a higher overall priority.

© IBM France ITD La Gaude Unix team 2010

- 10 -

4.1.2.1.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.1.2.1.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 14 -a

"ieee_virtual_eth=1,port_vlan_id=1101,is_trunk=1,trunk_priority=1"

m=managed system id=partition id s=slot number

VIO Profile update chsyscfg -r prof -m P01MMA03B49A04D07SN652F8BE -I

"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/1101//1/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.1.2.2. create control channel on VIO1

© IBM France ITD La Gaude Unix team 2010

- 11 -

4.1.2.2.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.1.2.2.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 15 -a

"ieee_virtual_eth=0,port_vlan_id=3101"

m=managed system id=partition id s=slot number

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=

15/0/3101///1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.1.2.2.2. rules and naming convention

slot number

all slots > 10 slots < 50 : LAN slots slots > 50 : VSCSI slots same slots on both VIO

PVID

1000 + Physical VLAN id

© IBM France ITD La Gaude Unix team 2010

- 12 -

( Use virtual VLAN id different from the network VLAN on the switch ) 2000 + Physical VLAN id

This rule is mandatory for configuration with several SEA on the same VLAN. For example, in case of VIO pro and pre-prod separated, you must use the same vlan on the SEA of prod and pre-prod. So you must use different VLANid between prod and pre-prod SEA.

PVID for control channel

3000 + Physical VLAN id

4.1.2.3. sea creation

This will generate a device on the VIO server ( cfgdev or cfgmgr ) To find the device, use the slot number in the command lsdev –slots

Exemple lsdev -slots

U9117.570.65A1DA0-V2-C14

U9117.570.65A1DA0-V2-C3

U9117.570.65A1DA0-V2-C15

Virtual I/O Slot ent8 ( slot 2 ) Virtual I/O Slot ent9 Virtual I/O Slot ent13

lsdev -Cc adapter|grep ent

ent0

ent8 Available

Available 0C-20 4-Port 10/100/1000 Base-TX PCI-X Adapter (14101103)

Virtual I/O Ethernet Adapter (l-lan)

mkvdev -sea ent0 -vadapter ent8 -default ent8 -defaultid 1101 –attr ha_mode=auto

ctl_chan=ent13

ent14

Important: If you do not define the ctl_chan attribute to the Shared Ethernet Adapters, they will not be able to negotiate which one is providing the bridging functionality. At this point, both SEAs will be bridging and a spanning tree loop can be formed. To avoid this, always specify the these parameters with the SEA creation.

4.1.3. create SEA on VIO2

Initialize all physical lan interfaces with the good settings :

For example : 1000_Full_Duplex on the 2 VIOs

4.1.3.1. Create virtual Ethernet adapter on VIO2

trunk priority = 2 on VIO2

A numerically lower priority receives a higher overall priority.

© IBM France ITD La Gaude Unix team 2010

- 13 -

4.1.3.1.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.1.3.1.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 14 -a

"ieee_virtual_eth=1,port_vlan_id=1101,is_trunk=1,trunk_priority=2"

m=managed system id=partition id s=slot number

VIO Profile update chsyscfg -r prof -m P01MMA03B49A04D07SN652F8BE -I

"name=Normal,lpar_id=3,virtual_eth_adapters+=14/1/1101//2/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.1.3.2. create control channel on VIO2

© IBM France ITD La Gaude Unix team 2010

- 14 -

4.1.3.2.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.1.3.2.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 15 -a

"ieee_virtual_eth=0,port_vlan_id=3101"

m=managed system id=partition id s=slot number

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=

15/0/3101///1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.1.3.3. sea creation

mkvdev -sea ent0 -vadapter ent8 -default ent8 -defaultid 1101 –attr ha_mode=auto

ctl_chan=ent13

ent14

Important: If you do not define the ctl_chan attribute to the Shared Ethernet Adapters, they will not be able to negotiate which one is providing the bridging functionality. At this point, both SEAs will be bridging and a spanning tree loop can be formed. To avoid this, always specify the these parameters with the SEA creation.

4.1.4. Client partition configuration

© IBM France ITD La Gaude Unix team 2010

- 15 -

4.1.4.1.

Create virtual Ethernet adapter on client partition

Create virtual Ethernet ad apter on client partition 4.1.4.2. IP configuration on client partition lscfg -vl

4.1.4.2. IP configuration on client partition

lscfg -vl ent*|grep ent ent0 U9117.MMA.652F8CE-V27-C2-T1 Virtual I/O Ethernet Adapter set IP address on the device ent0.

4.2. etherchannel

mkvdev -lnagg ent8 ent12 ent16 -attr mode=8023ad

4.3. SEA with vlan tagging on the same device

4.3.1. important : avoid this solution for our architectures

This solution works properly but don’t allow a dynamic update when you have to add a new vlan on your network. It needs to delete the SEA to add a new vlan.

4.3.2. SEA with vlan tagging on same interface logical view

© IBM France ITD La Gaude Unix team 2010

- 16 -

4.3.3. create sea on VIO1 4.3.4. create etherchannel on vio1 Etherchannel VLAN 101/501 mkvdev -lnagg

4.3.3. create sea on VIO1

4.3.4. create etherchannel on vio1

Etherchannel VLAN 101/501 mkvdev -lnagg ent8 ent12 ent16 -attr mode=8023ad

ent34

4.3.4.1. Create virtual Ethernet adapter on VIO1

Trunck priority 1 Not required

© IBM France ITD La Gaude Unix team 2010

- 17 -

4.3.4.1.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.3.4.1.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 14 -a

"ieee_virtual_eth=1,port_vlan_id=808,"addl_vlan_ids=101,501",is_trunk=1,trunk_priority=1"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/808/101,501/1/0"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.3.4.2. create control channel on VIO1

© IBM France ITD La Gaude Unix team 2010

- 18 -

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01 MMA02B2A69D07SN104700D -o a

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 16 -a

"ieee_virtual_eth=0,port_vlan_id=895"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=16/0/895///1"

4.3.4.2.1. rules and naming convention

The devices on VIO server must be set to “not required”.

slot number

all slots > 10 slots < 50 : LAN slots slots > 50 : VSCSI slots same slots on both VIO

PVID

1000 + Physical VLAN id ( Use virtual VLAN id different from the network VLAN on the switch ) 2000 + Physical VLAN id

This rule is mandatory for configuration with several SEA on the same VLAN.

© IBM France ITD La Gaude Unix team 2010

- 19 -

For example, in case of VIO pro and pre-prod separated, you must use the same vlan on the SEA of prod and pre-prod. So you must use different VLANid between prod and pre-prod SEA.

PVID for control channel decrement from 900

4.3.4.3. sea creation on VIO1

lsdev –slots U9117.MMA.652F8CE-V2-C14 Virtual I/O Slot ent36 U9117.MMA.652F8CE-V2-C16 Virtual I/O Slot ent38

lsdev –Cc adapter ent34 Available

EtherChannel / IEEE 802.3ad Link Aggregation

mkvdev -sea ent34 -vadapter ent36 -default ent36 -defaultid 808 -attr ha_mode=auto

ctl_chan=ent38

ent39

4.3.5. create sea on VIO2

4.3.6. create etherchannel on vio2

etherchannel VLAN 101/501 mkvdev -lnagg ent8 ent12 ent16 -attr mode=8023ad

ent34

4.3.6.1. Create virtual Ethernet adapter on VIO2

Trunck priority 2 Not required

© IBM France ITD La Gaude Unix team 2010

- 20 -

4.3.6.1.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.3.6.1.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 14 -a

"ieee_virtual_eth=1,port_vlan_id=808,"addl_vlan_ids=101,501",is_trunk=1,trunk_priority=2"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=14/1/808/101,501/2/0"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.3.6.2. create control channel on VIO2

© IBM France ITD La Gaude Unix team 2010

- 21 -

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01 MMA02B2A69D07SN104700D -o a

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 16 -a

"ieee_virtual_eth=0,port_vlan_id=895"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=16/0/895///1"

4.3.6.3. sea creation on VIO2

lsdev –slots U9117.MMA.652F8CE-V2-C14 Virtual I/O Slot ent36 U9117.MMA.652F8CE-V2-C16 Virtual I/O Slot ent38

lsdev –Cc adapter ent34 Available

EtherChannel / IEEE 802.3ad Link Aggregation

mkvdev -sea ent34 -vadapter ent36 -default ent36 -defaultid 808 -attr ha_mode=auto

ctl_chan=ent38

ent39

4.3.7. Client partition configuration

4.3.7.1. Create virtual Ethernet adapter on client partition

© IBM France ITD La Gaude Unix team 2010

- 22 -

4.3.7.2. IP configuration on client partition lscfg -vl ent*|grep ent ent0 U9117.MMA.652F8CE-V27- C2 -T1 Virtual

4.3.7.2. IP configuration on client partition

lscfg -vl ent*|grep ent ent0 U9117.MMA.652F8CE-V27-C2-T1 Virtual I/O Ethernet Adapter set IP address on the device ent0.

4.3.8. add a new vlan

As explained above, this action is not dynamic. That’s why we recommend not to use this solution in our environments.

Modify the virtual device on VIO1 and VIO2. See example below :

© IBM France ITD La Gaude Unix team 2010

- 23 -

Create the client virtual device and set the IP address on the client partition 4.4.

Create the client virtual device and set the IP address on the client partition

4.4. SEA with vlan tagging – recommended solution

4.4.1. SEA with vlan tagging logical view

© IBM France ITD La Gaude Unix team 2010

- 24 -

4.4.2. Why this solution is recommended This solution is recommended because adding new vlan on

4.4.2. Why this solution is recommended

This solution is recommended because adding new vlan on the SEA is dynamic. Even when a client network is stable, the client need can change at any time.

4.4.3. create sea on VIO1

4.4.4. create etherchannel on vio1

Etherchannel VLAN 101/501 mkvdev -lnagg ent8 ent12 -attr mode=8023ad

ent34

4.4.4.1. Create virtual Ethernet adapters on VIO1

Create one virtual Ethernet adapter per vlan on each VIO server.

Trunck priority 1

© IBM France ITD La Gaude Unix team 2010

- 25 -

4.4.4.1.1. command line mode VIO Dynamic Device creation ( dlpar ) © IBM France ITD
4.4.4.1.1. command line mode VIO Dynamic Device creation ( dlpar ) © IBM France ITD

4.4.4.1.1. command line mode

VIO Dynamic Device creation ( dlpar )

© IBM France ITD La Gaude Unix team 2010

- 26 -

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 14 –a

"ieee_virtual_eth=1,port_vlan_id=1101,"addl_vlan_ids=101",is_trunk=1,trunk_priority=1"

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 15 -a

"ieee_virtual_eth=1,port_vlan_id=1501,"addl_vlan_ids=501",is_trunk=1,trunk_priority=1"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=14/1/1101/101/1/1"

chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=15/1/1501/501/1/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.4.4.2. create control channel on VIO1

4.4.4.2. create control channel on VIO1 4.4.4.2.1. command line mode VIO Dynamic Device creation (

4.4.4.2.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 16 -a

"ieee_virtual_eth=0,port_vlan_id=895"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=16/0/895///1"

© IBM France ITD La Gaude Unix team 2010

- 27 -

4.4.4.2.2. rules and naming convention

slot number

all slots > 10 slots < 50 : LAN slots slots > 50 : VSCSI slots same slots on both VIO

PVID

1000 + Physical VLAN id ( Use virtual VLAN id different from the network VLAN on the switch ) 2000 + Physical VLAN id

This rule is mandatory for configuration with several SEA on the same VLAN. For example, in case of VIO pro and pre-prod separated, you must use the same vlan on the SEA of prod and pre-prod. So you must use different VLANid between prod and pre-prod SEA.

PVID for control channel

decrement from 900

4.4.4.3. sea creation on VIO1

lsdev –slots U9117.MMA.652F8CE-V2-C14 Virtual I/O Slot ent36 U9117.MMA.652F8CE-V2-C15 Virtual I/O Slot ent37 U9117.MMA.652F8CE-V2-C16 Virtual I/O Slot ent38

lsdev –Cc adapter ent34 Available

EtherChannel / IEEE 802.3ad Link Aggregation

mkvdev -sea ent34 -vadapter ent36 ent37 -default ent36 -defaultid 808 -attr ha_mode=auto

ctl_chan=ent38

ent39 ( new device genereted )

4.4.5. create sea on VIO2

4.4.6. create etherchannel on vio2

etherchannel VLAN 101/501 mkvdev -lnagg ent8 ent12 -attr mode=8023ad

ent34

4.4.6.1. Create virtual Ethernet adapter on VIO2

Trunck priority 2

© IBM France ITD La Gaude Unix team 2010

- 28 -

4.4.6.1.1. command line mode © IBM France ITD La Gaude Unix team 2010 - 29
4.4.6.1.1. command line mode © IBM France ITD La Gaude Unix team 2010 - 29

4.4.6.1.1. command line mode

© IBM France ITD La Gaude Unix team 2010

- 29 -

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 14 –a

"ieee_virtual_eth=1,port_vlan_id=1101,"addl_vlan_ids=101",is_trunk=1,trunk_priority=2"

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 15 -a

"ieee_virtual_eth=1,port_vlan_id=1501,"addl_vlan_ids=501",is_trunk=1,trunk_priority=2"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=14/1/1101/101/2/1"

chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=15/1/1501/501/2/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.4.6.2. create control channel on VIO2

4.4.6.2. create control channel on VIO2 4.4.6.2.1. command line mode VIO Dynamic Device creation (

4.4.6.2.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 3 --rsubtype eth -s 16 -a

"ieee_virtual_eth=0,port_vlan_id=895"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=3,virtual_eth_adapters+=16/0/895///1"

© IBM France ITD La Gaude Unix team 2010

- 30 -

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.4.6.3. sea creation on VIO2

mkvdev -sea ent34 -vadapter ent36 ent37 -default ent36 -defaultid 808 -attr ha_mode=auto

ctl_chan=ent38

ent39 ( new device genereted )

4.4.7. Client partition configuration

4.4.7.1. Create virtual Ethernet adapter on client partition

4.4.7.1. Create virtual Ethernet ad apter on client partition © IBM France ITD La Gaude Unix

© IBM France ITD La Gaude Unix team 2010

- 31 -

4.4.7.2. IP configuration on client partition lsdev –Cc adapter ent0 U9117.MMA.652F8CE-V27- C2 -T1 Virtual I/O

4.4.7.2. IP configuration on client partition

lsdev –Cc adapter ent0 U9117.MMA.652F8CE-V27-C2-T1 Virtual I/O Ethernet Adapter set IP address on the device ent0.

ent1 U9117.MMA.652F8CE-V27-C3-T1 Virtual I/O Ethernet Adapter set IP address on the device ent1.

4.4.8. add a new vlan

4.4.8.1. Create virtual Ethernet adapters on VIO1

© IBM France ITD La Gaude Unix team 2010

- 32 -

4.4.8.1.1. command line mode VIO Dynamic Device creation ( dlpar ) © IBM France ITD
4.4.8.1.1. command line mode VIO Dynamic Device creation ( dlpar ) © IBM France ITD

4.4.8.1.1. command line mode

VIO Dynamic Device creation ( dlpar )

© IBM France ITD La Gaude Unix team 2010

- 33 -

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 17 -a

"ieee_virtual_eth=1,port_vlan_id=1105,"addl_vlan_ids=105",is_trunk=1,trunk_priority=1"

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 18 -a

"ieee_virtual_eth=1,port_vlan_id=1505,"addl_vlan_ids=505",is_trunk=1,trunk_priority=1"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=17/1/1105/105/1/1

chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=18/1/1505/505/1/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.4.8.2. Update SEA on VIO1

lsdev -slots U9117.MMA.652F8CE-V3-C17 Virtual I/O Slot ent56 U9117.MMA.652F8CE-V3-C18 Virtual I/O Slot ent57

chdev -dev ent39 -attr virt_adapters=ent36,ent37, ent56, ent57

4.4.8.3. Create virtual Ethernet adapters on VIO2

ent56 , ent57 4.4.8.3. Create virtual Ethern et adapters on VIO2 © IBM France ITD La

© IBM France ITD La Gaude Unix team 2010

- 34 -

4.4.8.3.1. command line mode VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m

4.4.8.3.1. command line mode

VIO Dynamic Device creation ( dlpar ) chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 17 -a

"ieee_virtual_eth=1,port_vlan_id=1105,"addl_vlan_ids=105",is_trunk=1,trunk_priority=2"

chhwres -r virtualio -m P01MMA02B2A69D07SN104700D -o a --id 2 --rsubtype eth -s 18 -a

"ieee_virtual_eth=1,port_vlan_id=1505,"addl_vlan_ids=505",is_trunk=1,trunk_priority=2"

VIO Profile update chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=17/1/1105/105/2/1

chsyscfg -r prof -m P01MMA02B2A69D07SN104700D -i

"name=Normal,lpar_id=2,virtual_eth_adapters+=18/1/1505/505/2/1"

virtual-slot-number/is-IEEE/ port-vlan-ID/additional-vlan-IDs/trunk-priority/is-required

4.4.8.4. Update SEA on VIO2

lsdev -slots U9117.MMA.652F8CE-V3-C17 Virtual I/O Slot ent56 U9117.MMA.652F8CE-V3-C18 Virtual I/O Slot ent57

chdev -dev ent39 -attr virt_adapters=ent36,ent37, ent56, ent57

© IBM France ITD La Gaude Unix team 2010

- 35 -

4.4.8.5. Client partition configuration

4.4.8.5.1. Create virtual Ethernet on client partition

4.4.8.5.1. Create virtual Ethernet on client partition 4.4.8.5.2. IP configuration on client partition © IBM
4.4.8.5.1. Create virtual Ethernet on client partition 4.4.8.5.2. IP configuration on client partition © IBM

4.4.8.5.2. IP configuration on client partition

© IBM France ITD La Gaude Unix team 2010

- 36 -

lscfg -vl ent*|grep ent

ent1

U9117.MMA.652F8BE-V34-C5-T1 Virtual I/O Ethernet Adapter (l-lan)

ent0

U9117.MMA.652F8BE-V34-C4-T1 Virtual I/O Ethernet Adapter (l-lan)

set IP address on the device ent0 and ent1

5. san disks configuration

5.1. rules and naming convention

5.1.1. vtd naming convention

Depending on your ios version and fixpack level, it’s possible that the vtd name can be renamed by LPM. In this case, it’s useless to define vtd naming convention.

Depending on the customer specific context, it can be usefull that the vtd contains several information such as the geographical or functional information.

Here is the recommended vtd syntax :

[ additional information ] [ _ ] < client partition name > [ _ ] < lun_id >

additional information : any geographical or functional information

client partition name lun_id

: this name can be reduced if necessary : lun id on the storage subsystem

Examples :

usr210_0543

us : counry r210 : a part of the client partition name 0543 : lunid on DS8000

saxsap12phd4

saxsap12p : a part of the client partition name hd4 : corresponding disk on the client partition

5.1.2. slot numbers and vscsi server

5.1.2.1. rules with LPM

Vscsi server adapter slot number

No specific rule for slot number on VIO server if LPM is used (LPM redefines slot numbers).

Vscsi client adapter

© IBM France ITD La Gaude Unix team 2010

- 37 -

Use Odd numbered slots starting at 21 for connection to the first VIO server, and use even numbered slots starting at 20 for connection to the second VIO server. Allocate slots in pairs, with each client using two adjacent slots such as 21 and 20, or 31 and 30.

5.1.2.2. Rules without lpm

Vscsi server adapter

The first VIOS should use Odd numbered slots number starting at 51, and the second should use even numbered slots starting at 50. In a two server scenario, allocate slots in pairs, with each client using two adjacent slots such as 51 and 50, or 61 and 60.

Use a specific VSCSI server for each storage subsystem.

Vscsi client adapter

Use the same slot number between the vscsi server and client adapter.

5.2. redbook references for rules and naming convention

see explain in the following redbook :

“IBM System p Advanced POWER Virtualization Best Practices” redp4194.pdf chapter 4.3.2 : “Virtual device slot numbers”

“After establishing the naming conventions, also establish slot numbering conventions for the virtual I/O adapters. Slot numbers are shared between virtual storage and virtual network devices. In complex systems, there will tend to be far more storage devices than network devices because each virtual SCSI device can only communicate with one server or client. We recommend reserving the slot numbers through 20 for network devices on all LPARs in order to keep the network and storage devices grouped together. Management can be simplified by keeping slot numbers consistent between the virtual I/O client and server. However, when partitions are moved from one server to another, this might not be possible. For more information, see 4.6.3, “Storage planning” on page 108. In environments with only one virtual I/O server, add storage adapters incrementally starting with slot 21 and higher. When clients are attached to two Virtual I/O Servers, the adapter slot numbers should be alternated from one VIOS to the other. The first VIOS should use odd numbered slots starting at 21, and the second should use even numbered slots starting at 22. In a two server scenario, allocate slots in pairs, with each client using two adjacent slots such as 21 and 22, or 33 and 34.”

5.3. Virtual devices adapter vs queue_depth

As explained below, you have to take care of the maximum disks per virtual adapter server on the VIO. Use the following process to evaluate this max value : ( 512 – 2 ) / ( 3 + queue depth )

See redbook “power VM Virtualization Managing and monitoring” sg247590.pdf chapter 2.5.2 Virtual device slot numbers:

© IBM France ITD La Gaude Unix team 2010

- 38 -

Because virtual SCSI connections operate at memory speed, there is generally no performance gain from adding multiple adapters between a Virtual I/O Server and client. For AIX virtual I/O client partitions, each adapter pair can handle up to 85 virtual devices with the default queue depth of three. For IBM i clients, up to 16 virtual disk and 16 optical devices are supported. In situations where virtual devices per partition are expected to exceed that number, or where the queue depth on some devices might be increased above the default, reserve additional adapter slots for the Virtual I/O Server and the virtual I/O client partition. When tuning queue depths, the VSCSI adapters have a fixed queue depth. There are 512 command elements of which 2 are used by the adapter, 3 are reserved for each VSCSI LUN for error recovery and the rest are used for I/O requests. Thus, with the default queue depth of 3 for VSCSI LUNs, that allows for up to 85 LUNs to use an adapter: (512 - 2) / (3 + 3) = 85 rounding down. So if you need higher queue depths for the devices, then the number of LUNs per adapter is reduced. For example, if you want to use a queue depth of 25, that allows 510/28= 18 LUNs per adapter for an AIX client partition.”

5.4. Implementation in Typical configurations

5.4.1. Configuration 1

2 SAN VIO 1 storage subsystem 1 vscsi server per VIO server LPM used

storage subsystem 1 vscsi server per VIO server LPM used vscsi1 U9119.FHA.832EE32-V234- C21 -T1 VIO2 Storage

vscsi1

U9119.FHA.832EE32-V234-C21-T1

VIO2

Storage B4

vscsi0

U9119.FHA.832EE32-V234-C20-T1

VIO1

Storage B4

Client adapter slot numbers

© IBM France ITD La Gaude Unix team 2010

- 39 -

5.4.2.

configuration 2

2 SAN VIO 2 storage subsystems 2 vscsi server per VIO : 1 for each storage subsystem LPM used

server per VIO : 1 for each storage subsystem LPM used Storage bay B4 ( slot

Storage bay B4 ( slot nb range 20 to 29 ) Storage bay G1 ( slot nb range 30 to 39 )

vscsi3

U9119.FHA.832EE32-V234-C31-T1

VIO2

Storage G1

vscsi2

U9119.FHA.832EE32-V234-C30-T1

VIO1

Storage G1

vscsi1

U9119.FHA.832EE32-V234-C21-T1

VIO2

Storage B4

vscsi0

U9119.FHA.832EE32-V234-C20-T1

VIO1

Storage B4

Client adapter slot numbers

5.4.3. configuration 3

2 SAN VIO 2 storage subsystems 4 vscsi server per VIO : 2 for each storage subsystems ( for nb device limitation ) LPM used

Storage bay B4 ( slot nb range 20 to 29 ) Storage bay G1 ( slot nb range 30 to 39 )

vscsi7

U9119.FHA.834EE9F-V132-C33-T1

VIO2

Storage G1

vscsi6

U9119.FHA.834EE9F-V132-C32-T1

VIO1

Storage G1

© IBM France ITD La Gaude Unix team 2010

- 40 -

vscsi5

U9119.FHA.834EE9F-V132-C23-T1

VIO2

Storage B4

vscsi4

U9119.FHA.834EE9F-V132-C22-T1

VIO1

Storage B4

vscsi3

U9119.FHA.834EE9F-V132-C31-T1

VIO2

Storage G1

vscsi2

U9119.FHA.834EE9F-V132-C30-T1

VIO1

Storage G1

vscsi1

U9119.FHA.834EE9F-V132-C21-T1

VIO2

Storage B4

vscsi0

U9119.FHA.834EE9F-V132-C20-T1

VIO1

Storage B4

Client adapter slot numbers

5.5. Fiber channel and hdisk devices settings

For each disk device on the vio server :

Activate the pvid on all the disks devices on both vioservers : chdev -l hdiskX -a pv=yes Set the reserve_policy parameter : chdev -dev hdiskX -attr reserve_policy=no_reserve

For each Fiber channel card on the VIO server Initialize all physical fiber channel interfaces with the good settings chdev -dev fscsi0 -attr fc_err_recov=fast_fail dyntrk=yes –perm

5.6. Configuration 2 detailed implementation

Vio servers :

VIE1P01MMA04B4 VIE2P01MMA04B4

Client partition :

PODCLIENT_TEST6

building :

B4 and G1

dual storage :

1 storage subsystem in each building

LPM used

Doesn’t need the same slot number between server and client adapter

In this example, the values of vscsi server adapters slot number is not important as LPM is used.

5.6.1. create vscsi server adapter on VIO1 for storage B4

The adapter must be not required for LPM

© IBM France ITD La Gaude Unix team 2010

- 41 -

5.6.1.1. command line mode vio1 device, server slot 50, client slot 20, storage B4 chhwres

5.6.1.1. command line mode

vio1 device, server slot 50, client slot 20, storage B4

chhwres -r virtualio -m 9119-FHA-SN834EE9F-BA1A05D12 -o a --id 4 --rsubtype scsi -s 50 -a "adapter_type=server,remote_lpar_name=PODCLIENT_TEST6,remote_slot_num=20"

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=4,virtual_scsi_adapters+=50/server//PODCLIENT_TEST6/20/0"

5.6.2. create vscsi server adapter on VIO2 for storage B4

The adapter must be not required for LPM

© IBM France ITD La Gaude Unix team 2010

- 42 -

5.6.2.1. command line mode vio2 device, server slot 51, client slot 20, storage B4 chhwres

5.6.2.1. command line mode

vio2 device, server slot 51, client slot 20, storage B4

chhwres -r virtualio -m 9119-FHA-SN834EE9F-BA1A05D12 -o a --id 5 --rsubtype scsi -s 51 -a "adapter_type=server,remote_lpar_name=PODCLIENT_TEST6,remote_slot_num=21"

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=5,virtual_scsi_adapters+=51/server//PODCLIENT_TEST6/21/0"

5.6.3. create client partition device for VIO1 and storage B4

/0" 5.6.3. create client partition devi ce f or VIO1 and storage B4 © IBM France

© IBM France ITD La Gaude Unix team 2010

- 43 -

5.6.3.1. command line mode

client partition device, vio1, server slot 50, client slot 20, storage B4

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=128,virtual_scsi_adapters+=20/client//VIE1P01MMA04B4/50/1"

5.6.4. create client partition device for VIO2 and storage B4

create client partition devi ce f or VIO2 and storage B4 5.6.4.1. command line mode client

5.6.4.1. command line mode

client partition device,vio2, server slot 51, client slot 21, storage B4

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=128,virtual_scsi_adapters+=21/client//VIE2P01MMA04B4/40/1"

5.6.5. create vscsi server adapter on VIO1 for storage G1

The adapter must be not required for LPM

© IBM France ITD La Gaude Unix team 2010

- 44 -

5.6.5.1. command line mode vio1 device, server slot 60, client slot 30, storage G1 chhwres

5.6.5.1. command line mode

vio1 device, server slot 60, client slot 30, storage G1

chhwres -r virtualio -m 9119-FHA-SN834EE9F-BA1A05D12 -o a --id 4 --rsubtype scsi -s 60 -a "adapter_type=server,remote_lpar_name=PODCLIENT_TEST6,remote_slot_num=30"

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=4,virtual_scsi_adapters+=60/server//PODCLIENT_TEST6/30/0"

5.6.6. create vscsi server adapter on VIO2 for storage G1

© IBM France ITD La Gaude Unix team 2010

- 45 -

5.6.6.1. command line mode vio2 device, server slot 61, client slot 31, storage G1 chhwres

5.6.6.1. command line mode

vio2 device, server slot 61, client slot 31, storage G1

chhwres -r virtualio -m 9119-FHA-SN834EE9F-BA1A05D12 -o a --id 5 --rsubtype scsi -s 61 -a "adapter_type=server,remote_lpar_name=PODCLIENT_TEST6,remote_slot_num=31"

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=5,virtual_scsi_adapters+=61/server//PODCLIENT_TEST6/31/0"

5.6.7. create client partition device for VIO1 and storage G1

/0" 5.6.7. create client partition device for VIO1 and storage G1 © IBM France ITD La

© IBM France ITD La Gaude Unix team 2010

- 46 -

5.6.7.1. command line mode

client partition device, vio1, server slot 60, client slot 30, storage G1

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=128,virtual_scsi_adapters+=30/client//VIE1P01MMA04B4/60/1"

5.6.8. create client partition device for VIO2 and storage G1

create client partition device for VIO2 and storage G1 5.6.8.1. command line mode client partition device,

5.6.8.1. command line mode

client partition device, vio2, server slot 61, client slot 31, storage G1

chsyscfg -r prof -m 9119-FHA-SN834EE9F-BA1A05D12 -i

"name=default,lpar_id=128,virtual_scsi_adapters+=31/client//VIE2P01MMA04B4/61/1"

5.6.9. create virtual devices on VIO1

lsdev -slots U9119.FHA.834EE9F-V4-C50 Virtual I/O Slot vhost0 U9119.FHA.834EE9F-V4-C60 Virtual I/O Slot vhost5

vhost5 will be used for G1 storage subsystem.

vhost0 will be used for B4 storage subsystem.

rootvg

hdisk13 B4

hdisk52 G1

su - padmin -c ioscli mkvdev -vdev hdisk13 -vadapter vhost0 -dev p10_0230 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk52 -vadapter vhost5 -dev p10_0231 ;

© IBM France ITD La Gaude Unix team 2010

- 47 -

ssh PODCLIENT_TEST6 cfgmgr

hdisk2 to hdisk6 : storage subsystem B4

su - padmin -c ioscli mkvdev -vdev hdisk2 -vadapter vhost0 -dev usp010_3102 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk3 -vadapter vhost0 -dev usp010_3103 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk4 -vadapter vhost0 -dev usp010_3104 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk5 -vadapter vhost0 -dev usp010_3105 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk6 -vadapter vhost0 -dev usp010_3106 ; ssh PODCLIENT_TEST6 cfgmgr

hdisk39 to hdisk43 : storage subsystem G1

su - padmin -c ioscli mkvdev -vdev hdisk39 -vadapter vhost5 -dev usp010_2050 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk40 -vadapter vhost5 -dev usp010_2051 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk41 -vadapter vhost5 -dev usp010_2052 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk42 -vadapter vhost5 -dev usp010_2053 ; ssh PODCLIENT_TEST6 cfgmgr su - padmin -c ioscli mkvdev -vdev hdisk43 -vadapter vhost5 -dev usp010_2054 ; ssh PODCLIENT_TEST6 cfgmgr

5.6.10. create virtual devices on VIO2

lsdev -slots U9119.FHA.834EE9F-V4-C40 Virtual I/O Slot vhost0 U9119.FHA.834EE9F-V4-C45 Virtual I/O Slot vhost5

The vhosts are the same as the vhosts of VIO1, the script to generate the virtual devices is the same.

5.6.11. Client partition devices setting

5.6.11.1.

priority

Change the priority on all the disks :

You have to share the workload between both vioservers.

Here are the different strategies :

Set all the disks of each partition on one vioserver with priority 1 Spread all disks of each partition between both VIO servers

chpath -l hdisk0 -p vscsi5 -a priority=2 chpath -l hdisk1 -p vscsi5 -a priority=2

© IBM France ITD La Gaude Unix team 2010

- 48 -

Check : lspath -AHE -l hdiskx -p vscsiY

5.6.11.2. hcheck_interval

Change the following setting on each disk device :

hcheck_interval needs to be greater than the R/W timeout interval.

1)

check the rw_timeout value on the physical device disks on the vio

2)

put at least the same value on the hcheck_interval on the virtual device disks on the partition

chdev -l hdiskX -a hcheck_interval=60 –P

Check lsattr -El hdiskX| grep hcheck_interval

5.6.11.3. queue depth

For all the device disks of the client partitions, set the queue depth at the same value as the queue depth value of the physical disks of the VIO servers. See recommendation redbook "Virt Best Practices Redp4194" chapter 4.8 SCSI queue depth:

“We recommend that you have the same queue depth on the virtual disk as the physical disk”

5.6.11.4. vscsi_path_to

see redbook sg247940 “advanced power virtualization” chapter 4.2.1 MPIO in the client partition”

On all the virtual scsi devices of the client partition, put the following value :

chdev -l vscsiX -a vscsi_path_to=30 –P

6. scsi disk devices configuration for rootvg client partitions

Create the vhost in WebSm menu. properties -> Virtual IO Adapter->onglet scsi->create server adapter Naming convention.You can use the following rules :

Start the slots numbers to 20 for the Vscsi devices. Use < 20 for the LAN devices. Use even slots numbers for VIO1 and odd numbers for VIO2 Use the first slot number for Aix disk and the other for the swap disk Set the maximum slot number to a value that prevents the VIO partition stop.

assign the new virtual device to its specific client partition.

On the VIOs : cfgdev Check with lsdev -slots

6.1. Create the logical volumes and the virtual devices

On vioserver1

© IBM France ITD La Gaude Unix team 2010

- 49 -

Create the virtual server device in WebSm. assign the new virtual device to its specific client partition. See example below :

In our naming convention, we advice to chose the same slot number for the client and for the server virtual scsi adapter ( see example below ) :

the server virtual scsi adapte r ( see example below ) : mkvg -f -vg vioserver1VG

mkvg -f -vg vioserver1VG hdisk1 hdisk5 extendvg vioserver1VG hdisk2 hdisk6

partition mopesaxsap15p mklv -lv saxsap15phd0LV vioserver1VG 30G hdisk1 mklv -lv saxsap15phd2LV vioserver1VG 20G hdisk5 mkvdev -vdev saxsap15phd0LV -vadapter vhost0 -dev vtsaxsap15phd0 mkvdev -vdev saxsap15phd2LV -vadapter vhost1 -dev vtsaxsap15phd2

on vioserver2 Create the virtual server device in WebSm mkvg -f -vg vioserver2VG hdisk1 hdisk5 extendvg vioserver2VG hdisk2 hdisk6

partition mopesaxsap15p mklv -lv saxsap15phd1LV vioserver2VG 30G hdisk2 mklv -lv saxsap15phd3LV vioserver2VG 20G hdisk6 mkvdev -vdev saxsap15phd1LV -vadapter vhost0 -dev vtsaxsap15phd1 mkvdev -vdev saxsap15phd3LV -vadapter vhost1 -dev vtsaxsap15phd3

On the client partition mopesaxsap15p Create the vscsi client on the client partition profile ( same slot number as the vscsi server )

© IBM France ITD La Gaude Unix team 2010

- 50 -

Here is an example of the result after the creation of 4 virtual scsi client

Here is an example of the result after the creation of 4 virtual scsi client adapter :

© IBM France ITD La Gaude Unix team 2010

- 51 -

7. NTP configuration on VIO and HMC Enable NTP on all VIO servers and all

7. NTP configuration on VIO and HMC

Enable NTP on all VIO servers and all HMC.

7.1. Ntp configuration on VIO servers

See redbook sg247590 “power VM managing and monitoring” chapter 4.3 “Network Time Protocol configuration”

vi /home/padmin/config/ntp.conf startnetsvc xntpd ( stopnetsvc xntpd ) check your ntp.log file lsnetsvc xntpd ntpdate –d <server>

7.2. NTP configuration on HMC

See redbook sg247491 “HMC V7 handbook” chapter 2.5.14 “Change Date and Time”

© IBM France ITD La Gaude Unix team 2010

- 52 -

© IBM France ITD La Gaude Unix team 2010 - 53 -

© IBM France ITD La Gaude Unix team 2010

- 53 -