Вы находитесь на странице: 1из 18

Expert Oracle

Writings on Oracle Database, Applications & more


http://expertoracle.com

Add new multipath disk to ASM Diskgroup


Author : Brijesh Gogia

A Multipath disk provide failover by using redundant physical path components, eliminating a
single point of failure with the Storage Area Network (SAN). Multipathing is a software
technology implemented at the operating system device driver level.

With ASM, you can ensure the discovery of a multipath disk by setting the value of the
initialization parameter ASM_DISKSTRING equal to the name of the pseudo device that
represents the multipath disk.

Below steps can be followed to add a new disk to ASM Disk Group.

STEP 1) CHECK CURRENT DISKS

1 SQL> select GROUP_NUMBER,DISK_NUMBE


R,MOUNT_STATUS,HEADER_STATUS,MOD
2 E_STATUS,STATE,VOTING_FILE,name,path
from v$asm_disk;
3

4
GROUP_NUMBER DISK_NUMBER
5 MOUNT_S HEADER_STATU MODE_ST
STATE V NAME PATH
6
------------ ----------- ------- ------------ ------- -------- -
7 -------------- ---------------------------------

8 2 0 CACHED MEMBER ONLINE NORMAL N


ARC_LOG_0000
9 /dev/oracleasm/disks/ARC_LOG01

1 / 18
10 1 0 CACHED MEMBER ONLINE NORMAL Y
OCR_VOTE_0000
11 /dev/oracleasm/disks/OCR_VOTE01

12 3 4 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0004
13 /dev/oracleasm/disks/DB_DATA05

14 3 3 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0003
/dev/oracleasm/disks/DB_DATA04

3 2 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0002
/dev/oracleasm/disks/DB_DATA03

3 1 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0001
/dev/oracleasm/disks/DB_DATA02

3 0 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0000
/dev/oracleasm/disks/DB_DATA01

3 5 CACHED MEMBER ONLINE NORMAL N


DB_DATA_0005
/dev/oracleasm/disks/DB_DATA06

8 rows selected.

STEP 2) CHECK ASM TO PHYSICAL DISK MAPPING

If you have not documented earlier which ASM disk is mapped to which physical disk and what
all free disks you have in your server, your need t firs find out this information.

Use asm querydisk command first with every ASM disk name:

2 / 18
1 # /etc/init.d/oracleasm querydisk -d
DB_DATA01
2
Disk "DB_DATA01" is a valid ASM disk on
3 device /dev/sdl[8,176]

5 # /etc/init.d/oracleasm querydisk -d
DB_DATA02
6
Disk "DB_DATA02" is a valid ASM disk on
7 device /dev/sdm[8,192]

9 # /etc/init.d/oracleasm querydisk -d
DB_DATA03
10
Disk "DB_DATA03" is a valid ASM disk on
11 device /dev/sdn[8,208]

12

13 # /etc/init.d/oracleasm querydisk -d
DB_DATA04
14
Disk "DB_DATA04" is a valid ASM disk on
15 device /dev/sdo[8,224]

16

17 # /etc/init.d/oracleasm querydisk -d
DB_DATA05
18
Disk "DB_DATA05" is a valid ASM disk on
19 device /dev/sdp[8,240]

20

3 / 18
21 # /etc/init.d/oracleasm querydisk -d
DB_DATA06
22
Disk "DB_DATA06" is a valid ASM disk on
23 device /dev/sdq[65,0]

# /etc/init.d/oracleasm querydisk -d
ARC_LOG01

Disk "ARC_LOG01" is a valid ASM disk on


device /dev/sdr[65,16]

# /etc/init.d/oracleasm querydisk -d
OCR_VOTE01

Disk "OCR_VOTE01" is a valid ASM disk on


device /dev/sdb1[8,17]

Now use multipath -ll to check corresponding multipath disk and what all disks are free in your
server.

1 # multipath -ll

3 mpath7
(360060e8006d8e7000000d8e700001000)

4 / 18
4 dm-17 HP,OPEN-V

5 [size=50G][features=1
queue_if_no_path][hwhandler=0][rw]
6
\_ round-robin 0 [prio=0][active]
7
\_ 3:0:0:0 sdb 8:16 [active][ready]
8
\_ 4:0:0:0 sdv 65:80 [active][ready]
9

10
mpath18
11 (360060e8006d8e7000000d8e700000001)
dm-8 HP,OPEN-V
12
[size=100G][features=1
13 queue_if_no_path][hwhandler=0][rw]

14 \_ round-robin 0 [prio=0][active]

15 \_ 4:0:0:11 sdag 66:0 [active][ready]

16 \_ 3:0:0:11 sdm 8:192 [active][ready]

17

18 mpath19
(360060e8006d8e7000000d8e700000002)
19 dm-9 HP,OPEN-V

20 [size=100G][features=1
queue_if_no_path][hwhandler=0][rw]
21
\_ round-robin 0 [prio=0][active]
22
\_ 4:0:0:12 sdah 66:16 [active][ready]
23
\_ 3:0:0:12 sdn 8:208 [active][ready]
24

25
mpath20
26 (360060e8006d8e7000000d8e700000003)
dm-10 HP,OPEN-V

5 / 18
27 [size=100G][features=1
queue_if_no_path][hwhandler=0][rw]
28
\_ round-robin 0 [prio=0][active]
29
\_ 4:0:0:13 sdai 66:32 [active][ready]
30
\_ 3:0:0:13 sdo 8:224 [active][ready]
31

32
mpath21
33 (360060e8006d8e7000000d8e700000004)
dm-11 HP,OPEN-V
34
[size=100G][features=1
35 queue_if_no_path][hwhandler=0][rw]

36 \_ round-robin 0 [prio=0][active]

37 \_ 4:0:0:14 sdaj 66:48 [active][ready]

38 \_ 3:0:0:14 sdp 8:240 [active][ready]

39

40 mpath22
(360060e8006d8e7000000d8e700000005)
41 dm-12 HP,OPEN-V

42 [size=100G][features=1
queue_if_no_path][hwhandler=0][rw]
43
\_ round-robin 0 [prio=0][active]
44
\_ 4:0:0:15 sdak 66:64 [active][ready]
45
\_ 3:0:0:15 sdq 65:0 [active][ready]
46

47
mpath23
48 (360060e8006d8e7000000d8e700000006)
dm-13 HP,OPEN-V
49
[size=100G][features=1

6 / 18
queue_if_no_path][hwhandler=0][rw]

\_ round-robin 0 [prio=0][active]

\_ 4:0:0:16 sdal 66:80 [active][ready]

\_ 3:0:0:16 sdr 65:16 [active][ready]

mpath24
(360060e8006d8e7000000d8e700000007)
dm-14 HP,OPEN-V

[size=100G][features=1
queue_if_no_path][hwhandler=0][rw]

\_ round-robin 0 [prio=0][active]

\_ 4:0:0:17 sdam 66:96 [active][ready]

\_ 3:0:0:17 sds 65:32 [active][ready]

Looking at the above two outputs we can find out which ASM disk is attached to what physical
disk

For example DB_DATA02 (on /dev/sdm) is attached to Physical multi disk path mpath18.

mpath24 can be seen as free physical disk which is not linked to any ASM disk yet.

STEP 3) PARTITION THE DISK

We will use the free disk /dev/mapper/mpath24

7 / 18
1 # fdisk -l /dev/mapper/mpath24

3 Disk /dev/mapper/mpath24: 107.3 GB,


107374510080 bytes
4
255 heads, 63 sectors/track, 13054 cylinders
5
Units = cylinders of 16065 * 512 = 8225280
6 bytes

Disk /dev/mapper/mpath24 doesn't contain a


valid partition table

1 # fdisk /dev/mapper/mpath24

2 Device contains neither a valid DOS partition


table, nor Sun, SGI or OSF disklabel
3
Building a new DOS disklabel. Changes will
4 remain in memory only,

5 until you decide to write them. After that, of


course, the previous
6
content won't be recoverable.
7

8 / 18
8 The number of cylinders for this disk is set to
13054.
9
There is nothing wrong with that, but this is
10 larger than 1024,

11 and could in certain setups cause problems


with:
12
1) software that runs at boot time (e.g., old
13 versions of LILO)

14 2) booting and partitioning software from other


OSs
15
(e.g., DOS FDISK, OS/2 FDISK)
16
Warning: invalid flag 0x0000 of partition table 4
17 will be corrected by w(rite)

18

19 Command (m for help): u

20 Changing display/entry units to sectors

21

22 Command (m for help): n

23 Command action

24 e extended

25 p primary partition (1-4)

26 p

27 Partition number (1-4): 1

28 First sector (63-209715839, default 63): 2048

29 Last sector or +size or +sizeM or +sizeK


(2048-209715839, default 209715839):
30
Using default value 209715839

9 / 18
31

32 Command (m for help): w

33 The partition table has been altered!

34

35 Calling ioctl() to re-read partition table.

36

WARNING: Re-reading the partition table failed


with error 22: Invalid argument.

The kernel still uses the old table.

The new table will be used at the next reboot.

Syncing disks.

1 # /sbin/partprobe

10 / 18
1 # fdisk -l /dev/mapper/mpath24

3 Disk /dev/mapper/mpath24: 107.3 GB,


107374510080 bytes
4
255 heads, 63 sectors/track, 13054 cylinders
5
Units = cylinders of 16065 * 512 = 8225280
6 bytes

8 Device Boot Start End Blocks Id System

/dev/mapper/mpath24p1 1 13055 104856896


83 Linux

1 # kpartx -a /dev/mapper/mpath24

STEP 4) CREATE ASM DISK

11 / 18
1 # /etc/init.d/oracleasm createdisk ARC_LOG02
/dev/mapper/mpath24p1
2
Marking disk "ARC_LOG02" as an ASM disk: [
OK ]

Run below command on all database nodes (if using RAC):

1 # /etc/init.d/oracleasm scandisks

2 Scanning the system for Oracle ASMLib disks: [


OK ]

12 / 18
1 # /etc/init.d/oracleasm listdisks

2 ARC_LOG01

3 ARC_LOG02

4 DB_DATA01

5 DB_DATA02

6 DB_DATA03

7 DB_DATA04

8 DB_DATA05

9 DB_DATA06

10 OCR_VOTE01

The new disk should be visible from all database nodes now.

Now we have ASM disk ready to be added to ASM Disk Group

STEP 5) ADD ASM DISK TO DISKGROUP

Check Current Status >

1 SQL> set lines 132

2 SQL> col name format a14

3 SQL> col PATH format a33

13 / 18
4 SQL> select
GROUP_N
5 UMBER,NAME,STATE,TYPE
,TOTAL_MB,FREE_MB,VOTING_FILES from
6 v$asm_diskgroup;

8 GROUP_NUMBER NAME STATE TYPE


TOTAL_MB FREE_MB V
9
------------ -------------- ----------- ------ ----------
10 ---------- -

11 1 ARC_LOG MOUNTED EXTERN 102400


101705 N
12
2 DB_DATA MOUNTED EXTERN 614400
13 167345 N

14 3 OCR_VOTE MOUNTED EXTERN 51199


50803 Y
15

16
SQL> select GROUP_NUMBER,DISK_NUMBE
17 R,STATE,name,path
,MOUNT_STATUS,HEADER_STATUS from
18 v$asm_disk;

19

20 GROUP_NUMBER DISK_NUMBER STATE


NAME PATH MOUNT_S HEADER_STATU
21
------------ ----------- -------- --------------
22 --------------------------------- ------- ------------

23 0 0 NORMAL
/dev/oracleasm/disks/ARC_LOG02 CLOSED
24 PROVISIONED

25 2 0 NORMAL ARC_LOG_0000
/dev/oracleasm/disks/ARC_LOG01 CACHED
26 MEMBER

14 / 18
1 0 NORMAL OCR_VOTE_0000
/dev/oracleasm/disks/OCR_VOTE01 CACHED
MEMBER

3 4 NORMAL DB_DATA_0004
/dev/oracleasm/disks/DB_DATA05 CACHED
MEMBER

3 3 NORMAL DB_DATA_0003
/dev/oracleasm/disks/DB_DATA04 CACHED
MEMBER

3 2 NORMAL DB_DATA_0002
/dev/oracleasm/disks/DB_DATA03 CACHED
MEMBER

3 1 NORMAL DB_DATA_0001
/dev/oracleasm/disks/DB_DATA02 CACHED
MEMBER

3 0 NORMAL DB_DATA_0000
/dev/oracleasm/disks/DB_DATA01 CACHED
MEMBER

3 5 NORMAL DB_DATA_0005
/dev/oracleasm/disks/DB_DATA06 CACHED
MEMBER

9 rows selected.

As you can see that the new disk’s status is showing as ‘PROVISIONED’.

NOW ADD NEW DISK :

15 / 18
1 SQL> ALTER DISKGROUP ARC_LOG ADD
DISK '/dev/oracleasm/disks/ARC_LOG02'
NOFORCE ;

You can verify the rebalancing operation by below SQL command.

1 SQL> select * from v$asm_operation ;

3 GROUP_NUMBER OPERA STAT POWER


ACTUAL SOFAR EST_WORK EST_RATE
4 EST_MINUTES ERROR_CODE

5 ------------ ----- ---- ---------- ---------- ----------


---------- ----------------------------------

1 REBAL RUN 1 1 5046 5631 4780 0

STEP 6) CHECK NEW STATUS

16 / 18
1 SQL> select
GROUP_N
2 UMBER,NAME,STATE,TYPE
,TOTAL_MB,FREE_MB,VOTING_FILES from
3 v$asm_diskgroup;

5 GROUP_NUMBER NAME STATE TYPE


TOTAL_MB FREE_MB V
6
------------ -------------- ----------- ------ ----------
7 ---------- -

8 1 ARC_LOG MOUNTED EXTERN 204799


192963 N
9
2 DB_DATA MOUNTED EXTERN 614400
10 145525 N

11 3 OCR_VOTE MOUNTED EXTERN 51199


50803 Y
12

13
SQL> select GROUP_NUMBER,DISK_NUMBE
14 R,STATE,name,path
,MOUNT_STATUS,HEADER_STATUS from
15 v$asm_disk;

16

17 GROUP_NUMBER DISK_NUMBER STATE


NAME PATH MOUNT_S HEADER_STATU
18
------------ ----------- -------- --------------
19 --------------------------------- ------- ------------

20 2 0 NORMAL ARC_LOG_0000
/dev/oracleasm/disks/ARC_LOG01 CACHED
21 MEMBER

22 1 0 NORMAL OCR_VOTE_0000
/dev/oracleasm/disks/OCR_VOTE01 CACHED

17 / 18
23 MEMBER

3 4 NORMAL DB_DATA_0004
/dev/oracleasm/disks/DB_DATA05 CACHED
MEMBER

3 3 NORMAL DB_DATA_0003
/dev/oracleasm/disks/DB_DATA04 CACHED
MEMBER

3 2 NORMAL DB_DATA_0002
/dev/oracleasm/disks/DB_DATA03 CACHED
MEMBER

3 1 NORMAL DB_DATA_0001
/dev/oracleasm/disks/DB_DATA02 CACHED
MEMBER

3 0 NORMAL DB_DATA_0000
/dev/oracleasm/disks/DB_DATA01 CACHED
MEMBER

3 5 NORMAL DB_DATA_0005
/dev/oracleasm/disks/DB_DATA06 CACHED
MEMBER

2 1 NORMAL ARC_LOG_0001
/dev/oracleasm/disks/ARC_LOG02 CACHED
MEMBER

9 rows selected.

18 / 18
Powered by TCPDF (www.tcpdf.org)