ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM

Alejandro Vargas | Principal Support Consultant
Oracle Advanced Customer Services

INDEX
Part 1 : Configure devices for ASM using file system files and loop devices..........................................................................2
Check the amount of space available on file system....................................................................................................2
Create a directory to hold the files to be used as ASM devices...................................................................................2
Create the files to be used as Asm devices..................................................................................................................2
Check the created files..................................................................................................................................................3
Connect to root and check if there are loop devices
......................................................................................................................................................................................3
As root create the loop devices
.......................................................................................................................................................................................4
Bind the loop devices to raw devices
.......................................................................................................................................................................................4
Check the raw devices binding
.......................................................................................................................................................................................4
Setup raw devices permissions and ownership
.......................................................................................................................................................................................4
Part 2 : Configure devices for ASM using LUNs.....................................................................................................................6
Identify which devices are being used by the mounted file systems............................................................................6
Identify available LUNs and LUNs sizes.....................................................................................................................6
Create a single partition per LUN using fdisk................................................................................................................6
Create single partitions on evary other LUN ................................................................................................................8
Change LUNs Ownership to Oracle and Set Permissions...........................................................................................9
Part 3 : Configure File System Files to be Used as ASM Disks............................................................................................11
Create a directory to store the files that will be used as ASM DISKS.......................................................................11
Create the files to be used as ASM DISKS.................................................................................................................11
Change Ownership and Permissions of the ASM Files..............................................................................................12
Setup ASM instance to Use File System Files............................................................................................................12
1/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM

Part 1 : Configure devices for ASM using file system files and loop devices
Check the amount of space available on file system
[oracle@asmxpt ~]$ df -h
Filesystem
Size
/dev/sda3
18G
/dev/sda1
76M
tmpfs
633M

Used Avail Use% Mounted on

9.8G 6.8G 59% /
9.9M
63M 14% /boot
0 633M
0% /dev/shm

Create a directory to hold the files to be used as ASM devices

[oracle@asmxpt ~]$ mkdir /u01/app/asm_loop_devices
[oracle@asmxpt ~]$ ls -l /u01/app/
total 12
drwxr-xr-x 2 oracle dba 4096 Jan 31 11:31 asm_loop_devices
drwxr-xr-x 7 oracle dba 4096 Jan 26 09:34 oracle
drwxrwx--- 6 oracle dba 4096 Jan 24 18:29 oraInventory

Create the files to be used as Asm devices

[oracle@asmxpt ~]$ cd /u01/app/asm_loop_devices
[oracle@asmxpt asm_loop_devices]$ \
? dd if=/dev/zero of=/u01/app/asm_loop_devices/fs_dsk1 bs=1M count=500
500+0 records in
500+0 records out

2/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
524288000 bytes (524 MB) copied, 10.9365 seconds, 47.9
[oracle@asmxpt asm_loop_devices]$ \
? dd if=/dev/zero of=/u01/app/asm_loop_devices/fs_dsk2
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 17.6032 seconds, 29.8
[oracle@asmxpt asm_loop_devices]$ \
? dd if=/dev/zero of=/u01/app/asm_loop_devices/fs_dsk3
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 12.8595 seconds, 40.8
[oracle@asmxpt asm_loop_devices]$ \
? dd if=/dev/zero of=/u01/app/asm_loop_devices/fs_dsk4
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 12.4761 seconds, 42.0

Check the created files

[oracle@asmxpt asm_loop_devices]$
total 2050016
-rw-r--r-- 1 oracle dba 524288000
-rw-r--r-- 1 oracle dba 524288000
-rw-r--r-- 1 oracle dba 524288000
-rw-r--r-- 1 oracle dba 524288000

ls -ltr
Jan
Jan
Jan
Jan

31
31
31
31

12:39
12:42
12:43
12:44

Connect to root and check if there are loop devices

[oracle@asmxpt asm_loop_devices]$ su Password:
[root@asmxpt ~]# raw qa

3/13

fs_dsk1
fs_dsk2
fs_dsk3
fs_dsk4

MB/s
bs=1M count=500
MB/s
bs=1M count=500
MB/s
bs=1M count=500
MB/s

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
As root create the loop devices
[root@asmxpt
[root@asmxpt
[root@asmxpt
[root@asmxpt

~]#
~]#
~]#
~]#

losetup
losetup
losetup
losetup

/dev/loop1
/dev/loop2
/dev/loop3
/dev/loop4

/u01/app/asm_loop_devices/fs_dsk1
/u01/app/asm_loop_devices/fs_dsk2
/u01/app/asm_loop_devices/fs_dsk3
/u01/app/asm_loop_devices/fs_dsk4

Bind the loop devices to raw devices

[root@asmxpt ~]# raw /dev/raw/raw1 /dev/loop1
/dev/raw/raw1: bound to major 7, minor 1
[root@asmxpt ~]# raw /dev/raw/raw2 /dev/loop2
/dev/raw/raw2: bound to major 7, minor 2
[root@asmxpt ~]# raw /dev/raw/raw3 /dev/loop3
/dev/raw/raw3: bound to major 7, minor 3
[root@asmxpt ~]# raw /dev/raw/raw4 /dev/loop4
/dev/raw/raw4: bound to major 7, minor 4

Check the raw devices binding

[root@asmxpt ~]# raw -qa
/dev/raw/raw1: bound to
/dev/raw/raw2: bound to
/dev/raw/raw3: bound to
/dev/raw/raw4: bound to

major
major
major
major

7,
7,
7,
7,

minor
minor
minor
minor

1
2
3
4

Setup raw devices permissions and ownership

[root@asmxpt ~]# chmod 660 /dev/raw/raw[1-4]

4/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
[root@asmxpt ~]# chown oracle:dba /dev/raw/raw[1-4]
[root@asmxpt
crw-rw---- 1
crw-rw---- 1
crw-rw---- 1
crw-rw---- 1

~]# ls
oracle
oracle
oracle
oracle

-l /dev/raw/raw[1-4]
dba 162, 1 Jan 31 12:54
dba 162, 2 Jan 31 12:54
dba 162, 3 Jan 31 12:54
dba 162, 4 Jan 31 12:54

/dev/raw/raw1
/dev/raw/raw2
/dev/raw/raw3
/dev/raw/raw4

Note that after reboot the ownership and permisions will change back to root and -rw-r--r--, they can be setup on file
/etc/rc.local to be executed at startup:
echo "chmod 660 /dev/raw/raw[1-4]" >>/etc/rc.local
echo "chown oracle:dba /dev/raw/raw[1-4]" >>/etc/rc.local

To delete a loop device use the following sintax: losetup d /dev/loopX

5/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
Part 2 : Configure devices for ASM using LUNs
Identify which devices are being used by the mounted file systems
[root@asmxpt ~]# df -k
Filesystem
1K-blocks
/dev/sda3
18253712
/dev/sda1
77749
tmpfs
647744

Used Available Use% Mounted on

12243352
5068148 71% /
10095
63640 14% /boot
0
647744
0% /dev/shm

Identify available LUNs and LUNs sizes

[root@asmxpt ~]# fdisk -l | grep -v /dev/sda | grep ^Disk
Disk /dev/sdb: 2147 MB, 2147483648 bytes
Disk /dev/sdc: 2147 MB, 2147483648 bytes
Disk /dev/sdd: 2147 MB, 2147483648 bytes
Disk /dev/sde: 2147 MB, 2147483648 bytes
Disk /dev/sdf: 2147 MB, 2147483648 bytes
Disk /dev/sdg: 2147 MB, 2147483648 bytes
Disk /dev/sdh: 2147 MB, 2147483648 bytes
Disk /dev/sdi: 2147 MB, 2147483648 bytes

Create a single partition per LUN using fdisk

Discussions about LUN alignement can be found at:
Oracle Database 10g Release 2 Automatic Storage Management Overview and Technical Best Practices
May 2007
Author: Nitin Vengurlekar
Contributing Authors: Rich Long, Ara Shakian

6/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
And
Aligning ASM Disks on Linux
by Christo Kutrovsky
http://www.pythian.com/blogs/411/aligning-asm-disks-on-linux
[root@asmxpt ~]# fdisk /dev/sdb
Command (m for help): p
Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot

Start

End

Blocks

Id

System

Command (m for help): u

Changing display/entry units to sectors
Command (m for help): p
Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot

Start

Command (m for help): n

Command action
e
extended
p
primary partition (1-4)
p
Partition number (1-4): 1

7/13

End

Blocks

Id

System

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
First sector (63-4194303, default 63): 2048 <<<< Start at 1M
Last sector or +size or +sizeM or +sizeK (32768-4194303, default 4194303):
Using default value 4194303
Command (m for help): p
Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot
Start
End
Blocks
/dev/sdb1
2048
4194303
2080768
Partition 1 does not end on cylinder boundary.

Id
83

System
Linux

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Create single partitions on evary other LUN

After repeating the previous step for each LUN available for ASM you will have these partitions:
[root@asmxpt ~]# fdisk -l | grep /dev/sd | grep -v sda
Disk /dev/sdb: 2147 MB, 2147483648 bytes
/dev/sdb1
6
11916
2096128
83
Disk /dev/sdc: 2147 MB, 2147483648 bytes
/dev/sdc1
6
11916
2096128
83
Disk /dev/sdd: 2147 MB, 2147483648 bytes
/dev/sdd1
6
11916
2096128
83
Disk /dev/sde: 2147 MB, 2147483648 bytes
/dev/sde1
6
11916
2096128
83
8/13

Linux
Linux
Linux
Linux

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
Disk /dev/sdf:
/dev/sdf1
Disk /dev/sdg:
/dev/sdg1
Disk /dev/sdh:
/dev/sdh1
Disk /dev/sdi:
/dev/sdi1

2147 MB, 2147483648 bytes

6
11916
2147 MB, 2147483648 bytes
6
11916
2147 MB, 2147483648 bytes
6
11916
2147 MB, 2147483648 bytes
6
11916

2096128

83

Linux

2096128

83

Linux

2096128

83

Linux

2096128

83

Linux

Change LUNs Ownership to Oracle and Set Permissions

ASM can access block devices directly by setting the addequate ownership and permissions on them. Block devices can
be used when there is no ASMLib available.
[root@asmxpt
brw-r----- 1
brw-r----- 1
brw-r----- 1
brw-r----- 1
brw-r----- 1
brw-r----- 1
brw-r----- 1
brw-r----- 1
[root@asmxpt
[root@asmxpt
[root@asmxpt
brw-rw---- 1
brw-rw---- 1
brw-rw---- 1
brw-rw---- 1
9/13

~]# ls -l /dev/sd?1 | grep -v sda

root disk 8, 17 Jan 31 19:12 /dev/sdb1
root disk 8, 33 Jan 31 19:17 /dev/sdc1
root disk 8, 49 Jan 31 19:18 /dev/sdd1
root disk 8, 65 Jan 31 19:19 /dev/sde1
root disk 8, 81 Jan 31 19:19 /dev/sdf1
root disk 8, 97 Jan 31 19:21 /dev/sdg1
root disk 8, 113 Jan 31 19:23 /dev/sdh1
root disk 8, 129 Jan 31 19:23 /dev/sdi1
~]# ls /dev/sd?1 | grep -v sda | xargs chown oracle:dba
~]# ls /dev/sd?1 | grep -v sda | xargs chmod 660
~]# ls -l /dev/sd?1 | grep -v sda
oracle dba 8, 17 Jan 31 19:12 /dev/sdb1
oracle dba 8, 33 Jan 31 19:17 /dev/sdc1
oracle dba 8, 49 Jan 31 19:18 /dev/sdd1
oracle dba 8, 65 Jan 31 19:19 /dev/sde1

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
brw-rw---brw-rw---brw-rw---brw-rw----

10/13

1
1
1
1

oracle
oracle
oracle
oracle

dba
dba
dba
dba

8, 81 Jan
8, 97 Jan
8, 113 Jan
8, 129 Jan

31
31
31
31

19:19
19:21
19:23
19:23

/dev/sdf1
/dev/sdg1
/dev/sdh1
/dev/sdi1

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
Part 3 : Configure File System Files to be Used as ASM Disks
The use of file system files as ASM disks is implemented using the parameter _asm_allow_only_raw_disks = FALSE
Details can be found on Metalink Note: 293095.1
Create a directory to store the files that will be used as ASM DISKS
[oracle@asmxpt ~]$ mkdir /u01/app/asm_filesystem_devices

Create the files to be used as ASM DISKS

[oracle@asmxpt ~]$ dd if=/dev/zero of=/u01/app/asm_filesystem_devices/asm_filesystem_dsk1
bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 12.951 seconds, 40.5 MB/s
[oracle@asmxpt ~]$ dd if=/dev/zero of=/u01/app/asm_filesystem_devices/asm_filesystem_dsk2
bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 12.7267 seconds, 41.2 MB/s
[oracle@asmxpt ~]$ dd if=/dev/zero of=/u01/app/asm_filesystem_devices/asm_filesystem_dsk3
bs=1M count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 11.3377 seconds, 46.2 MB/s
[oracle@asmxpt ~]$ dd if=/dev/zero of=/u01/app/asm_filesystem_devices/asm_filesystem_dsk4
bs=1M count=500
11/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 8.0571 seconds, 65.1 MB/s
Change Ownership and Permissions of the ASM Files
[oracle@asmxpt ~]$ chown oracle:dba /u01/app/asm_filesystem_devices/*
[oracle@asmxpt ~]$ chmod 660 /u01/app/asm_filesystem_devices/*
[oracle@asmxpt ~]$ ls -l /u01/app/asm_filesystem_devices/
total 2050016
-rw-rw---- 1 oracle dba 524288000 Jan 31 19:58 asm_filesystem_dsk1
-rw-rw---- 1 oracle dba 524288000 Jan 31 19:58 asm_filesystem_dsk2
-rw-rw---- 1 oracle dba 524288000 Jan 31 19:58 asm_filesystem_dsk3
-rw-rw---- 1 oracle dba 524288000 Jan 31 19:58 asm_filesystem_dsk4
Setup ASM instance to Use File System Files
This part of the hands-on is to be done after creating the ASM Instance on Lab 2
In order to create a database with this type of files we need to set the following parameters on the ASM pfile

Include <path> into asm_diskstring parameter.

Set on init+ASM.ora asm_diskstring=ORCL:*,/u01/app/asm_filesystem_devices

If using spfile
Alter system set asm_diskstring=ORCL:*, /u01/app/asm_filesystem_devices

12/13

ASM HANDS-ON TRAINING

Lab 1 Configuration of Devices for ASM
The instance has to be restarted with parameter _asm_allow_only_raw_disks = FALSE.

Check view v$asm_disk to see if the new disks asm_filesystem_dsk* was discovered:
Select header_status,path from v$asm_disk;
Kfod can be also used:
kfod p='$ORACLE_HOME/dbs/init\+ASM.ora'

Create a diskgroup using the new file

Create diskgroup datadg_fs disk /u01/app/asm_loop_devices/fs_dsk1;
select TOTAL_MB, FREE_MB, NAME from v$asm_diskgroup;

Create a tablespace using this diskgroup:

create tablespace ts_fs datafile +dg_fs;

End of Lab1

13/13