Oracle 11gR2 RAC

mplementation & Concep

- Santosh
Kangane

Santosh Kangane Oracle DBA

santoshkangane.blogspot.in
www.linkedin.com/pub/santosh-kangane/1b/78b/6b

2012. Santosh Kangane

Highlights

System Requirement
Oracle 11gR2 RAC overview
Storage Option
SAN
NAS
SCAN
OCR, Voting Disk and Time Synchronisation
Oracle ASM
Oracle Enterprise manager for RAC
System Requirement

Oracle Certified Operating Systems for RAC :

Linux :
Red Hat Enterprise Linux : 4.7, 5.2, 5.5, 6
SUSE Linux Enterprise Server : 10 SP2, 11
Oracle Enterprise Linux : All versions
Oracle Solaris ( Most popular option for production systems )
Windows
* All Node must have same OS and Kernel Version

Memory Requirement for Linux platform:

RAM Swap Space Required

Between 1 GB and 2 1.5 times the size of the RAM
GB
Between 2 GB and 16 Equal to the size of the RAM
GB
More than 16 GB 16 GB
Shared Memory : Minimum 40% of RAM; Should be grater than
size of SGA
Oracle 11g Real Application Cluster
 Oracle 11g RAC PEG Lab setup

RAC Node 1
(PTS0009)
RHEL 5.0

eth0 (public
eth1 (private IP)
IP) 10.88.33.23
10.88.32.102 PTS0009-vip
10.88.33.28 LAN
eth0 (public
eth1 (private IP)
Storage IP) 10.88.33.22
10.88.32.101 PTS0006-vip DNS
Server 10.88.33.27
(PTS0012) ORARAC-SCAN
10.88.33.24
10.88.33.25
10.88.33.26
3 IPs with round robin
RAC Node 2 allocation mode
(PTS0006)
RHEL 5.0 Santosh Kangane
2012.
Oracle 11g Real Application Cluster Stack

3
7
6
5
4
2
1

* Number shows the startup

Storage Options
NAS and SAN
 IP Based Storage Area Network Configuration

Standard configuration options

- Tool provided by Storage vendors
- OpenFiler Open source DOS
- Manual Setup using iSCSI protocol

Target (Ubuntu 12.04 on PTS0012)

Install iscsitarget, iscsitarget-source, iscsitarget-dkms package
Format the disk and create the partitions as per your requirements. ( I
have 10 GB for OCR &Voting Disk; 61 GB for oracle storage).
edit /etc/default/iscsitarget. Change the default value of
ISCSITARGET_ENABLE=false over to ISCSITARGET_ENABLE=true.
Define LUNs in /etc/iet/ietd.conf
Target iqn.2012-08.in.co.persistent:storage.disk0. pts0012
LUN 0
Path=/dev/cciss/c0d1p5,Type=fileio,ScsiId=lun0,ScsiSN=lun0
Node :
0th LUN is mandatory.
Make sure there is no space in LUN Definition above and every
parameter is separated by ,

Restart the service iscsitarget

IP Based Storage Area Network

Initiator ( All RAC node RHEL5)

Install iscsi-initiator-util package.
(Make sure that you have iscsi version >= iscsi-initiator-utils-
6.2.0.868-0.7.el5)
Edit /etc/iscsi/iscsid.conf and set node.startup = automatic
Then restart the initiator: /etc/init.d/iscsi restart
Make sure iscsid service is configured to start at system boot up
$ chkconfig --list iscsid
iscsid 0:off 1:off 2:on 3:on 4:on 5:on
6:off
Using CHAP authentication for security : Edit /etc/iscsi/iscsid.conf
Enable following entries :
node.session.auth.authmethod = CHAP
node.session.auth.username = <<sanusername>>
node.session.auth.password = <<sanpassword>>

Discover the target using command

$ iscsiadm m discovery p <<Target IP>>:3260
View detected nodes
$ iscsiadm m node
IP Based Storage Area Network

Initiator ( All RAC node RHEL5) cont

Login to the LUNs ( use same sequence to login on all nodes so
that the attached devices names will be identical )
$iscsiadm -m node --targetname <<inq name of
first LUN>>" --portal << Target IP >>:3260" --login
Verify the sessions
$ iscsiadm -m session -P3 | less
Now in $fdisk l all newly attached iscsi drives will be shown
( make sure every LUN of storage has got same Device name on
all RAC nodes)
Format newly added drives from any ONE node and use
$partprobe to push partition table on all other nodes
Verify on all nodes if newly attached drives are showing valid
portions using
fdisk -l
For more details on Storage Initiator setup and RAC Ethernet tuning
refer :
(
http://www.scribd.com/doc/46183679/Red-Hat-Linux-v5-x-Software-iSC
SI-Initiator-Configuration-MPIO-and-Tuning-Guide#outer_page_5
 Oracle RAC 11gR2 Single Client Access Name (SCAN)

How do I connect to a cluster ?

How to configure Load balancing?
How to handle Node failure?
Who to isolate node addition/removal without
affecting client connections and downtime ?

Single client access name (SCAN) is the virtual hostname (of 1-15
Char) to provide for all clients connecting to the cluster and It should
be unique across the network domain.

SCAN is registered with DNS (or Grid Naming Service) with at least one
and up to three IP addresses from the same subnet as that of Public
and VIP address.

These IP addresses should be allocated in round robin fashion

 Oracle RAC 11gR2 Scan Single Client Access Name (SCAN)

1. On each Node there is pair of

SCAN Listener & Local Listener
4 3 SCAN IP address will be mapped
3 as:
PEGLAB . 3 Node : Each node will be
1 mapped with one SCAN IP
. More than 3 node : only 3
nodes will be mapped to SCAN
2 IPs
. 2 Node : One node will have
5 double SCAN IP & SCAN
2. Listener
PMON process of each
and other instance
node will
SQL> conn PEGLAB reportsingle
have the current
IP and work
SCANload to
user/pwd@orarac- 2 SCAN listener Service
Listener
scan/peglab (specified in
theREMOTE_LISTENERdatabas
6 e parameter.)

3. On connect request DNS resolves the orarac-scan and returns list of 3 SCAN IP addresses
4. Client select the first IP from the list and connect to the RAC; if it fails then request again with
next IP.
5. Depending on the Load SCAN listener re-direct the request to Local listener of lightly loaded
node. ( Note here in case
of More than 3 node cluster, this is how request can still be given to lightly loaded node via
SCAN listener service, even
6. Now all further
if it does communication
not have happens
SCAN IP mapped ) through local listener directly.
OCR And Voting Disk

Oracle Cluster Registry:

The OCR maintains cluster configuration information that is used by
each node of the cluster to determine the state of the cluster.
OCR also maintains information about cluster resources:
Databases
Instance
Services
Each node in the cluster maintains a copy of the OCR in memory for
better performance and also responsible for updating the OCR as
required in shared storage.
Voting Disk:
The Voting Disk Files are used by Oracle Clusterware to determine
which nodes are currently members of the cluster. The voting disk files
are also used in concert with other Cluster components such as CRS to
maintain the clusters integrity.
Auto Segment Management : Architecture

PEGLAB RDBMS PEGLAB

1 Instances 2

+ASM ASM +ASM

1 Instances 2

CLUSTERWARE Clusterware CLUSTERWARE

PEGLA
B

Database
Files
ASM Disk Groups, Disks and Database Files

Disk Disk Disk

Group 1 Group 2 Group
3

File 3 File 2

Fil
e1
File 4 File 5

File 5 File 6
Disk
3
Disk Disk
1 4 Disk Disk Disk Disk
2 5 6 7
ASM Failure Groups - External Redundancy

Disk Group

Disk Disk Disk

1 2 3
ASM Failure Groups Normal Redundancy

Disk Group

Failure Group 1 Failure Group 2

Disk Disk Disk Disk Disk Disk

1 2 3 4 5 6
ASM Failure Groups High Redundancy

Disk Group

Failure Group 1 Failure Group 2 Failure Group 3

Disk Disk Disk Disk Disk Disk

1 2 1 2 1 2
Extent Distribution External Redundancy 1 Disk

Metada
ta

0
1
2
3
4
5
6
7

Disk 0

Metada Data
ta
Extent Distribution External Redundancy 2 Disk

1
3
5
Metada 7
ta 9
11
13
0 15
2
4
6
8
10
12
14

Disk 0 Disk 1

Metada Data
ta
Extent Distribution Normal Redundancy 2 Disk

Failure Group 1 Failure Group 2

0 1 0 1
3 2 3 2
4 5 4 5
7 6 7 6
0 1 0 1
3 2 3 2
4 5 4 5
7 6 7 6
8 9 8 9
11 10 11 10
12 13 12 13
15 14 15 14
16 17 16 17
19 18 19 18
20 21 20 21
23 22 23 22

Disk 0 Disk 1 Disk 2 Disk 3

Metada Primar Secondar
ta
Data y
Primar y
Secondar
y y
Extent Distribution High Redundancy 1 Disk

Failure Group 1 Failure Group 2 Failure Group 3

0 0 0
1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
0 0 0
1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
6 6 6
7 7 7
8 8 8
9 9 9

Disk 0 Disk 1 Disk 2

Metada Primar Secondar Tertiar
ta
Data y
Primar y
Secondar y
Tertiar
y y y
 RAC Implementation Step By Step

Document Explains the step by steps

executions and seen dumps for
- Pre-requisite Linux packages installation
Oracle 11gR2 RAC - Oracle 11gR2 Clusterware installation
Setup on RHEL 5.0 - Oracle 11gR2 Database software

installation
- Creation of ASM Disk group
- Creation of Oracle 11g Database on
Cluster
 References

1. http://docs.oracle.com/cd/E11882_01/install.112/e24326/toc.h
tm
2. http://www.oracle.com/technetwork/products/clustering/overv
iew/scan-129069.pdf
3. https://support.oracle.com
4. http://www.scribd.com/doc/46183679/Red-Hat-Linux-v5-x-Softw
are-iSCSI-Initiator-Configuration-MPIO-and-Tuning-Guide#out
er_page_5
5. http://www.oracledba.org/11gR2/Pre_Install_11gR2.htm
6. http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOrac
le/tabid/648/TopicID/RACR2ARC6/Default.aspx
7. http://juliandyke.com/Presentations/Presentations.html

25
Thank You !
santoshkangane.blogspot.com