Oracle Gua
sep epepeempeepee Be«ces
DBA Tips Archive for Oracle
(Home ]
Add a Node to an Existing Oracle RAC 10g R2 Cluster on Linux - (RHEL 4.5)
by Jeff Hunter, Sr. Database Administrator
1. Overview
2. Hardware and Costs
3, Install the Linux Operating System
4. Network Configuration
5. Configure Network Security on the Opentiler Storage Server
6. Configure the iSCSI Initiator
7. Create “oracle” User and Directories
8. Configure the Linux Server for Oracle
9. Configure the "hangcheck-tiner” Kernel Module
10. Configure RAC Nodes for Remote Access using SSH
LL, All Startup Commands for New Oracle RAC Node
12: Install and Configure Oracle Cluster File System (OCFS2)
13, Install and Configure Automatic Storage Management (ASMLIb 2.0)
14, PresInstallation Tasks for Oraclel0g Release 2
15. Extend Oracle Clusterware Software to the New Node
16. Extend Oracle Database Software to the New Node
17. Add Listener to New Node
18, Add Database Instance to the New Node
19, About the Author
Overview.
‘As your organization grows so too does your need for more application and database resources to suppor the company’s IT systems.
Oracle RAC 10g provides a scalable framework which allows DBA’s to effortlessly extend the database tier to support this increased
‘demand. As the number of users and transactions increase, addtional Oracle instances can be added tothe Oracle database cluster to
distribute the extra load
‘This document is an extension to my article "Building an Inexpensive Oracle RAC 10g Release 2 on Linux - (CentOS 4.5 /iSCSI)"
Contained inthis new atile are the steps required to adda single node to an already running and configured two-node Oracle RAC
10g Release 2 envizonment on the CentOS 32-bit (x86) platform. Although this article was writen and tested on CentOS 4.5 Linux, it
should work unchanged with Red Hat Enterprise Linux 4 Update 5
‘This article assumes the Following:
A The reader has already built and configured a two-node Oracle RAC 10g Release 2 environme
the article "Building an Inexpensive Oracle RAC 10g Release 2 on Linux - (CentOS 4.5 /iSCSI
artile provides eomprshensive instructions for building a two-node RAC cluster, cach with a single
processor running CentOS 4.5, Oracle RAC 10g Release 2, OCFS2, and ASMLib 2.0, The curent
twwo-nade RAC environment actully consists of tree machines — two named Minwxd and Lénx2
whieh each run an Oracle10g instance and a third node to run the network storage server named
opentilert.
Note: The current two-node Oracle RAC environment has becn upgraded from its base release
(10.2.0.1.6) to version 10.2.0.3.0 by applying the 5337014 patehset (95237034, 16202_L1Wx.2ip). The
patchset was applied to Oracle Clusterware aad the Oracle Database software. also applied the one-
oft patchset- "BUNDLE Patch for Base Bug 6000740" (MLRT ON TOP OF 10.2.03) tothe Oracle
‘Clusterware and Oracle Database software. The procedures for installing both patchsets are not
included in any of the parent article(s).
YTomaintan the current naming convention,
cluster will be named Hoe (running a new instance
he new Oracle RAC node to be added tothe existing
smed ore13) making it a three-node cluster
‘Y The new Oracle RAC node should have the same operating system version and installed patches as
the curent two-node cluster
‘Each node in the existing Oracle RAC clusters copy ofthe Oracle Clsterware and Oracle
Database software installed on their cel disks. The cumentwo-node Oracle RAC envionment does not use shared Oracle homes for the Clusterware of Database software
‘Y The software owner for the Oracle Clusterware and Oracle Database installs will be "orseie", It is
important thatthe UID and GID ofthe oracie user account be identical to that ofthe existing RAC
‘nodes. For the purpose of this example, the oracie user account will be defined a flows:
[oracieplinuxt ~1$ id oracle
Uulassei(oracie) gié-50a(oinstat) grovps-sei(oinstai1),3@2(cba)
A Thoexisting Oracle RAC 10g environment makes use ofa clustered filesystem (OCES2) to store the
‘two files required fo be shared by Oracle Clusterwate; namely the Oracle Cluster Registry (OCR) file
snd the Voting Disk. Instructions for installing and adding the new Oracle RAC nods tothe "ive"
(OCES? filesystem will be included,
A Automatic Storage Management (ASM) is being used asthe file system and volume manager forall
Oracle physical database files (data, online redo logs, contol files, archived redo logs) and a Flash
Recovery Area, In addition to ASM, we wil also be configuring ASMLib on the new Oracle RAC.
node.
‘To add instances oan existing RAC database, Oracle Corporation recorsmends using the Oracle
cloning procedures which s described in the Oracle Universal Installer and OPatch Users Guide
“Tis atele, however, ues manval procedures to add odes and instances to the existing Oracle RAC
cliser The manvalproedres method describe inthis acl invalvs extending the RAC databace
by fnt extending the Oracle Cluserware Rome fo the new Oracle RAC node an then extending the
Oracle Database home. nother word, you extend he software ono the new node in these order
‘as youinaled he clusterware and Oracle database software components on the existing wo-node
RAC.
Y _Dusing the eeation ofthe existing two-node cluster, the installation of Oracle Chsterware and the
Oracle Database software were only performed from one node inthe RAC cluster —namely froma
{sms as the oracle user account. The Oracle Universal Installer (OUD) on that particular node would
‘then use the san and sep commands to run emote commands on and copy files (the Oracle software)
toall other nodes within the RAC cluster. The oracte user account on the node runing the OUL
(Cuntnstatier) had to be rusted by sll other nodes in the RAC cluster. This meant thatthe oracle
user account had to run the secure shell commands (ssh o sep) on the Linux sever executing the OUL
(Qinusa) against al other Linux servers in the cluster without being prompted fora password. The
Sanne security requirements hold tue for this tile. User equivalence willbe configured s thatthe
Oracle Chusterware and Oracle Database software willbe Securely copied from lin tothe new
Oracle RAC node (Linux) using ssh and sep without being prompted for a password.
AY Allshared disk storage forthe existing Oracle RAC is hase on iSCSI using a Network Storage
Server namely Opener Release 2.2 (eespin 2) Powered by «Pa Linux, Opener ia fre browser-
‘based network storage management uti that delivers file-based Network Atached Storage (NAS)
and block-based Storage Area Networking (SAN) ina single framework. Opeafilr supports CIFS,
NFS, HTTP DAV, FTP, however, we will only be making use of iSCSI capabilities to implement
1 inexpensive SAN forthe shared storage components roqired by Oracle RAC 10g. This solution
offers a low-cost altemative to fibre channel for testing and edacaional purpose, but given the lo=
cau hardware being used, it should not be used ina production envzonment
A These articles provide a low cost alternative for those who want to become familar with Oracle RAC
10g using commercial off the self components and downloadable software. Bear in mind that these
articles are provided for educational purposes only so the setup is kept simple to demonstrate ideas
and concepts. For example, the disk mirroring configured in this article will be setup on one physical
disk only, while im practice that should be performed on multiple physical drives. In addition, each
Linux node will only be configured with two network cards — one forthe public network (et0) and
‘one for the private cluster interconnees “and” network storage server for shared iSCSI access (eth),
For a production RAC implementation, the private interconnect should be st least gigabit (or more)
and “only” be used by Oracle to transfer Custer Manager and Cache Fusion related dat A third
dedicated network interface (i.e. eth2) should be configured on another gigabit network for access to
the network storage server (Opeatile).
‘The following isa conceptual look at what the environment wil look like afler adding the third Oracle RAC node (23m) to the
cluster. Click onthe graphic below to enlarge the image: tenet ren 9
Se, ee,
Figure 1: Adding line tothe current Oracle RAC 10g Release 2 Environment
RAC Lg system, itis by no means a substitute for the offcial Oracle documentation, In addition to thi article,
‘users should wso consult the following Oracle documents to gin a full understanding of allemative
configuration options, installation, and administration with Oracle RAC 10g. Oracle’ official documentation
site is does.oracle.com,
B ‘While this article provides comprehensive instructions for successfully adding anode wo an existing Oracle
WB onteCtastervare and Oracle Real Application Clusters Isalation Guide 1g Release? (10.2) or
Liner
TB Ontectuserware and Oracle Rel Applicaton Clusters Administration and Deployment Guide - 7g
Release? (102)
2 Day + Real Application Clusters Guide 10g Release 210.2)
Hardware and Costs
‘The hardware used in this article to build the third node (2inux3) consists ofa Linux workstation and components which can be
purchased at many local computer stores or over the Interne,
Oracle RAC Node 3 - (linux3)
3 ot Dimension 000 Series
Intl) Pena) 4 Pocesorat2 802
2G8 DDR SDRAM (a 333M)
00187200 RPM Intra iad Dave
Invented ne! 3D AGP Gpbies
Integrate 10/00 Eeret-(roadcom BCM6A01)
CDROM (48X Max Vane)
3.8" Floppy
[No Keyout, Monitor or Mouse (Connected to KVM Sith) ssso0
DL teraet LAN
Esch Linux sever for Oncle RAC should contain two NIC adapters. The Dell Dimeasion
tncindesan integrated 10/100 Etemet adapter that wil be wed to connect he public
‘network. The send NIC adapter wll buted forthe private network (RAC infreonnet and
pence actworked song) Sele! he appropite NIC apr xcrmpatibl with he
rmximam data taasmsson sped ofthe nctwork sith o Be usod forthe pvate network. For
thepurpote ofthis ance, used a Gigabit here switch (nd 1G Ehret cards) forthe
praetor
Used or RAC inteconnest linux, aux? and Opeatler networked some
Gigabit ibernss
ussss
10/100/1000Ntps CI Desktop Adaper-(PWLABISIGT) Ne 2 network Cables
usss
> category 5 patch able (Connect inux3 to pubic network) sss
Category Se atch eable (Comet nux3 to atewonnect ethemet switch)
We ate about to start the installation process. As we start to go into the details ofthe installation, it should he noted that most ofthe
‘asks within this docurnent will need tobe performed on the new Oracle RAC node (4nux3) [wil indicate a the beginning af each
section whether or nat the task{s) should be performed on the new Oracle RAC node, the current Oracle RAC node(s), oF on the
network storage server (opener),
Install the Linux Operating System
[Preto hc otoving nation on th ew Ora RAC ade
‘After procuring the required hardware, ts time to start the configuration process. The first tsk we need to perform i o install the
“Linux operating system, As alzeady mentioned, ths article will use CentOS 4.5, Although I have used Red Hat Fedora in the past 1
wanted to switch to a Linux environment that would guarantee all ofthe functionality contained with Oracle. This is where CentOS.
comes in, The CentOS project takes the Red Hat Enterprise Linux 4 source RPMs and compiles them into a free clone ofthe Red Hat
Enterprise Server 4 product, This provides a free and stable version ofthe Red Hat Enterprise Linux 4 (AS/ES) operating environment
that ean now use for esting different Oracle configurations. Ihave moved away from Fedora as Ineed a stable environment that is
‘not only fee, but as close to the actual Oracle supported operating system as possible. While CentOS is not the only project
performing the same funetionaity Iter t stick with it 8 i is stable and reacts fast with regards to updates by Red Hat.
Downloading CentOS
Use the links (below) to download CentOS 4.5. After downloading CentOS, you will hen want to burn each ofthe ISO
images to CD.
® centos.org
+ CemOS-4.5.i386-binloft iso (622 MB)
+ GemtOS-4.5.:386.bin20f¢ 80 (636 MB)
+ CemOS-4.52:386-bin3oF¢ iso (638 MB)
+ CemOS-4'5-:386-bindoft iso (313 MB)
you are downloading the above ISO files to a MS Windows machine, there are
‘many options for burning these images (ISO files) to a CD. You may alzeady be
‘amiliar with and have the proper software to bura images to CD. Ifyou are not
‘familiar wit this process and do not have the required software to bum images to
CD, here are just two (oF many) software packages that can be use:
@ Uitratso
© Magic ISO Maker
Installing CentOS,
‘This section provides a summary of the sereens used to install CentOS. For more detailed installation instructions tis
possible to use the manuals from Red Hat Linax hip: ww redhat com/dacs/manvals. l would suggest, however, thatthe
instructions have provided below be used for this Oracle RAC 10 configuration,
‘Before installing the Linux operating system on the new Oracle RAC node, you
should have the two NIC interfaces (cards) installed,
‘After downloading and burning the CentOS images (ISO files) to CD, insert CentOS Disk #1 into the new Oracle RAC
server (2inue inthis example), power it on, and answer the installation sereen prompts as noted below:
Boot Sereen
The frst sereen isthe CentOS boot screen. At the boot: prompt, ht [Enter] to start the installation process.
Media Test
When asked totes the CD media, tab over to [Ship] and hit [Enter]. If here were any evvors, the media
burning software would have warned us. After several seconds, the installer should then detect the video card
‘monitor, and mouse. The installer then goes into GUI mode, Welcome to Centos
At the welcome sereen, elick [Nest] to continue,
anguage / Keyboat
The next two sereens prompt you for the Language and Keyboard settings. In almost all case, you can accept
the defaults. Make the appropriate selection for your configuration and click [Nex] to continue
nstallation Type
(Choose the [Custom] option and click [Next] to continue.
Disk Partitioning Setup
Select [Automatically partition] and click [Next] continue.
If there were a previous installation of Linwx om this machin, the next screen will ask f you want to "remove
{or "keep” old partitions, Select the option to [Remove all partitions on ths system. Also, ensure thatthe
{hia} drive is selected for this installation. 1 also keep the checkbox [Review (and modify if needed) the
partitions created] selected. Click [Next] ta continue.
You wll hen be prompted witha dialog window asking ifyou really want to remove al partitions. Click
[Yes] ta acknowledge this warning.
‘The installer will then allow you to view (and modify if needed) the disk partitions it automaticly selected
For mast automatic laws, the installer will choose 100MB for boot, double the amount of RAM (stems
with <2GB RAM) oran amount equa o RAM (stems with > 2GB RAM) for sap. and the rest going to the
root () parton. Starting with EL, the installer wil eeate the same dik configuration as ust noted but will
‘reat hem using the Logical Volume Manager (LVM). For example, it wil partition the first hard drive
(dewhda for my configuration) into two parttions — one forthe oot partition (devihal) and the
remainder ofthe disk dedicate o a LYM named VolGroup00 (/dev/hda2). The LYM Volume Group
(YolGroupO0) isthe parttioned into no LYM partitions - one for the root flesptem (and another for
swap,
The main concern during the partitioning phase isto ensure enough swap space is allocated as required by
(Oracle (which is a multiple ofthe available RAM). The following is Oracle's requirement for swap space:
‘AvallableRAM | Swap Space Required
Between 1 GB and 2 GB|15 times the size of RAM
Between 2 GB and & GB [Fqual tothe size of RAM
‘More than § GB 75 times the size of RAM
For the purpose ofthis install, Iwill accept all automatically preferred sizes. (Including 2GB for swap since T
have 2GB of RAM installed )
If for any reason, the automatic layout does not configure an adequate amount of swap space, you can easily
‘change that rom this screen. To increase the size ofthe swap partition, [Edit] the volume group VolGroup00.
This wil Bring up the "Edit LYM Volume Group: VolGreup00" dialog. First, [Edi and decrease the size of
the root filesystem () by the amount you want to add fo the swap partion, For exwmple, to add another
512MB io swup, you would decrease the sizeof the root file system by 512MB (Le. 36,032MB - 512ME
35,320MB). Now add the space you decreased from the root filesystem (512MB) to the swap partition. When
completed, click [OK] on the "Edit LVM Volume Group: VolGroup00" dialog.
Once you are satisfied with the disk layout, lick [Next] to continue,
‘Boot Loader Configuration
The installer will use the GRUB boot loader by default. To use the GRUB bootloader, accept all default
values and click [Next] t0 continue
Network Configuration
| made sure to install oth NIC interfaces (cards) inthe new Linx machine before starting the operating
system installation. This soreen should have successfully detected each ofthe network devices
First, make sure that each ofthe network devices are checked to [Active on boot}. The installer may choose to
not activate ethl by default,
Second, (alit] both eth0 and eth! as follows. You may choose to use different IP addresses for both eth0 and
‘thi and that i OK. Canfigure eth! (he interconnect and storage network) an a different subnet than ethd
(the public network)
eth
= Check OFF the option to [Configure using DHCP]
= Leave the Activate on boot} checked ON IP Aaddrest: 192.168.1.107
= Netmask: 285.255.255.0
chk:
~ Check OFF the option to {Configure using DHCP]
= Leave the Activate on boot} checked ON
“IP Address: 192.168 2.107
= Netmask: 255.255.255.0
Continue by seting your hostname manually. used *linws3" for this new Oracle RAC node. Finish this
dialog off by supphsing your gateway and DNS servers.
Eirewall
(On this screen, make sure to selet [No firewall}. Also under the option to "Enable SELimux?", select
{Disabled} and click [Next] to continue.
You will be prompted with a warning dialog about not seting the firewall. [this occurs, simply hit [Proceed]
Additional Language Support / Time Zone
The next twa screens allow you to select additional language support and time zone information. In almast all
‘eases, ou can accept the defaults. Make the appropriate selection for your configuration and click [Next] to
continue
Sel Root
word
Select a root password and eck [Next] to continue,
Package Group Selection
Scroll down tothe bottom of this sereen and select [Everything] under the "Miscellaneous" section. Click
{Next] 10 continue
Please note that the installation of Oracle does not require all Linux packages to be installed. My decision to
instal all packages was forthe sake of brevity. Please see section "Pre-Insiallaion Tasks for Oraclel0g
Release 2"for a more detailed look at the critical packages required for a successful Oracle installation,
Also note that with some RHEL 4 distributions, you will not get the "Package Group Selection” screen by
“default, There, you are asked to simply "Install default software packages” oF "Customize software packages
to be installed’, Select the option to "Customize software packages to be installed” and click [Next] to
‘continue. This will then bring up the “Package Group Selection" screen. Now, scroll down tothe bottom of |
this screen and select [Eversthing} under the "Miscellaneous" section. Click [Next] to continue.
About to Install
This sereen is basically a confirmation screen. Click [Next] on this screen and then the {Continue} button on
the dialog box to star the installation. During the installation process, you will be asked to switch disks 10
Disk #2, Disk #3, and then Disk 44
[Nore that with CentOS 45, the installer would ask to switch to Disk #2, Disk #3, Disk #4, Disk #1, and then
back to Disk #4
Graphical Interface (X) Configuration
With most RHEL 4 distributions (not the case with CentOS 4.5), when the installation is complete, the
Installer will attempr to detect your video hardware. Ensure that the installer has detected and selected the
correct video hardivare (graphics card and monitor) to properly use the X Windows server. You will continue
‘with the X configuration inthe next serveral sereens.
Congratulations
And that's it, You have successfully installed CentOS on the new Oracle RAC node (inu3). The installer will
‘jet the CD from the CD-ROM drive. Take out the CD and click [Reboot] to reboot the system.
When the system boots into Linux for the first time, it will prompt you with another Welcome sereen. The
following wizard allows you to configure the date and time, add any additional users, test the sound card, and
to install any additional CDs. The only seveen I eare about isthe time and date (and ifyou are using CentOS
44x, the monitoriésplay settings). As fr the others, simply run through them as there is nothing additional
that needs to be installed (at this point anyways}. everything was successful you should now be presented
with the iogin screen,
Network Configuration
Perform the following network configuration tasks on the new Oracle RAC |p nese
Introduction to Network Settings
Altuough we configured several of the network settings during the installation of CentOS, it is important to not skip this
section as it contains eritcal steps that are required fora successful RAC environmen,
During the Linux OVS install we already configured the IP adress and host name for the new Oracle RAC node. We now
eed to configure the /ete/nosts file as well as adjusting several of the network settings forthe interconnect.
All nodes in the RAC cluster should have one stati IP address forthe public network and one static IP address forthe
private cluster interconnect, The private interconnect should only be used by Oracle to tansfer Cluster Manager and
Cache Fusion related data along with data forthe network storage server (Opentiler) Note that Oracle does not support
using the public network interface forthe interconnect. You must have one network interface for the public network and
‘another network interface for the private interconnect. For a production RAC implementation, the interconnect should be
at lest gigabit (or more) and only be used by Oracle as well a having the network storage server (Opentiler) on a separate
sigabit network
Configuring Public and Private Network
With the new Oracle RAC node, we need to configure the network for access to the public network as well a the private
intereonneet.
‘The easiest way to configure network settings in Red Hat Linux is with the program Network Configuration. This
application can be started from the command-line as the “root” user account a follows:
# Jusr/bin/aysten-configenetvork &
‘Do not use DIICP naming forthe public IP adress or the interconneeis- we need
static IP addresses!
Using the Network Configuration application, we will need to configure both NIC devices as well asthe /etc/nosts file on
all nodes inthe RAC cluster. Both ofthese tasks ean be completed using the Network Configuration GUI.
Please note that forthe purpose of ths example configuration the etc/hosts entries will be the same forall three Oracle
RAC nodes (25nuxt, 1inux2, and Linux) as Well asthe network storage server (opener?)
‘Our example configuration will use the following settings for all nodes
‘Oracle RAC Node 3 (nun:
[iP Address _[Subnet___[Gateway_[Pu
fn [1921681107 255-255255 0] 92,168.11 [Connovs uns ote public neo
[Conners wx Garewonnest 0
bic oux2 Gaunt pina? gH)
lem 192.1682107]255.2552550]
[econ
hir.e.e Tocalhest:Tocaldenain Tocalvost
le public Network - (etho)
hrsa.tee.2-t00° isnot
hisaiaeesacae1 Inox
fsa ‘t6e!2-367 — Tinoed
lp private interconnect ~ (etht)
fsa268.2.200° Jimoxa-priv
lysasteel2.3e1 inuna-priy
j2-268.2.267 imuns-oriv
ls pubic virtual IP (VIP) addresses = (eth@:1)
lisa-t68.2.260 lindrl-vip
hisaiasea.2e1 — linona-vip
fis2:i68.1.207 Linwes-vip
ls Private storage Network for Openfiler
fi92:168.1.295 opentilert
hs2:268.21195 epentilert-priv
Oracle RAC
[Devic] IP Address —Suuet [Gateway _[Purpore
fo _[192.168 101 s5-255255 0] 192168. [Connects Tuna oe public nena
[Connctsian2 Gavewonncet to
ruins (iu -pivinse-pe0)
lem [192.1682 101)255255 2550] ls pubise Netmork = (ethe)
lnso.t68.2.300 "lama
fsa‘aee'a-ae1 Inox
ls Private interconnect - (ethi)
hao tee.2.200° iinuxt “priv
jis2:268:2.261 inunzoriv
js2-t68.2.267 inux3-oriv
|» pubiie virtual 1 (VIP) addresses - (etha:1)
lnsa.268.2.260 lindxl-vip
fis2ri68.2-201 inoe2-vip
lysalieela267 — lnona-vip
ls Private storage Network for Opent iter
fisa268.1.295 opentilerd
fn92:168.2.195 __opentileri-oriv
‘Oracle RAC
[Devic] iP Addren—[Suboet [Gateway —[Parpore
feo [roa teu 1o0)assa55 355 0] 192 168.11 [Connects nant oe pbc nena
rie Tina
la [Connces inant Gatonncan 0
tay [192 1682.100)2552552559) fox linux’ dinun2 prvi.)
[eteons
f7.0.0.3 Toesihest,Tacaienain TocoTvost
ly public Network - (ethe)
hsa.tee.2-200 isnoet
kisaiteeate1 lino?
hisaiaee.aae7 linux
ls private znterconnect = (eth)
jnsa.268.2.200° limuxi-priv
fis2-t68.2.201 Tinux2-riv
lp Public virtual 1 (VIP) addresses - (etho:1)
fis2.168.1.208 inoxi-vip
lysariee.a2e1 — linuna-vip
hsaiaee.2.267 noea-vip
|» private storage Network for Opentiter
lns2.268.1.295 openeilera
fis2:168.2.195 _opentileri-priv
In the screenshots below, only the new Oracle RAC node (Liruna) is shown. Ensue thatthe /ete/nosts file is updated on
all participating nodes to access the new Oracle RAC node!
Pe
BH ® # *
New Edt Copy Delete Activate Deactivate
evices| var [se [ONS Hse
iG) Younay congue newah devices seca ih
a5
piysical hardware here. Mutipl logical devices can be
associated with a singe piece of hardware.
Profie[Status [Device [Nickname [Type
F Active Bh etd eth Ethernet
SST TT Em
[Active profile: Common
we 2: Network Configuration Screen - Node 3 (lia3) ‘General | Rout | Hardware Device
‘Nickname: |
Activate device when computer stats
(Allow all users to enable and disable the device
(7 Enable 1Pvcofigration fo this interface
© Automaticaly obtain IP address setings wih:
DHCP Serings
Hostname option:
chen [=
Automatically obtain DNS infomation fom provider
© Siatcaly set IP adresses:
Manual IP Address Settings
Adress:
192.168.1107
‘Subnet mask: 255.255.255.0
Default gateway address: 192.681.
Px | |W Cane!
Figure 3: Ethernet Device Screen - eth (limus3)
‘General | Route [Hardware Device
‘Nickname: [EH
© Activate device when computer stars
Allow al users to enable and disable the device
C Enable 1Pv configuration fortis interface
© Automaticaly obtain IP address setings wih:
DHCP Serings
hep [=
Hostname option:
Automatically obtain DNS infomation fom provider
© Statcaly set IP adresses:
Manual IP Address Settings
Adress
192.168.2107
‘Subnet mask: 255.255.2550
Default gateway adress:
Px | |W Cane!
Figure 4: Ethernet Device Screen ~ eth (linws3) Pees
le Profle Help
&
New
ara)
Eat Covy Delete
Devices |
ae e045] Hs |
‘You may specity static computer hostname to IP adress
mappings here. If ONS is in use, these setings wil ake
precedence over any information it may provide.
i
= [Aiaces
192 1681100 — owed
192.168.1.101 we2
192.168.1107 tinued
192.1682.100 tinuxl-piv
192.1682.101 —tinux2-piv
192.168.2107 tinax-priv
192.168.1200 tnwed-vip
192.168.1.201 —tinwx2-ip
192.168.1207 tinwx3 ip
192.168.1195 opener.
192.1682.195 opener priv
[Active pofle: Common
Figure 5: Network Configuration Screen -/etchhosts(linws3)
‘Once the network is configured, you can use the ifeonfig command to verify everything is working. The following
‘example is from the new Oracle RAC node Linux
/sbin/itcontig -2
ene
Link encapstthernet baer 08:18:24: 37:68:9
nee sders192168.4.107 -Seast192,168.1.255. Mask:255.255.255.0
nets adde: #080: 2ie:2a¢4: 439 ‘nde/64Scope:Link
Up BROADCAST RUNNING MULTICAST HTUs2500 Beenie"
RX packets:1167677 errors:® dropped:8 overruns:® frane:0
1% packets: 1842517 errors:0 deopped:® overruns:B carrler:®
collisions:@ txquevelen: 1000
x oytes:576629151 (548.9 HIB) 1X bytes:2143836828 (1.8 GiB)
‘ncerrupt:209 Sase adaress:0xef00
cet Link eneapstthernet _falsedr 00:06 :@C:€0: 79:68
nets addr: 260: 200: cf: fec0:7864/50 Scope: Link
ip BRoaDcasr RUNNING MULTICAST. RIV'2500° metric"
Rx packets:48 errors:@ dropped:@ overruns:@ frane:®
1K packets:59 errors:@ cropped:8 overruns:® carrier:
collisions:@ txquevelen: 1000
x bytes:4782 (4.6 KiB) TK bytes:556e (5.4 KIB)
ss adress @xdd60 Metoryfe9-8000-Fe0e0300
te Link encapstoeal Loopback
net sddrs127.0,0.1 Mask:255.0.0.€
nets addr: 112/128 seope:host
i LoopaaCe RUNNING “MIUSI6836. MeErse:1
RX packets:2034 erors:@ dropped:@ overruns:@ rat
1% packets:2034 errorsi@ dropped:@ overvunsi@ carrier:@
collisions:@ txquevelen:e
x ytess23R6565 (2-2,MB) TK bytes:2386686 (2.2 Mia)
site Link encap:1Pv6-in-1hva
onap"MTU:1489. Metrie:t
x packets:@ errors:e dropped:© everruns:@ frane:@
1K packets:@ errors:€ dropped: overruns:® carrser:®
collisions:@ txqueuelen:8
RX nytes:0 (2.0.0) TK bytes:0 (8.2 b)
‘Verify Network Access to All Nodes
Verify that the new Oracle RAC node has acces to the public and private network for all curent nodes. From Yénux3
# ping ©
2 packets
# ping =
4 packets
# ping -¢
2 packets
# ping
4 pockets
4 Linuna | grep ° packets transmitted”
transmitted, 1 received, 6X packet loss, time On
4 Linuxa-priv | grep ‘2 packets transnitt
‘transmitted, 1 received, 8X packet loss, time Ons
1 Linux2 | grep ‘2 packets transmitted”
‘transmitted, 1 received, 6X packet oss, tine Ons
1 Linuxa-priv | grep "1 packets transnitt
transmitted, 1 received, 8X packet 105s, # ping -c 1 opentiert | grep ‘1 packets transnitted
4 packets tranenitted, 1 Feetived, eX packet oss, tine On
4 ping +¢ 2 opentiiers-priv | grep "2 packets transmitted’
4 packets tranenitted, 1 received, @X packet Lossy, tine Ox
mnfirm the RAC Node Nam Lis A
Ensure that the new Oracle RAC node (23nuxa) is not included for the loopback addess in the /ete/nosts file, IFthe
‘machine name is listed in the inthe loopback address entry as below:
227.0.0.1 inuxd localhost. localdomoin localhost
‘will need to be removed as shown below:
Ifthe RAC node name is listed forthe loopback address, you will receive the
following error during the RAC installation:
(tA-00602: ORACLE server session terminated by fatal error
0m-29702: error accurred in Cluster Grovp Service aperation
Confirm localhost is d
od in the /ete/hosts
le for the lopback address
Ensure that the entry for localhost. ecaldonain and localhost are included for the loopback address in the /ete/hosts
file forthe new Oracle RAC node:
27.0.8. ocalhost.docaldonain localhost
{fan entry does not exist for localhost inthe /ete/hosts file, Oracle Clusterware
will be unable to start the application resources — notably the ONS process. The
‘error would indicate "Failed to get IP for localhost" and will be writen to the log
file for ONS, For example:
(5.0235 could not start resource “ora. Linux3-ons". cheek log file
"1u@t/app/crs/Log/inuk3/nacg/ora-2inbe3.0ns- 10g"
for nore details:
‘The ONS log file will contin ines similar to the following:
oracle Database 10 CRS Release 10.
Oracle. All rights reserved.
2007-06-14 13:18:62. 729: | RACG)[3086871296] [23316] {3086871296}
{ora 1ina.ons]: Failed to get IP for localhost (1)
Failed to gov iP tor localhost (2)
Failed to get IP for localhost (1)
fonsctl: ong falled to start
1.8 Production Copyright 1996, 2005
Adjusting Network Settings
With Oracle 9.2.0.1 and onwards, Oracle now makes use of UDP asthe default protocol on Linux for inter-process
communication (IPC), such as Cache Fusion snd Cluster Manager bulter tansfers between instances within the RAC
cluster
Oracle strongly suggests to adjust the default and maximum send buffer size (so_sNosur socket option) to 256 KB, and the
_evnocr og file
(98/29/2005 22:17:39
(8/35/2005 22:17:19 .
fAnstriz!: Client anit unsuccessful + (32)
proprinit:prablen reading the boot
Lock or superbloc 22
‘When experiencing this type of eror, the solution is to remove the udp ICMP (iptables) rejection rule - orto simply have
the firewall option tumed off, The Oracle Clusterware software will then start o operate normally and not crash. The
following commands should be executed asthe root user account
‘Check to ensure that the firewall option is turned off. If he firewall option is stopped (like it sin my example
‘below) you do not have to proceed withthe following steps.
f fetc/re.a/init.d/iptables status
Firewall is stopped
2. Ifthe firewall option is operating you will need to frst manually disable UDP ICMP rejections:
4 Jete/re.d/init.d/iptables stop
Flushing firewall rules: [0X]
Setting chains to policy ACCEPT: #llter [ O«
Untoading sptables modules: (Ok)
5. Then, to tum UDP ICMP rejections off for next server reboot (which should always b
4 chkcontig iptables off
Configure Network Security on the Openfiler Storage Server Perform the following configuration tasks on the network storage server,
(openfilert)!
With the network now setup, the next step isto configure network access in Openfiler so that the new Oracle RAC node (Lénux3) has
‘permissions tothe shared iSCSI volumes used inthe current Oracle RAC 10g environment. For the purpose of this example, all iSCSI
trafic will use the private metwork interface etna which inthis article ison the 192,268.20 network.
Openfiler administration is performed using the Openfiler Storage Contral Center —a browser based tool over an https connection on
port 446, For example:
hetps:/opentilert:a4s/
rom the Openfiler Storage Control Center home page, login as an administrator. The default administration login eredentials for
Opentiler ae:
‘The first page the administrator sees isthe [Accounts] / [Authentication] screen, Configuring user accounts and groups is not necessary
{for this article and will therefore not be discussed.
Services
‘This article assumes that the current Oracle RAC 10g environment is operational and therefore the SCSI services should
already be enabled within Opener.
‘To verify the iSCSI serves are running, use the Openfiler Storage Control Center and navigate to [Services]
([Bnable/Disable}
Enable/Disable services
Service Name | status | Modification
waves Diabied
ness Enabled
HITP/WebORV Disables
re Diesine
Rene Disbled | Enable
uss istbled | enable
Figure 6: Verify SCSI Services are Enabled
Another method isto SSH into the Openfier server and verify the isei-trget service is running:
[rootgopentitert =]# service iscsi-target status
ete (pid 3788) 1s running
[Network Access Restriction
‘The next step is fo configure network access in Openfiler so thatthe new Oracle RAC node (13ni33) has permissions to
the shared iSCSI volumes used inthe current Oracle RAC 10g environment.
Again, this task can be completed sing the Openfiler Storage Control Center by navigating to [General] / [Local
[Networks], The Local Networks sereen allows an administrator to setup networks andor hosts that will be allowed to
access resources exported by the Openfler appliance. For the purpose ofthis article, we will want to add the new Oracle
RAC node individually rather than allowing the entre 192.168. 2.0 network have access to Opentiler resources. ‘When entering the new Oracle RAC node, note that the ‘Name’ field is just logical name used for reference only. As @
convention when entering nodes, I simply use the node name defined for that IP address, Next, when entering the actual
noe in the ‘Network/Hos field, always use i's IP address even though its host name may already be defined in your
‘etcshosts file or DNS, Lasily, when entering actual hosts in our Class C network, use a subnet mask of 255.255.255.255,
It is important to remember that you wil be entering the IP address of the privae network (ethl) for the new Oracle RAC
node.
‘The following image shows the results of adding the new Oracle RAC node Linux to the local network configuration:
Local networks configuration
Delete Nome Network/Mlost_Netmask ype
Figure 7: Configure Openiler Host Access for new Oracle RAC Node
Current Logical iSCSI Volumes
‘The current Openfiler configuration contain five logical iSCSI volumes ina single volume group named recs.
SCSI Logical Volumes a Volume Group ed
Volume Name] Volume Deseription [Required Space (MB) [Fleystem Type
oo Oracle Chsterware 2s) _ iSCSI
asm Jorace ASM Volume Tia720|iscst
snd [Orane ASM Volume? Tia720| 80st
asm Oracle ASM Volumes T3720] 851
asm Oracle ASM Volume 114720] __18¢S
‘To view the available iSCSI volumes ftom within the Openfier Storage Control Center, navigate to [Volumes] / [List of
Existing Volumes]. There we will se all five logical volumes within the volume group rac]: Volumes in volume group “raci" (476928 MB)
Yolume | volume | volume fH | Fe | reused | Fees
teuend ame descripon “size system system U0" Nace
fort Noho e | he spline copheste
wnt Volume” SSS appeale alate applcele
Delete cat
kis | ie
ft
ceeate
sds to be granted the appropriate permissions. In
this section, we need fo grant access fo each of the five local ISCSI Volumes to the new Oracle RAC node Linus,
From the Opentiler Storage Control Center, navigate to [Volumes] / [List of Existing Volumes). This will present the
screen shown in the previous section. For each ofthe five logical volumes, click on the Ei’ Tink (under the Properties
column). This will bring up the Edit properties’ screen for that volume, Scroll to the bottom of this sereen; change the
access for host 1inux3-priv from ‘Deny’ to Allow’ and click the "Update button, Perform this task forall five logleal
ee ee
Figure 9: Grant Host Access to Logical (iSCSI) Volumes
Configure the iSCSI Initiator
‘An iSCSI client ean be any system (Linux, Unix, MS Windows, Apple Mac, etc.) for which iSCS
‘ur ease, the clients are the three Oracle RAC nodes, (Iino),
suppor (a drive) is available. In
rux2, and Lirux3), running Red Hat 4 In this section we will he configuring the iSCSI initiator on the new Oracle RAC node 1inux3. Tis involves configuring the
Jete/ssest.con¢ file on the new Oracle RAC node with the name of the network storage Server (openers) soit ean discover the
current iSCSI volumes,
(On the new Oracle RAC node, we have to make sure the iSCSI (initiator) service is up and running. Ino installed as part
of the operating system setup te iesi-intitor-tils RPM (Le. ises-initator-utls-4.0.3.0-5.:586.spm) should be
downloaded and installed on the new Oracle RAC node,
“The new Oracle RAC node must have the isesi-intiator-utils RPM installed, To
determine if this package is installed, perform the following:
1 ope -aa | grep iscsi
Seest-inttiotor-uthls-#.0,3,0-5
1 not installed, the isesi-intitor-uils RPM package can be found on disk 3 of 4 ofthe
RHEL4 Update 5 distribution or downloaded from one ofthe Internet RPM resources.
Use the following command to install the isesi-intitor-utils RPM package if not
present
4 rpm -wh Ascst-inttiator-utfls-£.0.3.0-5.4386.rom
Mlarmings Iaericinieiator-utlls-4.0.3.@-5,1586.r9m
"VS OSA elgnature! NDKEY, key 10 2e3ei832
Preparing. = ‘pernvessernanneoeenrenannsvererenarssternre (200%)
[Hsceicinitiator-utils _sesessssuueanansseuutanasseesanensasseeene (100%)
‘After verifying thatthe ises-iniiator-utls RPM is installed, the only configuration step required on the new Oracle RAC.
‘node (iSCSI client) is to specify the network storage server (iSCSI server) in the /ete/tses4.con¢ file. ait the
Jetc/ isesi.conF file and include an entry for Discoveryadéress which specifies the hostname of the Opentiler network
Storage server. In our case that was
DiscoveryAddress-openfilert-priv
‘After making the change tothe fete/isest.con¢ file on the new Oracle RAC node, we ean start (or estat the isesi
initiator service on that node:
4 service dsest restart
Searching for isesi-baseé multipath naps
Found 8 naps
Stopping isesid: sscsid not running
Ccacking Sscst configs (ok J
Tossing Ssest-orivert (ok)
Starting tsestas [OC]
‘We should also configure the iSCSI service t be active across machine reboots forthe new Oracle RAC node, The Linux
command chkconfig can be used to achieve that as follows:
1 ehkeonfig ~-Level 345 tsesi on
Discovering iSCSI Targets
‘The iSCSI initiator service should now be configured and started on the new Oracle RAC node. In te parent to this atile
(*Building an Inexpensive Oracle RAC 10g Release 2 on Linux ~ (CentOS 4.5 iSCSI)", we needed go through the
arduous task of mapping the SCSI target names discovered frora Openfiler tothe local SCSI device name on one ofthe
(Oracle RAC nodes. Given that al ive logical iSCSI volumes were partitioned and formatted with labels in that article, we
{dont have to perform that task again. Note tha one ofthe iSCSI volumes was formatted and labeled using OCFS2 while
the other four were labeled for use by ASM.
In ths section, we simply want to verify thatthe new Oracle RAC node was able to successfully discover the five logical
iSCSI volumes on the Openfiler server.
‘When the Openfiler server publishes available iSCSI trgets, configured clients get the message that new iSCSI disk are
‘pow available. This happens when the iesi-arget service gets startedestarted on the Opentiler server or when the iSCSI
initiator service is staredirestarted on the client. We would see something lke this inthe clients /var/log/nessages fle
35 25 16:41:29 Linwed iscsi: isesid startup succeeded
dan 21 16:41:29 linges isesia[13922]- Comectes to Discovery Address 192.168.2.195,
Jan 23 46:41:29 Linond Kernel! iscsi-sfnetinost®: Session established
San 25 16:41:29 Linuna kernel: Isent-stnet:nost2: Session established
Jan 23 16:41:29 Ldnona kernel: tacet-stnetinosti: Sezaion established
San 22 36:41:29 Linvxd kennel: scsi? Sehew SSSI driver
Jan 25 16:41:29 Iinued Kernel: seat? + Shee SCSI dniver
Jan 23 1esalig9 Lnued kernel: sesit : Met S081 driver
Jan 21 16:41:29 linoes Kernel: Vendor: Openfale Model: Virtual disk fev: @
Jan 21 16:41:29 Linoe3 Kernel: Type: Direct-Aecess AIST SCSI revision: 08
Jan 25 16:41:29 linued kernel: SCSI device saa: 249138560 512-byte hdwe sectors (124487 HB)
Jan 23 16:41:29 Linond kernel: SESE device sda: drive cache: write throvgn Jan 23 16:41:29 Inwx3 Kernel: Vendor: Openfsle Hodel: Virtual disk Rev: @
Jan 21 16:41:29 Linuea Kernel: Type: Direct-Aecess AMEE SST revision
Jan 23 16:41:29 Linuna kernel: SCSY device sda: 243138560 S12-byte hdwe sectors (124487 08)
Jan 23 16:41:29 Linund kernel: Isest-sfnetsnost3: Session established,
3an 23 desais29 Linund kernel: dacet-stnetcnosta: Sexaion established
Jan 23 de:4li29 Linona kernel: scsi? : srket i8¢S0 dedver
Jan 22 36:41:28 Lind kernel: SCSI device sear drive cache: write through
an 25 46:41:29 Linues Kernel: saa unknown partition eable
Jan 21 16:41:29 linges Kernel: Attached scsi disk sda at scsi@, chamel @, id 0, lun 0
Jan 24 16:41:29 Tina kernel: Vendor: Openfile Rodel: Virtual disk "Rev! @
Jan 21 16:81:29 Linoed sestvagent(13934]: disk at /devices/plat fora/hoste/tangete:e:0/0:0:0:@
Jan 21 16:41:29 linuxs Kernel! Type: Dlrect-Access MED SST revision:
Jan 21 16:41:29 linues Kernel: Vendor: Openfiie Model: Virtual disk fev: @
Jan 21 16:81:29 Linowa Kernel: Type: irect-Access AISI SCSI revision
Jan 24 Lesals99 Linond kernel: scala: seat i8CS4 driver
dan 21 16:41:29 lanues Kernel: SCSI device seb: 243138360 $12-byze hdkr sectors (120487 M8)
Jan 25 46:41:29 Tinoed Kernel: Vendor Openfste Hogel? Vireual disk Rev: @
Jan 22 16:41:29 Linuea kernel: — Type: Direct-Aecess MSE SeST revision
Jan 23 6:41:29 Linona kernel: SCST cevice sob: drive cache: wrive throvgn
Jan 21 16:41:29 Linoe3 sest agent [13983]; disk at /devices/platfora/host2/tanget2:0:0/2:0:0:0
Jan 23 16:41:29 Linued scsi. agent (19996): disk at /devices/plattora/hast3/carget3:0:0/3:0:0:2
Jan 21 16:ali4@ linund kernels Sest device sob: 242190560 ¢12-byte haw sectors (128487 8)
Jan 23 16:41:30 Tinona kernel: SCSI device sab: drive cache: wrive throvn
Jan 23 46:41:30 Linved kernel: —sabs unknown partition eable
Jan 24 1e:al:30 Linuxd kernel: attached seri Gisk sdb at scei2, chanel @, 44 0, lun @
Jan 21 16:41:38 linoxs Kernel: SCSI device sec: 243138560 S12-tyze hdsr séctors| (120487 6)
Jan 23 16:41:39 Linoed Kernel: SCSI device ade: drive cache: wrive thravg
Jan 21 16:41:30 Linund kernel: S€St device se: 2421985e0 S12-byte hdwe sectors (124487 M8)
Jan 23 16:41:39 Lnona kernel: SCSI device ade: drive cache: wrive throug
Jan 23 46:41:29 Tnoed Kernel: ader unknown partition *able
Jan 21 desal:30 linuea kernel: attached scsi isk sde at sce43, chanel @, 44 0, lun 0
Jan 23 16:41:20 Linuna kernel: StSt device sd: 242198560 £12-byte haw sectors (124457 M8)
Jan 24 36:41:30 Linwxd kernel: SCSI device sod: drive cache: write through
an 25 16:41:39 Linues kernel: SCSI device sad: 243198560 512-byte hdke sectors (124487 MB)
Jan 23 1e:ali0 Linona kernel: SCSI device sad: drive cache: write through
San 23 26:41:38 Langa kennel: SoG: unknown partition table
Jan 23 16:41:39 Linued Kernel: Attached ses isk dd at scs41, chanel @, 44 0, lun 0
Jan 23 16:41:30 Linuna kernel: SCSt device ade: 419¢304 S22-byte hour sectors (2147 MB)
San 2 i6:41:38 Tinos sest.agent{14052]: disk at /devices/plattora/hosta/cargeta:e:0/4:0:0:8
an 23 16:41:99 linued sestcagent|14085]= disk at /devices/platforn/hasti/eargetl:0:0/2:0:0:8
Jan 24 1e:ali30 Linund kernel! SCst device ade: drive cache: write through
Jan 25 16:41:29 Lamond kernel: SCSI device age: 4194364 S22-byte hur sectors (2147 #8)
Jan 28 36241130 Linxd Kernel: SCSI device sae: drive cache: write through
an 22 6:41:39 Linues Kernel: sae unknown partition eable
Jan 21 16:41:30 linues Kernel: attached scsi Gisk sde at scsi4, chamel @, id 0, lun 0
‘The above entries show that the client (Linux3) was able to establish the SCSI sessions with the iSCSI storage server
(open¢ilera-priv at 192.168.2195)
Another method not only checks forthe existence ofthe iSCSI volumes, but also displays how the local SCSI device
‘names map to iSCSI targels' host IDs and LUNs. Use the following script which was provided by Martin Jones to display
these mappings:
|p FILE: tsest-isowap.sh
un usearD-root
Jespare RUN USERID
ruzoe"i | ank -F\¢ {print $2)" law -F\) “{print $3)"
PE Sindtoy 12 “SRUN USERID" ]]sthen
f2eho "You must be Logged in as $AWN_USERID to run this script."
fecho "Exiting scripts”
ede a
es
Jaese | grep “attach” \
| Suk f°" *C print "Vdeus"$4 "$6" \
| Sea Se 4/17" | sed se "s/sesti/™ \
| Sore tn ia
| sea -e' 7etskaya" > /enp/tap_sest_dev
lsest-is | egrep -e "TARGET NAME" 2 "host 2D"
i
J aa oF "°F ($8 = /MTARGET.*/) print $45 1F ( $8 ~ /OH0ST/) print * XS\n", $4)"
| Sore en iz \
Jawa tet
| cue “ar. “42° > 'venprtnp_sest targets
lsoin -t * -4 2 -2 2 /tmp/tme_scsi_dev /tap/tnp_scas_targets > MAP
Jecho *host / SCSI 19 SCST Device Name {5CSt Target ane”
echo
Jat MAP | sed -e s/s re
\ ea
Example run:
8 s/isest-Asomp.sh
host / SCSI ID.” SCSI Device Kane {set Target Wane
e jaecjaaa san
1 fae sas asm
2 faevisee asnz
3 faeersae asad
a fae sae ere
(Create “oracle” User and Directories
© restorm the ftowing tasks onthe new Oracle RAC noe!
1 will be using the Oracle Cluster File System, Release 2 (OCFS2) to store the files required tobe shared forthe Oracle Clusterware
software, When using OCFS2, the UID of the UNIX user "oracle" and GID of the UNIX group "oinstall” must be the same on all of
the Oracle RAC nodes in the cluster. Ifether the UID or GID are different, the files on the OCS? file system will show up as
"unowned” or may even be owned by a different user. For this article and its parent article, I will use $O} for the "oraete” UID and $01
for the “instal” GID.
Note that members ofthe UNIX group oinstalt are considered the "owners" ofthe Oracle software, Members of the aba group can
administer Oracle databases, for example starting up and shutting down databases. In this article, we are creating the oracle user
‘account to have both responsibilities!
This guide adheres tothe Optimal Flexible Architecture (OFA) for naming
conventions used in creating the directory structure
(Create Group and User for Oracle
Lets star this section by ereatng the UNIX oinstal and éba group and oracle user account:
# groupadd -g Sei oinstall
4 groupadd “2 502 dba
4 Useradd -m-u 501 -g olnstall -G dba -d /hone/oracte -s /bin/bash -¢ “Oracle Software Omer oracle
# id oracle
Ulde50i(oracle) gideSe1(oinstall) groupseSe1(oinstall) ,502(dba)
‘Set the password for the orseie account:
4 passwd oracle
Canging password for user oracle.
Nw UNIX passnord: eae
Retype new UNIX password? xeon
passwd: all authentication tokens updated successfully.
‘Verify That the User nobody Exists
Before installing the Oracle software, complete the following procedure to verify that the user nobody exists on the
system:
1. To determine if the user exists, enter the following command:
44 44 nobody
Uidss8(nobody) gi4-88(nobody) groups-99(nobody)
[IF this command displays information about the nobody user, then you do not have to create that use
2. I'he user nobody does nol exist, then enter the following command to crete it
4 Jusr/sbin/useradd nobody
(Create the Oracle Base Directory
‘The next sep isto create «new directory that will be used to store the Oracle Database software. When configuring the
coracle users environment (later inthis section) we will be assigning the location ofthis directory tothe SORACL=_8ASE
‘environment variable,
‘The following assumes that the directories are being created in the root filesystem. Please note that this is being done for
the sake of simplicity and is not recommended asa general practice, Normally, these directories Would be ereated on a
separate fle system, ‘After dhe directory is ereated, you must then specify the correct owner, group, and permissions for it, Perform the
following on the new Oracle RAC node:
4 mkair =p /vex/app/oracte
4 Chown “R oraclesoinstall /utt/app/oracle
1 chmod “R 775 /u01/app/eracie
Atthe end ofthis procedure, you will have the following:
+ 01 owned by root
+ 8x/3pp owned by toot.
+ 1401/app/oraele owned by oracie:otnstait with 775 permissions. This ownership and permissions enables the
(OUL to create the oratnventory dtcctory, in the path /v01/app/oracie/eraTaventory
Create the Oracle Clusterware Home Directory
ext, create a now directory that will be used to store the Oracle Clusterware software. When configuring the oracle user's
environment (later in this section) we willbe assigning the location ofthis directory to the $oRA_cA5_HOME environment
variable,
‘As noted in the previous section, the following assumes thatthe directories are being created in the root filesystem, This is
boeing done forthe sake of simplicity and is not recommended as a general practice. Normally these directories would be
created on a separate filesystem.
‘After the directory is ereated, you must then specify the correct owner, group, and permissions fori, Perform the
following on the new Oracle RAC node:
kai -p /vor/app/ers
4 Chown “R oraclesoinstall /u01/app/crs
4 ehmod “R775 /u0%/ap9/ers
At the end ofthis procedure, you will have the following:
+ (102 owned by root
+ /ex/app owned by root
+ 1er/2pp/crs owned by oracie:oinstall with 775 permissions. These permissions are required for Oracle
CClusterware installation and ate changed during te installation process.
‘Create Mount Point for OCES2/ Clusterware
Lets now create the mount point forthe Oracle Cluster File System, Release 2 (OCFS2) that will be used to store the two
(Oracle Clusterwate shared fies
Perform the following on the new Oracle RAC node:
kai -p (02
4 chown “R oracle:oinstall /ut2
1 mod “R775: /u02
ript for oracle
Account
‘To ensure that the environment is setup correctly forthe Yoracle" UNIX userid on the new Oracle RAC node, use the
following .cash_provite:
‘When you are secting the Oracle environment variables for each Oracle RAC node,
ensure to assign each RAC node a unigue Oracle SID!
For this example, used:
4 Mimuxd ORACLE s1D-orelt
wna ! onaCLE_S1D-ore12
Login tothe new Oracle RAC node as the oracle user acount
# su - oracte
Pash profite for Oracle U
lv ash prottie
ls Got the aitases and functions
ibe'T + =/-basnre J}. then
obasne
es
lsas ts-"ae 7A"
Jsiias s-"screen -DRRS SPad -* sPad™ Jexport 3AVA_HOME=/usr/lecal/Java
lr user specsfic environnant and startup pragrans
export ORACLE. 8ASE~/92/app/oracle
Jesport ORACLE_NOME-SORACLE”BASC/product/10.2.0/40 3
Jexport ona cRS HonE=/u01/250/crs
Jexport ORACLE, PATH-SORACLE_BASE/dba_scripts/sql:.$ORACLE_MOME/rdbms/3énin
Jespore Cv J0KTOnE=/Usr/1ee31/ java
Js each aac node must have a unique ORACLE SID. (Lue, orel1, orcl2, orel3,..+)
Jesport ORACLE, s1Deorci3
Jexpore pare. :$(2AVA, HOME) /oin$ (PATH) :SHOME/in: SORACLE HOME/bAn
Jespore PaTa-&(PAIM}* usr/bin: /bin:/usr/bin/¥41:/uar/ local sn,
Jexpore PATH-${PATH) SORACLE GASC/aba_scripts/bin
Jexpore ORACLE_TeaMextern
Jesport INS_ADRIN-SORACLE_ HOME /networss/adnin
fesport ORAWLSte-SORRCLE-HOME/nis/ dae
Jesport NUS-DATE.FORMAT="DD-MON-YWVY f24: 4:55"
a ORACLE HOME/I|
LSCLEBRaRYParicg (UD. LIBRARY. PATA}: SORACLE_NOME/oracn/23b
Jexport Lo-L2BRAtY_pario8{Lb_LIaRARY_PxT}:/24b: /var/ib:/usr/ioeal/11D
Jespore CLASSPATASORACLE.HORE/IRE
Jesport CLASSPATHOS{CLASSPATH):SORACLE_MOME/3119
Jesport CLASSPATIn${CLASSPATH}SORACLE_HIME/rdons/j14b
Jexpore CLASSPATH=${CLASSPAT) : SORACLE_HOME network /3130|
lesport THREADS FLAG-native
Tees /tie
Configure the Linux Server for Oracle
[Eby Form ac tnenng tats catar ocr Once RAC
‘The kernel parameters and shell limits discussed inthis section will nce to be defined on the new Oracle RAC node every time the
machine is booted. This section will not go into great depth in explaining the purpose of those kernel parameters that are required by
Oracle (These parameters are described in detail in the parent to this article). Provided in this section, however ae instructions on
how to set all required kernel parameters for Oracle and how to have them enabled when the node boots, Further instructions for
configuring kemel parameters ina starup serpt(/etc/sysct1 conf) is included in the section “Al Startup Commands for New Oracle
RAC Node"
‘Swap Space Considerations
+ Installing Oracle1 0g Release 2 requires ¢ minimum of $12MB of memory.
(Gn inadequate amount of swap during the installation will cause the Oracle Universal Installer to either "hang" or "die")
+ To check the amount of memory you have, type
4 cat /procimeninfo | grep Nentotal
Pemtotale 20728 ke
+ To check the amount of swap you have allocated, type:
4 cat /proc/meninfo | grep swapTotal
SwapTotel: 203160813
+ Ifyou have less than $12MB of memory (between your RAM and SWAP), you can add temporary swap space by ereating 2
temporary swap file, This way you do not have to use a raw device or even more drastic, rebuild your system,
‘As root, make fle that will act as additional swap space, lets say about SOOMB:
1 ad Afs/dev/zero of=tenpswap bs=Ik count=500000
[Now we should change the file permissions:
* chmod 660 tenpexap
Finally we format the "paritio
a mke2ts tenpswap
4 mkswap tenpsiap
1 Saapon tempsap
as swap and add i tothe swap space:
‘Configuring Kernel Parameters and Shell Limits
‘The kemel parameters and shell limits presented inthis section are recommended values only as documented by Oracle,
For production database systems, Oracle recommends that you tune these values to optimize the performance of the
system.
(On the new Oracle RAC node, verify thatthe kemel parameters described inthis section are set to values greater than or
‘equal to the recommended values, Also note that when setting the our semaphore values that al four values need tobe centered on one line
Selting Shared Memory / Semaphores/ File Handles / Local IP Ran
Set the following kernel parameters inthe /ete/sysct1.cont file onthe new Oracle RAC node,
1 lost sors anda Fn ci}
2) SoGE eect ata |
|
11 | Setting the maximum amount oF File handles, and’ seveing
4 | the IP local pore range.
a]
al
a1
4 | SEms_value SEMNS value SEHOPH value SENM_value
al
era
net-ipvatp_local_port_range=1024 65000
To improve the performance of the software on Linux systems, Oracle recommends you increase the following shell limits
for the oracle user:
Maximum numberof open Ble dseapiom note esi
axiom number of poses avalablevo a single wie [apo 16
TTo make these changes, run the following as rot:
cat >> /ete/securdty/Lnits.cont <> /ete/pan.d/login <
Вам также может понравиться
- The 7 Habits of Highly Effective People: The Infographics EditionОт EverandThe 7 Habits of Highly Effective People: The Infographics EditionРейтинг: 4 из 5 звезд4/5 (2484)
- The Perfect Marriage: A Completely Gripping Psychological SuspenseОт EverandThe Perfect Marriage: A Completely Gripping Psychological SuspenseРейтинг: 4 из 5 звезд4/5 (1116)
- Art of War: The Definitive Interpretation of Sun Tzu's Classic Book of StrategyОт EverandArt of War: The Definitive Interpretation of Sun Tzu's Classic Book of StrategyРейтинг: 4 из 5 звезд4/5 (3321)
- The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeОт EverandThe Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeРейтинг: 4 из 5 звезд4/5 (5807)
- Never Split the Difference: Negotiating As If Your Life Depended On ItОт EverandNever Split the Difference: Negotiating As If Your Life Depended On ItРейтинг: 4.5 из 5 звезд4.5/5 (3285)
- Pride and Prejudice: Bestsellers and famous BooksОт EverandPride and Prejudice: Bestsellers and famous BooksРейтинг: 4.5 из 5 звезд4.5/5 (20479)
- Habit 3 Put First Things First: The Habit of Integrity and ExecutionОт EverandHabit 3 Put First Things First: The Habit of Integrity and ExecutionРейтинг: 4 из 5 звезд4/5 (2507)
- Habit 6 Synergize: The Habit of Creative CooperationОт EverandHabit 6 Synergize: The Habit of Creative CooperationРейтинг: 4 из 5 звезд4/5 (2499)
- The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeОт EverandThe Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeРейтинг: 4.5 из 5 звезд4.5/5 (20048)
![American Gods [TV Tie-In]: A Novel](https://imgv2-2-f.scribdassets.com/img/audiobook_square_badge/626321117/198x198/22ab6b48b6/1712683119?v=1){kind=icon}
