OracleDBAInterviewQuestionsandAnswersRAC

OracleRACInterviewQuestionsandAnswers

HowdoesOCSSDstartsfirstifvotingdisk&OCRresidesinASMDiskgroups?
YoumightwonderhowCSSD,whichisrequiredtostarttheclusteredASMinstance,canbestartedif
votingdisksarestoredinASM?
Thissoundslikeachickenandeggproblem:
withoutaccesstothevotingdisksthereisnoCSS,hencethenodecannotjointhecluster.
Butwithoutbeingpartofthecluster,CSSDcannotstarttheASMinstance.
TosolvethisproblemtheASMdiskheadershavenewmetadatain11.2:
youcanusekfedtoreadtheheaderofanASMdiskcontainingavotingdisk.
Thekfdhdb.vfstartandkfdhdb.vfendfieldstellCSSwheretofindthevotingfile.ThisdoesnotrequiretheASM
instancetobeup.
Oncethevotingdisksarelocated,CSScanaccessthemandjoinsthecluster.

WhatisgsdctlinRAC?listgsdctlcommandsinOracleRAC?
GSDCTLstandsforGlobalServiceDaemonControl,wecanusegsdctlcommandstostart,stop,andobtainthe
statusoftheGSDserviceonanyplatform.

Theoptionsforgsdctlare:
$gsdctlstartTostarttheGSDservice
$gsdctlstopTostoptheGSDservice
$gsdctlstatToobtainthestatusoftheGSDservice

Logfilelocationforgsdctl:
$ORACLE_HOME/srvm/log/gsdaemon_node_name.log

WhatisRAC?
RACstandsforRealApplicationcluster.
ItisaclusteringsolutionfromOracleCorporationthatensureshighavailabilityofdatabasesbyproviding
instancefailover,mediafailoverfeatures.
OracleRACisaclusterdatabasewithasharedcachearchitecturethatovercomesthelimitationsoftraditional
sharednothingandshareddiskapproachestoprovideahighlyscalableandavailabledatabasesolutionforall
thebusinessapplications.
OracleRACprovidesthefoundationforenterprisegridcomputing.

WhatisOracleRACOneNode?
OracleRAConeNodeisasingleinstancerunningononenodeoftheclusterwhilethe2ndnodeisincold
standbymode.IftheinstancefailsforsomereasonthenRAConenodedetectitandrestarttheinstanceonthe
samenodeortheinstanceisrelocatetothe2ndnodeincasethereisfailureorfaultin1stnode.Thebenefitof
thisfeatureisthatitprovidesacoldfailoversolutionanditautomatestheinstancerelocationwithoutany
downtimeanddoesnotneedamanualintervention.Oracleintroducedthisfeaturewiththereleaseof11gR2
(availablewithEnterpriseEdition).

WhatisRACandhowisitdifferentfromnonRACdatabases?
OracleRealApplicationclustersallowsmultipleinstancestoaccessasingledatabase,theinstanceswillbe
runningonmultiplenodes.
InRealApplicationClustersenvironments,allnodesconcurrentlyexecutetransactionsagainstthesame
database.
RealApplicationClusterscoordinateseachnode'saccesstotheshareddatatoprovideconsistencyand
integrity.

WhataretheadvantagesofRAC(RealApplicationClusters)?
Reliabilityifonenodefails,thedatabasewon'tfail
Availabilitynodescanbeaddedorreplacedwithouthavingtoshutdownthedatabase
Scalabilitymorenodescanbeaddedtotheclusterastheworkloadincreases

WhatisOracleRACOneNode?
OracleRAConeNodeisasingleinstancerunningononenodeoftheclusterwhilethe2ndnodeisincold
standbymode.IftheinstancefailsforsomereasonthenRAConenodedetectitandrestarttheinstanceonthe
samenodeortheinstanceisrelocatetothe2ndnodeincasethereisfailureorfaultin1stnode.Thebenefitof
thisfeatureisthatitprovidesacoldfailoversolutionanditautomatestheinstancerelocationwithoutany
downtimeanddoesnotneedamanualintervention.Oracleintroducedthisfeaturewiththereleaseof11gR2
(availablewithEnterpriseEdition).

WhatisCacheFusion?
OracleRACiscomposedoftwoormoreinstances.Whenablockofdataisreadfromdatafilebyaninstance
withintheclusterandanotherinstanceisinneedofthesameblock,itiseasytogettheblockimagefromthe
instancewhichhastheblockinitsSGAratherthanreadingfromthedisk.Toenableinterinstance
communicationOracleRACmakesuseofinterconnects.TheGlobalEnqueueService(GES)monitorsand
Instanceenqueueprocessmanagesthecachefusion.

WhatcommandwouldyouusetochecktheavailabilityoftheRACsystem?
crs_stattv(tvareoptional)

HowdoweverifythatRACinstancesarerunning?
SQL>select*fromV$ACTIVE_INSTANCES
ThequerygivestheinstancenumberunderINST_NUMBERcolumn,host_:instancenameunderINST_NAME
column.

HowcanyouconnecttoaspecificnodeinaRACenvironment?
tnsnames.oraensurethatyouhaveINSTANCE_NAMEspecifiedinit.
Whichisthe"MASTERNODE"inRAC?
Thenodewiththelowestnodenumberwillbecomemasternodeanddynamicremasteringoftheresourceswill
takeplace.
Tofindoutthemasternodeforparticularresource,youcanqueryv$ges_resourceforMASTER_NODE
column.
Tofindoutwhichisthemasternode,youcanseeocssd.logfileandsearchfor"masternodenumber".
whenthefirstmasternodefailsintheclusterthelowestnodenumberwillbecomemasternode.

WhatcomponentsinRACmustresideinsharedstorage?
Alldatafiles,controlfiles,SPFIles,redologfilesmustresideonclusterawareshredstorage.

Givefewexamplesforsolutionsthatsupportclusterstorage?
ASM(automaticstoragemanagement),
Rawdiskdevices,
Networkfilesystem(NFS),
OCFS2and
OCFS(OracleClusterFiesystems).

WhatareOracleClusterComponents?
1.ClusterInterconnect(HAIP)
2.SharedStorage(OCR/VotingDisk)
3.Clusterwaresoftware
4.OracleKernelComponents

WhatareOracleRACComponents?
VIP,Nodeappsetc.

WhatareOracleKernelComponents?
BasicallyOraclekernelneedtoswitchedonwithRACOnoptionwhenyouconverttoRAC,thatisthedifference
asitfacilitatesfewRACbgprocesslikeLMON,LCK,LMD,LMSetc.

HowtoturnonRAC?
#linktheoraclelibraries
$cd$ORACLE_HOME/rdbms/lib
$makefins_rdbms.mkrac_on
#rebuildoracle
$cd$ORACLE_HOME/bin
$relinkoracle

DiskarchitechtureinRAC?
SAN(StorageAreaNetworks)generallyusingfibretoconnecttotheSAN
NAS(NetworkAttachedStorage)generallyusinganetworktoconnecttotheNASusingeitherNFS,ISCSI

WhatisOracleClusterware?
TheClusterwaresoftwareallowsnodestocommunicatewitheachotherandformstheclusterthatmakesthe
nodesworkasasinglelogicalserver.
ThesoftwareisrunbytheClusterReadyServices(CRS)usingtheOracleClusterRegistry(OCR)thatrecords
andmaintainstheclusterandnodemembershipinformationandthevotingdiskwhichactsasatiebreaker
duringcommunicationfailures.Consistentheartbeatinformationtravelsacrosstheinterconnecttothevoting
diskwhentheclusterisrunning.

RealApplicationClusters
OracleRACisaclusterdatabasewithasharedcachearchitecturethatovercomesthelimitationsoftraditional
sharednothingandshareddiskapproachestoprovideahighlyscalableandavailabledatabasesolutionforall
yourbusinessapplications.OracleRACprovidesthefoundationforenterprisegridcomputing.

OraclesRealApplicationClusters(RAC)optionsupportsthetransparentdeploymentofasingledatabase
acrossaclusterofservers,providingfaulttolerancefromhardwarefailuresorplannedoutages.OracleRAC
runningonclustersprovidesOracleshighestlevelofcapabilityintermsofavailability,scalability,andlowcost
computing.

OneDBopenedbymultipeinstancessothethedbllbeHighlyAvailableifaninstancecrashes.
ClusterSoftware.OraclesClusterwareorproductslikeVeritasVolumeManagerarerequiredtoprovidethe
clustersupportandalloweachnodetoknowwhichnodesbelongtotheclusterandareavailableandwith
OracleCluterwaretoknowwhichnodeshavefailedandtoejectthenfromthecluster,sothaterrorsonthat
nodecanbecleared.

OracleClusterwarehastwokeycomponentsClusterRegistryOCRandVotingDisk.

Theclusterregistryholdsallinformationaboutnodes,instances,servicesandASMstorageifused,italso
containsstateinformationietheyareavailableanduporsimilar.

Thevotingdiskisusedtodetermineifanodehasfailed,i.e.becomeseparatedfromthemajority.Ifanodeis
deemedtonolongerbelongtothemajoritythenitisforciblyrebootedandwillaftertherebootadditselfagain
thethesurvivingclusternodes.

WhataretheOracleClusterwarekeycomponents?
OracleClusterwarehastwokeycomponentsClusterRegistryOCRandVotingDisk.

WhatisVotingDiskandOCR?
VotingDisk
OracleRACusesthevotingdisktomanageclustermembershipbywayofahealthcheckandarbitratescluster
ownershipamongtheinstancesincaseofnetworkfailures.Thevotingdiskmustresideonshareddisk.
Anodemustbeabletoaccessmorethanhalfofthevotingdisksatanytime.
Forexample,ifyouhave3votingdisksconfigured,thenanodemustbeabletoaccessatleasttwoofthe
votingdisksatanytime.Ifanodecannotaccesstheminimumrequirednumberofvotingdisksitisevicted,or
removed,fromthecluster.

OracleClusterRegistry(OCR)
Theclusterregistryholdsallinformationaboutnodes,instances,servicesandASMstorageifused,italso
containsstateinformationietheyareavailableanduporsimilar.
TheOCRmustresideonshareddiskthatisaccessiblebyallofthenodesinyourcluster.

Whataretheadministrativetasksinvolvedwithvotingdisk?
Followingadministrativetasksareperformedwiththevotingdisk:
1)Backingupvotingdisks
2)RecoveringVotingdisks
3)Addingvotingdisks
4)Deletingvotingdisks
5)Movingvotingdisks

Canyouaddvotingdiskonline?Doyouneedvotingdiskbackup?
Yes,asperdocumentation,ifyouhavemultiplevotingdiskyoucanaddonline,butifyouhaveonlyonevoting
disk,bythatclusterwillbedownasitslostyoujustneedtostartcrsinexclusivemodeandaddthevotedisk
using
crsctladdvotedisk<path>

WhatistheOracleRecommendationforbackingupvotingdisk?
Oraclerecommendsustousetheddcommandtobackupthevotingdiskwithaminimumblocksizeof4KB.

Howdowebackupvotingdisks?
1)Oraclerecommendsthatyoubackupyourvotingdiskaftertheinitialclustercreationandafterwecomplete
anynodeadditionordeletionprocedures.
2)First,asrootuser,stopOracleClusterware(withthecrsctlstopcrscommand)onallnodes.Then,determine
thecurrentvotingdiskbyissuingthefollowingcommand:
crsctlqueryvotediskcss
3)Then,issuetheddorocopycommandtobackupavotingdisk,asappropriate.
Givethesyntaxofbackingupvotingdisks:
OnLinuxorUNIXsystems:
ddif=voting_disk_nameof=backup_file_name
where,
voting_disk_nameisthenameoftheactivevotingdisk
backup_file_nameisthenameofthefiletowhichwewanttobackupthevotingdiskcontents
OnWindowssystems,usetheocopycommand:
copyvoting_disk_namebackup_file_name

HowdoweverifyanexistingcurrentbackupofOCR?
WecanverifythecurrentbackupofOCRusingthefollowingcommand:ocrconfigshowbackup

YouhavelostOCRdisk,whatisyournextstep?
Theclusterstackwillbedownduetothefactthatcssdisunabletomaintaintheintegrity,thisistruein10g,
From11gR2onwards,thecrsdstackwillbedown,thehasdstillupandrunning.Youcanaddtheocrbackby
restoringtheautomaticbackuporimportthemanualbackup,

WhatarethemajorRACwaitevents?
InaRACenvironmentthebuffercacheisglobalacrossallinstancesintheclusterandhencetheprocessing
differs.Themostcommonwaiteventsrelatedtothisaregccrrequestandgcbufferbusy

GCCRrequest:thetimeittakestoretrievethedatafromtheremotecache
Reason:RACTrafficUsingSlowConnectionorInefficientqueries(poorlytunedquerieswillincreasethe
amountofdatablocksrequestedbyanOraclesession.Themoreblocksrequestedtypicallymeansthemore
oftenablockwillneedtobereadfromaremoteinstanceviatheinterconnect.)

GCBUFFERBUSY:Itisthetimetheremoteinstancelocallyspendsaccessingtherequesteddatablock.

WhatdoyoudoifyouseeGCCRBLOCKLOSTintop5TimedEventsinAWRReport?
Thisismostlikelyduetoafaultininterconnectnetwork.
Checknetstats
ifyousee"fragmentsdropped"or"packetreassembliesfailed",Workwithyoursystemadministratorfindthe
faultwithnetwork.

Howdoyoutroubleshootnodereboot?
Pleasecheckmetalink...
Note265769.1TroubleshootingCRSReboots
Note.559365.1UsingDiagwaitasadiagnostictogetmoreinformationfordiagnosingOracleClusterwareNode
evictions.

Srvctlcannotstartinstance,IgetthefollowingerrorPRKP1001CRS0215,howeversqlpluscanstartit
onbothnodes?Howdoyouidentifytheproblem?
SettheenvironmentalvariableSRVM_TRACEtotrue..Andstarttheinstancewithsrvctl.Nowyouwillget
detailederrorstack.

WhatareOracleClusterwareprocessesfor10gonUnixandLinux?
ClusterSynchronizationServices(ocssd)Managesclusternodemembershipandrunsastheoracle
userfailureofthisprocessresultsinclusterrestart.
ClusterReadyServices(crsd)Thecrsprocessmanagesclusterresources(whichcouldbeadatabase,an
instance,aservice,aListener,avirtualIP(VIP)address,anapplicationprocess,andsoon)basedonthe
resource'sconfigurationinformationthatisstoredintheOCR.Thisincludesstart,stop,monitorandfailover
operations.Thisprocessrunsastherootuser

Eventmanagerdaemon(evmd)Abackgroundprocessthatpublisheseventsthatcrscreates.

ProcessMonitorDaemon(OPROCD)ThisprocessmonitortheclusterandprovideI/Ofencing.OPROCD
performsitscheck,stopsrunning,andifthewakeupisbeyondtheexpectedtime,thenOPROCDresetsthe
processorandrebootsthenode.AnOPROCDfailureresultsinOracleClusterwarerestartingthenode.
OPROCDusesthehangchecktimeronLinuxplatforms.

RACG(racgmain,racgimon)ExtendsclusterwaretosupportOraclespecificrequirementsandcomplex
resources.RunsservercalloutscriptswhenFANeventsoccur.

WhatareOracledatabasebackgroundprocessesspecifictoRAC?
OracleRACiscomposedoftwoormoredatabaseinstances.TheyarecomposedofMemorystructuresand
backgroundprocessessameasthesingleinstancedatabase.OracleRACinstancesusetwoprocesses
GES(GlobalEnqueueService),GCS(GlobalCacheService)thatenablecachefusion.OracleRACinstancesare
composedoffollowingbackgroundprocesses:
ACMSAtomicControlfiletoMemoryService(ACMS)
GTX0jGlobalTransactionProcess
LMONGlobalEnqueueServiceMonitor
LMDGlobalEnqueueServiceDaemon
LMSGlobalCacheServiceProcess
LCK0InstanceEnqueueProcess
RMSnOracleRACManagementProcesses(RMSn)
RSMNRemoteSlaveMonitor
ToensurethateachOracleRACdatabaseinstanceobtainstheblockthatitneedstosatisfyaqueryor
transaction,OracleRACinstancesusetwoprocesses,theGlobalCacheService(GCS)andtheGlobal
EnqueueService(GES).TheGCSandGESmaintainrecordsofthestatusesofeachdatafileandeachcached
blockusingaGlobalResourceDirectory(GRD).TheGRDcontentsaredistributedacrossalloftheactive
instances.

WhatisGRD?
GRDstandsforGlobalResourceDirectory.TheGESandGCSmaintainsrecordsofthestatusesofeach
datafileandeachcahedblockusingglobalresourcedirectory.Thisprocessisreferredtoascachefusionand
helpsindataintegrity.

WhatisACMS?
ACMSstandsforAtomicControlfileMemoryService.InanOracleRACenvironmentACMSisanagentthat
ensuresadistributedSGAmemoryupdate(ie)SGAupdatesaregloballycommittedonsuccessorglobally
abortedineventofafailure.

WhatisSCANlistener?
Ascanlistenerissomethingthatadditionaltonodelistenerwhichlistenstheincomingdbconnectionrequests
fromtheclientwhichgotthroughthescanIP,itgotendpointsconfiguredtonodelistenerwhereitroutesthedb
connectionrequeststoparticularnodelistener.

SCANIPcanbedisabledifnotrequired.HoweverSCANIPismandatoryduringtheRACinstallation.
Enabling/disablingSCANIPismostlyusedinoracleappsenvironmentbytheconcurrentmanager(kindofjob
schedulerinoracleapps).
StepstodisabletheSCANIP,
i.DonotuseSCANIPattheclientend.
ii.Stopscanlistener
srvctlstopscan_listener
iii.Stopscan
srvctlstopscan(thiswillstopthescanvip's)
iv.Disablescananddisablescanlistener
srvctldisablescan

Whatarethedifferentnetworkcomponentsarein10gRAC?
public,private,andvipcomponents
Privateinterfacesisforintranodecommunication.
VIPisallaboutavailabilityofapplication.WhenanodefailsthentheVIPcomponentfailovertosomeother
node,thisisthereasonthatallapplicationsshouldbasedonvipcomponentsmeanstnsentriesshouldhavevip
entryinthehostlist

Whatisaninterconnectnetwork?
Aninterconnectnetworkisaprivatenetworkthatconnectsalloftheserversinacluster.Theinterconnect
networkusesaswitch/multipleswitchesthatonlythenodesintheclustercanaccess.

Whatistheuseofclusterinterconnect?
ClusterinterconnectisusedbytheCachefusionforinterinstancecommunication.

Howcanweconfiguretheclusterinterconnect?
ConfigureUserDatagramProtocol(UDP)onGigabitEthernetforclusterinterconnects.
OnUNIXandLinuxsystemsweuseUDPandRDS(Reliabledatasocket)protocolstobeusedbyOracle
Clusterware.
WindowsclustersusetheTCPprotocol.

WhatisthepurposeofPrivateInterconnect?
Clusterwareusestheprivateinterconnectforclustersynchronization(networkheartbeat)anddaemon
communicationbetweenthetheclusterednodes.ThiscommunicationisbasedontheTCPprotocol.
RACusestheinterconnectforcachefusion(UDP)andinterprocesscommunication(TCP).CacheFusionis
theremotememorymappingofOraclebuffers,sharedbetweenthecachesofparticipatingnodesinthecluster.

WhatisavirtualIPaddressorVIP?
AvirtualIPaddressorVIPisanalternateIPaddressthattheclientconnectionsuseinsteadofthestandard
publicIPaddress.ToconfigureVIPaddress,weneedtoreserveaspareIPaddressforeachnode,andtheIP
addressesmustusethesamesubnetasthepublicnetwork.

WhatistheuseofVIP?
Ifanodefails,thenthenode'sVIPaddressfailsovertoanothernodeonwhichtheVIPaddresscanacceptTCP
connectionsbutitcannotacceptOracleconnections.

WhydowehaveaVirtualIP(VIP)inOracleRAC?
WithoutusingVIPsorFAN,clientsconnectedtoanodethatdiedwilloftenwaitforaTCPtimeoutperiod(which
canbeupto10min)beforegettinganerror.Asaresult,youdon'treallyhaveagoodHAsolutionwithoutusing
VIPs.
Whenanodefails,theVIPassociatedwithitisautomaticallyfailedovertosomeothernodeandnewnodere
arpstheworldindicatinganewMACaddressfortheIP.SubsequentpacketssenttotheVIPgotothenew
node,whichwillsenderrorRSTpacketsbacktotheclients.Thisresultsintheclientsgettingerrors
immediately.

GivesituationsunderwhichVIPaddressfailoverhappens?
VIPaddressesfailoverhappenswhenthenodeonwhichtheVIPaddressrunsfailsallinterfacesfortheVIP
addressfails,allinterfacesfortheVIPaddressaredisconnectedfromthenetwork.

WhatisthesignificanceofVIPaddressfailover?
WhenaVIPaddressfailoverhappens,ClientsthatattempttoconnecttotheVIPaddressreceivearapid
connectionrefusederror.Theydon'thavetowaitforTCPconnectiontimeoutmessages.

WhatistheuseofaserviceinOracleRACenvironment?
ApplicationsshouldusetheservicesfeaturetoconnecttotheOracledatabase.Servicesenableustodefine
rulesandcharacteristicstocontrolhowusersandapplicationsconnecttodatabaseinstances.

WhatarethecharacteristicscontrolledbyOracleservicesfeature?
Thecharacteristicsincludeauniquename,workloadbalancing,failoveroptions,andhighavailability.

WhatenablestheloadbalancingofapplicationsinRAC?
OracleNetServicesenabletheloadbalancingofapplicationconnectionsacrossalloftheinstancesinan
OracleRACdatabase.

Whatarethetypesofconnectionloadbalancing?
ConnectionWorkloadmanagementisoneofthekeyaspectswhenyouhaveRACinstancesasyouwantto
distributetheconnectionstospecificnodes/instanceorthosehavelessload.
Therearetwotypesofconnectionloadbalancing:
1.ClientSideloadbalancing(alsocalledasconnecttimeloadbalancing)
2.Serversideloadbalancing(alsocalledasListenerconnectionloadbalancing)

Whatisthedifferencebetweenserversideandclientsideconnectionloadbalancing?
Clientsidebalancinghappensatclientsidewhereloadbalancingisdoneusinglistener.Incaseofserverside
loadbalancinglistenerusesaloadbalancingadvisorytoredirectconnectionstotheinstanceprovidingbest
service.

ClientSideloadbalancing:Oracleclientsideloadbalancingfeatureenablesclientstorandomizethe
connectionrequestsamongalltheavailablelistenersbasedontheirload.

Antnsentrythatcontainsallnodesentriesanduseload_balance=on(defaultitson)willusetheconnecttime
loadbalancingorclientsideloadbalancing.

SampleClientSideTNSEntry:

finance=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=myrac2vip)(PORT=2042))
(ADDRESS=(PROTOCOL=TCP)(HOST=myrac1vip)(PORT=2042))
(ADDRESS=(PROTOCOL=TCP)(HOST=myrac3vip)(PORT=2042))
(LOAD_BALANCE=yes)
(CONNECT_DATA=
(SERVER=DEDICATED)
(SERVICE_NAME=FINANCE)(FAILOVER=ON)
(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC)(RETRIES=180)(DELAY=5))
)
)

Serversideloadbalancing:Thisimprovestheconnectionperformancebybalancingthenumberofactive
connectionsamongmultipleinstancesanddispatchers.Inasingleinstanceenvironment(sharedservers),the
listenerselectstheleastdispatchertohandletheincomingclientrequests.Inaracenvironments,PMONis
awareofallinstancesloadanddispatchers,anddependingontheloadinformationPMONredirectsthe
connectiontotheleastloadednode.

InaRACenvironment,*.remote_listenerparameterwhichisatnsentrycontainingallnodesaddressesneedto
settoenabletheloadbalanceadvisoryupdatestoPMON.

SampleTnsentryshouldbeinaninstancesofRACcluster,
local_listener=LISTENER_MYRAC1
remote_listener=LISTENERS_MYRACDB

WhataretheadministrativetoolsusedforOracleRACenvironments?
OracleRACclustercanbeadministeredasasingleimageusingthebelow
OEM(EnterpriseManager),
SQL*PLUS,
Servercontrol(SRVCTL),
ClusterVerificationUtility(CLUVFY),
DBCA,
NETCA

NamesomeOracleClusterwaretoolsandtheiruses?
OIFCFGallocatinganddeallocatingnetworkinterfaces.
OCRCONFIGCommandlinetoolformanagingOracleClusterRegistry.
OCRDUMPIdentifytheinterconnectbeingused.
CVUClusterverificationutilitytogetstatusofCRSresources.

WhatisthedifferencebetweenCRSCTLandSRVCTL?
crsctlmanagesclusterwarerelatedoperations:
StartingandstoppingOracleClusterware
EnablinganddisablingOracleClusterwaredaemons
Registeringclusterresources

srvctlmanagesOracleresourcerelatedoperations:
Startingandstoppingdatabaseinstancesandservices
Alsofrom11gR2managestheclusterresourceslikenetwork,vip,disksetc

HowdoweremoveASMfromaOracleRACenvironment?
Weneedtostopanddeletetheinstanceinthenodefirstininteractiveorsilentmode.Afterthatasmcanbe
removedusingsrvctltoolasfollows:
srvctlstopasmnnode_name
srvctlremoveasmnnode_name
WecanverifyifASMhasbeenremovedbyissuingthefollowingcommand:
srvctlconfigasmnnode_name

HowdoweverifythataninstancehasbeenremovedfromOCRafterdeletinganinstance?
Issuethefollowingsrvctlcommand:
srvctlconfigdatabaseddatabase_name
cdCRS_HOME/bin
./crs_stat
WhatarethemodesofdeletinginstancesfromORacleRealApplicationclusterDatabases?
WecandeleteinstancesusingsilentmodeorinteractivemodeusingDBCA(DatabaseConfigurationAssistant).

Whatarethebackgroundprocessthatexistsin11gr2andfunctionality?
ProcessNameFunctionality
crsdTheCRSdaemon(crsd)managesclusterresourcesbasedonconfigurationinformationthatisstored
inOracleClusterRegistry(OCR)foreachresource.Thisincludesstart,stop,monitor,andfailoveroperations.
Thecrsdprocessgenerateseventswhenthestatusofaresourcechanges.
cssdClusterSynchronizationService(CSS):Managestheclusterconfigurationbycontrollingwhichnodes
aremembersoftheclusterandbynotifyingmemberswhenanodejoinsorleavesthecluster.Ifyouareusing
certifiedthirdpartyclusterware,thenCSSprocessesinterfaceswithyourclusterwaretomanagenode
membershipinformation.CSShasthreeseparateprocesses:theCSSdaemon(ocssd),theCSSAgent
(cssdagent),andtheCSSMonitor(cssdmonitor).Thecssdagentprocessmonitorstheclusterandprovides
input/outputfencing.ThisserviceformerlywasprovidedbyOracleProcessMonitordaemon(oprocd),also
knownasOraFenceServiceonWindows.AcssdagentfailureresultsinOracleClusterwarerestartingthenode.
diskmonDiskMonitordaemon(diskmon):Monitorsandperformsinput/outputfencingforOracleExadata
StorageServer.AsExadatastoragecanbeaddedtoanyOracleRACnodeatanypointintime,thediskmon
daemonisalwaysstartedwhenocssdisstarted.
evmdEventManager(EVM):IsabackgroundprocessthatpublishesOracleClusterwareevents
mdnsdMulticastdomainnameservice(mDNS):AllowsDNSrequests.ThemDNSprocessisabackground
processonLinuxandUNIX,andaserviceonWindows.
gnsdOracleGridNamingService(GNS):IsagatewaybetweentheclustermDNSandexternalDNS
servers.TheGNSprocessperformsnameresolutionwithinthecluster.
onsOracleNotificationService(ONS):IsapublishandsubscribeserviceforcommunicatingFast
ApplicationNotification(FAN)events
oraagentoraagent:ExtendsclusterwaretosupportOraclespecificrequirementsandcomplexresources.It
runsservercalloutscriptswhenFANeventsoccur.ThisprocesswasknownasRACGinOracleClusterware
11gRelease1(11.1).
orarootagentOraclerootagent(orarootagent):IsaspecializedoraagentprocessthathelpsCRSDmanage
resourcesownedbyroot,suchasthenetwork,andtheGridvirtualIPaddress
oclskdClusterkilldaemon(oclskd):Handlesinstance/nodeevictionsrequeststhathavebeenescalatedto
CSS
gipcdGridIPCdaemon(gipcd):Isahelperdaemonforthecommunicationsinfrastructure
ctssdClustertimesynchronisationdaemon(ctssd)tomanagethetimesyncrhonizationbetweennodes,
ratherdependingonNTP

Underwhichuserorownertheprocesswillstart?
ComponentNameoftheProcessOwner
OracleHighAvailabilityServiceohasdinit,root
ClusterReadyService(CRS)ClusterReadyServicesroot
ClusterSynchronizationService(CSS)ocssd,cssdmonitor,cssdagentgridowner
EventManager(EVM)evmd,evmloggergridowner
ClusterTimeSynchronizationService(CTSS)octssdroot
OracleNotificationService(ONS)ons,eonsgridowner
OracleAgentoragentgridowner
OracleRootAgentorarootagentroot
GridNamingService(GNS)gnsdroot
GridPlugandPlay(GPnP)gpnpdgridowner
Multicastdomainnameservice(mDNS)mdnsdgridowner

Whatisthemajordifferencebetween10gand11gRAC?
Thereisnotmuchdifferencebetween10gand11gR(1)RAC.Butthereisasignificantdifferencein11gR2.

Priorto11gR1(10g)RAC,thefollowingweremanagedbyOracleCRS
Databases
Instances
Applications
NodeMonitoring
EventServices
HighAvailability

From11gR2(onwards)itscompletedHAstackmanagingandprovidingthefollowingresourcesasliketheother
clustersoftwarelikeVCSetc.
Databases
Instances
Applications
ClusterManagement
NodeManagement
EventServices
HighAvailability
NetworkManagement(providesDNS/GNS/MDNSDservicesonbehalfofothertraditionalservices)and
SCANSingleAccessClientNamingmethod,HAIP
StorageManagement(withhelpofASMandothernewACFSfilesystem)
Timesynchronization(ratherdependingupontraditionalNTP)
RemovedOSdependenthangcheckeretc,manageswithownadditionalmonitorprocess

Whatishangchecktimer?
Thehangchecktimerchecksregularlythehealthofthesystem.Ifthesystemhangsorstopthenodewillbe
restartedautomatically.
Thereare2keyparametersforthismodule:
>hangchecktick:thisparameterdefinestheperiodoftimebetweenchecksofsystemhealth.Thedefault
valueis60secondsOraclerecommendssettingitto30seconds.
>hangcheckmargin:thisdefinesthemaximumhangdelaythatshouldbetoleratedbeforehangchecktimer
resetstheRACnode.

StatetheinitializationparametersthatmusthavesamevalueforeveryinstanceinanOracleRAC
database?
Someinitializationparametersarecriticalatthedatabasecreationtimeandmusthavesamevalues.Theirvalue
mustbespecifiedinSPFILEorPFILEforeveryinstance.Thelistofparametersthatmustbeidenticalonevery
instancearegivenbelow:
ACTIVE_INSTANCE_COUNT
ARCHIVE_LAG_TARGET
COMPATIBLE
CLUSTER_DATABASE
CLUSTER_DATABASE_INSTANCE
CONTROL_FILES
DB_BLOCK_SIZE
DB_DOMAIN
DB_FILES
DB_NAME
DB_RECOVERY_FILE_DEST
DB_RECOVERY_FILE_DEST_SIZE
DB_UNIQUE_NAME
INSTANCE_TYPE(RDBMSorASM)
PARALLEL_MAX_SERVERS
REMOTE_LOGIN_passWORD_FILE
UNDO_MANAGEMENT

WhatisRAC?WhatisthebenefitofRACoversingleinstancedatabase?
InRealApplicationClustersenvironments,allnodesconcurrentlyexecutetransactionsagainstthesame
database.RealApplicationClusterscoordinateseachnode'saccesstotheshareddatatoprovideconsistency
andintegrity.
Benefits:
Improveresponsetime
Improvethroughput
Highavailability
Transparency

AdvantagesofRAC(RealApplicationClusters)

Reliabilityifonenodefails,thedatabasewon'tfail
Availabilitynodescanbeaddedorreplacedwithouthavingtoshutdownthedatabase
Scalabilitymorenodescanbeaddedtotheclusterastheworkloadincreases

WhatisavirtualIPaddressorVIP?

AvirtualIPaddressorVIPisanalternateIPaddressthattheclientconnectionsuseinsteadofthestandard
publicIPaddress.ToconfigureVIPaddress,weneedtoreserveaspareIPaddressforeachnode,andtheIP
addressesmustusethesamesubnetasthepublicnetwork.

WhatistheuseofVIP?
Ifanodefails,thenthenode'sVIPaddressfailsovertoanothernodeonwhichtheVIPaddresscanacceptTCP
connectionsbutitcannotacceptOracleconnections.
GivesituationsunderwhichVIPaddressfailoverhappens:
VIPaddressesfailoverhappenswhenthenodeonwhichtheVIPaddressrunsfails,allinterfacesfortheVIP
addressfails,allinterfacesfortheVIPaddressaredisconnectedfromthenetwork.
UsingvirtualIPwecansaveourTCP/IPtimeoutproblembecauseOraclenotificationservicemaintains
communicationbetweeneachnodesandlisteners.

WhatisthesignificanceofVIPaddressfailover?
WhenaVIPaddressfailoverhappens,ClientsthatattempttoconnecttotheVIPaddressreceivearapid
connectionrefusederror.Theydon'thavetowaitforTCPconnectiontimeoutmessages.

Whatisvotingdisk?
VotingDiskisafilethatsitsinthesharedstorageareaandmustbeaccessiblebyallnodesinthecluster.All
nodesintheclusterregisterstheirheartbeatinformationinthevotingdisk,soastoconfirmthattheyareall
operational.Ifheartbeatinformationofanynodeinthevotingdiskisnotavailablethatnodewillbeevicted
fromthecluster.TheCSS(ClusterSynchronizationService)daemonintheclusterwaremaintainstheheart
beatofallnodestothevotingdisk.Whenanynodeisnotabletosendheartbeattovotingdisk,thenitwill
rebootitself,thushelpavoidingthesplitbrainsyndrome.

Forhighavailability,Oraclerecommendsthatyouhaveaminimumofthreeoroddnumber(3orgreater)of
votingdisks.

VotingDiskisfilethatresidesonsharedstorageandManagesclustermembers.Votingdiskreassignscluster
ownershipbetweenthenodesincaseoffailure.

TheVotingDiskFilesareusedbyOracleClusterwaretodeterminewhichnodesarecurrentlymembersofthe
cluster.ThevotingdiskfilesarealsousedinconcertwithotherClustercomponentssuchasCRStomaintain
theclustersintegrity.

OracleDatabase11gRelease2providestheabilitytostorethevotingdisksinASMalongwiththeOCR.Oracle
ClusterwarecanaccesstheOCRandthevotingdiskspresentinASMeveniftheASMinstanceisdown.Asa
resultCSScancontinuetomaintaintheOracleclustereveniftheASMinstancehasfailed.

Howmanyvotingdisksareyoumaintaining?

http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOracle/tabid/648/TopicID/RACR2ARC6/Default.asp
x

BydefaultOraclewillcreate3votingdiskfilesinASM.

Oracleexpectsthatyouwillconfigureatleast3votingdisksforredundancypurposes.Youshouldalways
configureanoddnumberofvotingdisks>=3.Thisisbecauselossofmorethanhalfyourvotingdiskswill
causetheentireclustertofail.

Youshouldplanonallocating280MBforeachvotingdiskfile.Forexample,ifyouareusingASMandexternal
redundancythenyouwillneedtoallocate280MBofdiskforthevotingdisk.IfyouareusingASMandnormal
redundancyyouwillneed560MB.

Whyweneedtokeepoddnumberofvotingdisks?
Oracleexpectsthatyouwillconfigureatleast3votingdisksforredundancypurposes.Youshouldalways
configureanoddnumberofvotingdisks>=3.Thisisbecauselossofmorethanhalfyourvotingdiskswill
causetheentireclustertofail.

WhatareOracleRACsoftwarecomponents?
OracleRACiscomposedoftwoormoredatabaseinstances.TheyarecomposedofMemorystructuresand
backgroundprocessessameasthesingleinstancedatabase.OracleRACinstancesusetwoprocesses
GES(GlobalEnqueueService),GCS(GlobalCacheService)thatenablecachefusion.OracleRACinstancesare
composedoffollowingbackgroundprocesses:
ACMSAtomicControlfiletoMemoryService(ACMS)
GTX0jGlobalTransactionProcess
LMONGlobalEnqueueServiceMonitor
LMDGlobalEnqueueServiceDaemon
LMSGlobalCacheServiceProcess
LCK0InstanceEnqueueProcess
RMSnOracleRACManagementProcesses(RMSn)
RSMNRemoteSlaveMonitor

WhatareOracleClusterwareprocessesfor10g?
ClusterSynchronizationServices(ocssd)Managesclusternodemembershipandrunsastheoracleuser
failureofthisprocessresultsinclusterrestart.
ClusterReadyServices(crsd)Thecrsprocessmanagesclusterresources(whichcouldbeadatabase,an
instance,aservice,aListener,avirtualIP(VIP)address,anapplicationprocess,andsoon)basedonthe
resource'sconfigurationinformationthatisstoredintheOCR.Thisincludesstart,stop,monitorandfailover
operations.Thisprocessrunsastherootuser
Eventmanagerdaemon(evmd)Abackgroundprocessthatpublisheseventsthatcrscreates.
ProcessMonitorDaemon(OPROCD)ThisprocessmonitortheclusterandprovideI/Ofencing.OPROCD
performsitscheck,stopsrunning,andifthewakeupisbeyondtheexpectedtime,thenOPROCDresetsthe
processorandrebootsthenode.AnOPROCDfailureresultsinOracleClusterwarerestartingthenode.
OPROCDusesthehangchecktimeronLinuxplatforms.
RACG(racgmain,racgimon)ExtendsclusterwaretosupportOraclespecificrequirementsandcomplex
resources.RunsservercalloutscriptswhenFANeventsoccur.

WhatareOracledatabasebackgroundprocessesspecifictoRAC?
LMSGlobalCacheServiceProcess
LMDGlobalEnqueueServiceDaemon
LMONGlobalEnqueueServiceMonitor
LCK0InstanceEnqueueProcess
OracleRACinstancesusetwoprocesses,theGlobalCacheService(GCS)andtheGlobalEnqueueService
(GES).TheGCSandGESmaintainrecordsofthestatusesofeachdatafileandeachcachedblockusinga
GlobalResourceDirectory(GRD).TheGRDcontentsaredistributedacrossalloftheactiveinstances.

WhatisCacheFusion?
Transforofdataacrossinstancesthroughprivateinterconnectiscalledcachefusion.OracleRACiscomposed
oftwoormoreinstances.Whenablockofdataisreadfromdatafilebyaninstancewithintheclusterand
anotherinstanceisinneedofthesameblock,itiseasytogettheblockimagefromtheinsatncewhichhasthe
blockinitsSGAratherthanreadingfromthedisk.ToenableinterinstancecommunicationOracleRACmakes
useofinterconnects.TheGlobalEnqueueService(GES)monitorsandInstanceenqueueprocessmanagesthe
cahcefusion

WhatisSCAN?(11gR2feature)
SingleClientAccessName(SCAN)issanewOracleRealApplicationClusters(RAC)11gRelease2feature
thatprovidesasinglenameforclientstoaccessanOracleDatabaserunninginacluster.Thebenefitisclients
usingSCANdonotneedtochangeifyouaddorremovenodesinthecluster.

SCANprovidesasingledomainnamevia(DNS),allowinganduserstoaddressaRACclusterasifitwerea
singleIPaddress.SCANworksbyreplacingahostnameorIPlistwithvirtualIPaddresses(VIP).

Singleclientaccessname(SCAN)ismeanttofacilitatesinglenameforallOracleclientstoconnecttothe
clusterdatabase,irrespectiveofnumberofnodesandnodelocation.Untilnow,wehavetokeepadding
multipleaddressrecordsinallclientstnsnames.ora,whenanewnodegetsaddedtoordeletedfromthe
cluster.

SingleClientAccessName(SCAN)eliminatestheneedtochangeTNSNAMESentrywhennodesareaddedto
orremovedfromtheCluster.RACinstancesregistertoSCANlistenersasremotelisteners.Oracle
recommendsassigning3addressestoSCAN,whichwillcreate3SCANlisteners,thoughtheclusterhasgot
dozensofnodes..SCANisadomainnameregisteredtoatleastoneanduptothreeIPaddresses,eitherin
DNS(DomainNameService)orGNS(GridNamingService).TheSCANmustresolvetoatleastoneaddress
onthepublicnetwork.Forhighavailabilityandscalability,OraclerecommendsconfiguringtheSCANtoresolve
tothreeaddresses.
http://www.freeoraclehelp.com/2011/12/scansetupfororacle11grelease211gr2.html
WhatareSCANcomponentsinacluster?
1.SCANName
2.SCANIPs(3)
3.SCANListeners(3)
WhatisFAN?
FastapplicationNotificationasitabbreviatestoFANrelatestotheeventsrelatedtoinstances,servicesand
nodes.ThisisanotificationmechanismthatOracleRAcusestonotifyotherprocessesabouttheconfiguration
andservicelevelinformationthatincludesservicestatuschangessuchas,UPorDOWNevents.Applications
canrespondtoFANeventsandtakeimmediateaction.

WhatisTAF?
TAF(TransparentApplicationFailover)isaconfigurationthatallowssessionfailoverbetweendifferentnodes
ofaRACdatabasecluster.
TransparentApplicationFailover(TAF).Ifacommunicationlinkfailureoccursafteraconnectionisestablished,
theconnectionfailsovertoanotheractivenode.Anydisruptedtransactionsarerolledback,andsession
propertiesandserversideprogramvariablesarelost.Insomecases,ifthestatementexecutingatthetimeof
thefailoverisaSelectstatement,thatstatementmaybeautomaticallyreexecutedonthenewconnectionwith
thecursorpositionedontherowonwhichitwaspositionedpriortothefailover.

AfteranOracleRACnodecrashesusuallyfromahardwarefailureallnewapplicationtransactionsare
automaticallyreroutedtoaspecifiedbackupnode.Thechallengeinreroutingistonotlosetransactionsthat
were"inflight"attheexactmomentofthecrash.Oneoftherequirementsofcontinuousavailabilityistheability
torestartinflightapplicationtransactions,allowingafailednodetoresumeprocessingonanotherserver
withoutinterruption.Oracle'sanswertoapplicationfailoverisanewOracleNetmechanismdubbed
TransparentApplicationFailover.TAFallowstheDBAtoconfigurethetypeandmethodoffailoverforeach
OracleNetclient.
TAFarchitectureofferstheabilitytorestarttransactionsateitherthetransaction(SELECT)orsessionlevel.

WhataretherequirementsforOracleClusterware?
1.ExternalSharedDisktostoreOracleClusterwarefile(VotingDiskandOracleClusterRegistryOCR)
2.Twonetwrokcardsoneachclusterwarenode(andthreesetofIPaddress)
NetworkCard1(withIPaddressset1)forpublicnetwork
NetworkCard2(withIPaddressset2)forprivatenetwork(forinternodecommunicationbetweenracnodes
usedbyclusterwareandracdatabase)
IPaddressset3forVirtualIP(VIP)(usedasVirtualIPaddressforclientconnectionandforconnectionfailover)
3.StorageOptionforOCRandVotingDiskRAW,OCFS2(OracleClusterFileSystem),NFS,..
WhichenabletheloadbalancingofapplicationsinRAC?
OracleNetServicesenabletheloadbalancingofapplicationconnectionsacrossalloftheinstancesinan
OracleRACdatabase.

HowtofindlocationofOCRfilewhenCRSisdown?
IfyouneedtofindthelocationofOCR(OracleClusterRegistry)butyourCRSisdown.
WhentheCRSisdown:
Lookintoocr.locfile,locationofthisfilechangesdependingontheOS:
OnLinux:/etc/oracle/ocr.loc
OnSolaris:/var/opt/oracle/ocr.loc
WhenCRSisUP:
SetASMenvironmentorCRSenvironmentthenrunthebelowcommand:
ocrcheck

In2nodeRAC,howmanyNICsarerusing?
2networkcardsoneachclusterwarenode
NetworkCard1(withIPaddressset1)forpublicnetwork
NetworkCard2(withIPaddressset2)forprivatenetwork(forinternodecommunicationbetweenracnodes
usedbyclusterwareandracdatabase)

In2nodeRAC,howmanyIPsarerusing?
63setofIPaddress
##eth1Public:2
##eth0Private:2
##VIP:2

HowtofindIPsinformationinRAC?
Editthe/etc/hostsfileasshownbelow:
#Donotremovethefollowingline,orvariousprograms
#thatrequiresnetworkfunctionalitywillfail.
127.0.0.1localhost.localdomainlocalhost
##PublicNodenames
192.168.10.11node1pub.hingu.netnode1pub
192.168.10.22node2pub.hingu.netnode2pub
##PrivateNetwork(Interconnect)
192.168.0.11node1prvnode1prv
192.168.0.22node2prvnode2prv
##PrivateNetwork(NetworkAreastorage)
192.168.1.11node1nasnode1nas
192.168.1.22node2nasnode2nas
192.168.1.33nasservernasserver
##VirtualIPs
192.168.10.111node1vip.hingu.netnode1vip
192.168.10.222node2vip.hingu.netnode2vip

WhatisdifferencebetweenRACipaddresses?
PublicIPadressisthenormalIPaddresstypicallyusedbyDBAandSAtomanagestorage,systemand
database.PublicIPaddressesarereservedfortheInternet.
PrivateIPaddressisusedonlyforinternalclusteringprocessing(CacheFusion)(akaasinterconnect).Private
IPaddressesarereservedforprivatenetworks.
VIPisusedbydatabaseapplicationstoenablefailoverwhenoneclusternodefails.Thepurposeforhaving
VIPissoclientconnectioncanbefailovertosurvivingnodesincasethereisfailure
Canapplicationdeveloperaccesstheprivateip?
No.privateIPaddressisusedonlyforinternalclusteringprocessing(CacheFusion)(akaasinterconnect)
http://rajeevjhaoracle.wordpress.com/2012/01/02/oracleracinterviewquestionanswer/

WhatisRAC?
RACArchitechture?

WhatisaSCANListener?

WhatisVIP?

WhatisTAF?

WhatisFAN?

WhatisLMON?

WhatisCACHEFUSION?

WhatisSPLITBRAINinRAC?

HowLoadbalancingworksinORACLERAC?

WhatisVotingDisk?
HowmanybackupsarethereforVotingDisk?
WHYODDNUMBEROFVOTINGDISKISUSED?
HowtotakeVotingDiskbackup?DoyoutakeVotingDiskbackup?
IfyoulostVotingDisk,howdoyouRestore?

WhatisOCR?
HowmanybackupsarethereforOCR?
HowtotakeOCRbackup?DoyoutakeOCRbackup?
IfyoulostOCR,howdoyouRestore?

WhatisaMasterNode?
HowdoyouknowwhichistheMasterNodewithoutconnectingtoDatabase?

CanwehaveNode1inSOLARISOSandNode2inLINUXOS?

CommomnlyusedcommandsinRAC?

HowtoconvertstandalonedatabasetoRAC?