RAC/ASM/VOTINGDISKInterviewQuestions&Answer

10

Amrhassan

oracledatabaseconsultant

QWhatisSCAN?

SingleClientAccessName(SCAN)issanewOracleRealApplicationClusters(RAC)11gRelease2feature
thatprovidesasinglenameforclientstoaccessanOracleDatabaserunninginacluster.Thebenefitisclients
usingSCANdonotneedtochangeifyouaddorremovenodesinthecluster.

Qwhatisdynamicremastering?Whenwillthedynamicremasteringhappens?

dynamicremasteringisabilitytomovetheownershipofresourcefromoneinstancetoanotherinstanceinRAC.
dynamicresourceremasteringisusedtoimplementforresourceaffinityforincreasedperformance.resource
affinityoptimizedthesysteminsituationwhereupdatetransactionsarebeingexecutedinoneinstance.when
activityshifttoanotherinstancetheresourceaffinitycorrespondinglymovetoanotherinstance.Ifactivityisnot
localizedthenresourceownershipishashedtotheinstance.

In10gdynamicremasteringhappensinfile+objectlevel.theprocessofremasteringisverystringent.Forone
instanceshouldtouchmorethan50timesthantheotherinstanceinparticularperiod(say10mints).thistouch
ratioandtimecanbetunedbygc_affinity_limitand_gc_affinity_timeparameter.

Qwhywerequiredtomaintainoddnumberofvotingdisks?

Oddnumberofdiskaretoavoidsplitbrain,WhenNodesinclustercan'ttalktoeachothertheyruntolockthe
Votingdiskandwhoeverlockthemorediskwillsurvive,ifdisknumberareeventherearechancesthatnode
mightlock50%ofdisk(2outof4)thenhowtodecidewhichnodetoevict.
whereaswhennumberisodd,onewillbehigherthanotherandeachforclustertoevictthenodewithless
number

QHowyoucheckthehealthofYourRACDatabase?

'crsctl'commandfromrootororacleusercanbeusedtochecktheclusterwarehealthButforstartingor
stoppingwehavetouserootuseroranyprivilegeuser.

[oracle@TEST_NODE1~]$crsctlcheckcrs
CSSappearshealthy
CRSappearshealthy
EVMappearshealthy

QHowyouchecktheservicesinRACNode?

Wecanchecktheserviceorstarttheserviceswith'srvctl'command.loadbalanced/TAFservicenamedRAC
online.

[oracle@TEST_NODE1~]$srvctlstartservicedorclsRAC
[oracle@TEST_NODE1~]$crsstat

QIfthereissomeissuewithvirtualIPhowwillyoutroubleshootit?Howwillyouchangevirtualip?

TochangetheVIP(virtualIP)onaRACnode,usethecommand
[oracle@testnodeoracle]$srvctlmodifynodeappsAnew_address

QHowyouwillbackupyourRACDatabase?

BackupstrategyofRACDatabase:
AnRACDatabaseconsistsof
1)OCR
2)Votingdisk&
3)Databasefiles,controlfiles,redologfiles&Archivelogfiles

QDoyouhaveanyideaofloadbalancinginapplication?Howloadbalancingisdone?

http://practicalappsdba.wordpress.com/category/formasterappsdbas/

QWhatisRAC?

RACstandsforRealApplicationcluster.ItisaclusteringsolutionfromOracleCorporationthatensureshigh
availabilityofdatabasesbyprovidinginstancefailover,mediafailoverfeatures.

QWhatisRACandhowisitdifferentfromnonRACdatabases?

RACstandsforRealApplicationCluster,youhavennumberofinstancesrunningintheirownseparatenodes
andbasedonthesharedstorage.Clusteristhekeycomponentandisacollectionofserversoperationsasone
unit.RACisthebestsolutionforhighperformanceandhighavailably.NonRACdatabaseshassinglepointof
failureincaseofhardwarefailureorservercrash.

QGivetheusageofsrvctl?

srvctlstartinstanceddb_namei"inst_name_list"[ostart_options]

srvctlstopinstancednamei"inst_name_list"[ostop_options]
srvctlstopinstancedorcli"orcl3,orcl4"oimmediate

srvctlstartdatabasedname[ostart_options]

srvctlstopdatabasedname[ostop_options]

srvctlstartdatabasedorclomount

QMentiontheOracleRACsoftwarecomponents?

OracleRACiscomposedoftwoormoredatabaseinstances.TheyarecomposedofMemorystructuresand
backgroundprocessessameasthesingleinstancedatabase.OracleRACinstancesusetwoprocesses
GES(GlobalEnqueueService),GCS(GlobalCacheService)thatenablecachefusion.OracleRACinstancesare
composedoffollowingbackgroundprocesses:

ACMSAtomicControlfiletoMemoryService(ACMS)
GTX0jGlobalTransactionProcess
LMONGlobalEnqueueServiceMonitor
LMDGlobalEnqueueServiceDaemon
LMSGlobalCacheServiceProcess
LCK0InstanceEnqueueProcess
RMSnOracleRACManagementProcesses(RMSn)
RSMNRemoteSlaveMonitor

QWhatisGRD?

GRDstandsforGlobalResourceDirectory.TheGESandGCSmaintainsrecordsofthestatusesofeach
datafileandeachcahedblockusingglobalresourcedirectory.Thisprocessisreferredtoascachefusionand
helpsindataintegrity.

QWhatarethedifferentnetworkcomponentsarein10gRAC?

public,private,andvipcomponents
Privateinterfacesisforintranodecommunication.VIPisallaboutavailabilityofapplication.Whenanodefails
thentheVIPcomponentfailovertosomeothernode,thisisthereasonthatallapplicationsshouldbasedonvip
componentsmeanstnsentriesshouldhavevipentryinthehostlist

QGiveDetailsonACMS:

ACMSstandsforAtomicControlfileMemoryService.InanOracleRACenvironmentACMSisanagentthat
ensuresadistributedSGAmemoryupdate(ie)SGAupdatesaregloballycommittedonsuccessorglobally
abortedineventofafailure.

QWhatarethemajorRACwaitevents?
InaRACenvironmentthebuffercacheisglobalacrossallinstancesintheclusterandhencetheprocessing
differs.Themostcommonwaiteventsrelatedtothisaregccrrequestandgcbufferbusy

GCCRrequest:thetimeittakestoretrievethedatafromtheremotecache

Reason:RACTrafficUsingSlowConnectionorInefficientqueries(poorlytunedquerieswillincreasethe
amountofdatablocksrequestedbyanOraclesession.Themoreblocksrequestedtypicallymeansthemore
oftenablockwillneedtobereadfromaremoteinstanceviatheinterconnect.)

GCBUFFERBUSY:Itisthetimetheremoteinstancelocallyspendsaccessingtherequesteddatablock.

QGivedetailsonGTX0j

TheprocessprovidestransparentsupportforXAglobaltransactionsinaRACenvironment.Thedatabase
autotunesthenumberoftheseprocessesbasedontheworkloadofXAglobaltransactions.

QGivedetailsonLMON

Thisprocessmonitorsglobalenquesandresourcesacrosstheclusterandperformsglobalenqueuerecovery
operations.ThisiscalledasGlobalEnqueueServiceMonitor.

QGivedetailsonLMD

Thisprocessiscalledasglobalenqueueservicedaemon.Thisprocessmanagesincomingremoteresource
requestswithineachinstance.

QGivedetailsonLMS

ThisprocessiscalledasGlobalCacheserviceprocess.Thisprocessmaintainsstatusesofdatafilesandeach
cahedblockbyrecordinginformationinaGlobalResourceDectory(GRD).Thisprocessalsocontrolstheflowof
messagestoremoteinstancesandmanagesglobaldatablockaccessandtransmitsblockimagesbetweenthe
buffercachesofdifferentinstances.Thisprocessingisapartofcachefusionfeature.

QGivedetailsonLCK0

ThisprocessiscalledasInstanceenqueueprocess.Thisprocessmanagesnoncachefusionresourcerequests
suchaslibryandrowcacherequests.

QGivedetailsonRMSn

ThisprocessiscalledasOracleRACmanagementprocess.Thesepocessesperformmanagabilitytasksfor
OracleRAC.TasksincludecreationofresourcesrelatedOracleRACwhennewinstancesareaddedtothe
cluster.

QHowtoexportandimportcrsresourceswhilemigratingOracleRACtonewserver.
Belowscriptgeneratesvrctladdscriptfordatabase,instance,serviceand11GlistenersfromOCRfromcurrent
RAC.
SavetheresultofthescriptandrunitatnewRAC.

forDBNAMEin$(srvctlconfigdatabase)
do

#GenerateDBresource

srvctlconfigdatabased$DBNAMEa|awkvdbname="$DBNAME"\
'BEGIN{FS=":"}
$1~/Oraclehome/||$1~/ORACLE_HOME/{dbhome="o"$2}
$1~/Spfile/||$1~/SPFILE/{spfile="p"$2}
$1~/DiskGroups/{dg="a"$2}
END{if(avail=="a"){avail=""}printf"%s%s%s%s%s\n","srvctladddatabased",dbname,dbhome,
spfile,dg}'

#GenerateInstanceresource

srvctlstatusdatabased$DBNAME|awkvdbname="$DBNAME"\
'$4~/running/{printf"%s%s%s%s%s%s\n","srvctladdinstanced",dbname,"i",$2,"n",$7}
$5~/running/{printf"%s%s%s%s%s%s\n","srvctladdinstanced",dbname,"i",$2,"n",$8}'

#Modifyinstancefor10GASMdependency

if[$(echo$ORACLE_HOME|grep"1020"|wcl)eq1]
then
srvctlstatusdatabased$DBNAME|awkvdbname="$DBNAME"\
'$2~/1$/{printf"%s%s%s%s%s\n","srvctlmodifyinstanced",dbname,"i",$2,"s+ASM1"}
$2~/2$/{printf"%s%s%s%s%s\n","srvctlmodifyinstanced",dbname,"i",$2,"s+ASM2"}
$2~/3$/{printf"%s%s%s%s%s\n","srvctlmodifyinstanced",dbname,"i",$2,"s+ASM3"}
$2~/4$/{printf"%s%s%s%s%s\n","srvctlmodifyinstanced",dbname,"i",$2,"s+ASM4"}'
fi

echo"srvctlstartdatabased$DBNAME"

#GenerateServiceresource

snamelist=$(srvctlstatusserviced$DBNAME|awk'{print$2}')

forsnamein$snamelist
do
srvctlconfigserviced$DBNAMEs$sname|awkvdbname="$DBNAME"vsname=$sname\
'BEGIN{FS=":"}
$1~/Preferredinstances/{pref="r"$2}
$1~/PREF/{pref="r"$2sub(/AVAIL/,"",pref)}
$1~/Availableinstances/{avail="a"$2}
$2~/AVAIL/{avail="a"$3}
$1~/Failovertype/{ft="e"$2}
$1~/Failovermethod/{fm="m"$2}
$1~/RuntimeLoadBalancingGoal/{g="B"$2}
END{if(avail=="a"){avail=""}printf"%s%s%s%s%s%s%s%s%s%s\n","srvctladdserviced
",dbname,"s",sname,pref,avail,ft,fm,g,"PBASIC"}'
echo"srvctlstartserviced$DBNAMEs$sname"
done
done

#Listenerat11GHome.10Glistenercan'tbaaddedwithsrvctl.

srvctlconfiglistener|awk\
'BEGIN{FS=":"state=0}
$1~/Name/{lname="l"$2state=1}
$1~/Home/&&state==1{ohome="o"$2state=2}
$1~/Endpoints/&&state==2{lport="p"$3state=3}
state==3{if(ohome!="o"){printf"%s%s%s%s\n","srvctladdlistener",lname,ohome,lport}state=0}'

QGivedetailsonRSMN

ThisprocessiscalledasRemoteSlaveMonitor.Thisprocessmanagesbackgroundslaveprocesscreation
anddcommunicationonremoteinstances.Thisisabackgroundslaveprocess.Thisprocessperformstaskson
behalfofacoordinatingprocessrunninginanotherinstance.

QWhatcomponentsinRACmustresideinsharedstorage?

Alldatafiles,controlfiles,SPFIles,redologfilesmustresideonclusterawareshredstorage.

QWhatisthesignificanceofusingclusterawaresharedstorageinanOracleRACenvironment?

AllinstancesofanOracleRACcanaccessallthedatafiles,controlfiles,SPFILE's,redologfileswhenthesefiles
arehostedoutofclusterawaresharedstoragewhicharegroupofshareddisks.

QGivefewexamplesforsolutionsthatsupportclusterstorage

ASM(automaticstoragemanagement),rawdiskdevices,networkfilesystem(NFS),OCFS2andOCFS(Oracle
ClusterFiesystems).
QWhatisaninterconnectnetwork?

Aninterconnectnetworkisaprivatenetworkthatconnectsalloftheserversinacluster.Theinterconnect
networkusesaswitch/multipleswitchesthatonlythenodesintheclustercanaccess.

QHowcanweconfiguretheclusterinterconnect?

ConfigureUserDatagramProtocol(UDP)onGigabitethernetforclusterinterconnect.Onunixandlinuxsystems
weuseUDPandRDS(Reliabledatasocket)protocolstobeusedbyOracleClusterware.Windowsclustersuse
theTCPprotocol.

QCanweusecrossovercableswithOracleClusterwareinterconnects?

No,crossovercablesarenotsupportedwithOracleClusterwareintercnects.

QWhatistheuseofclusterinterconnect?

ClusterinterconnectisusedbytheCachefusionforinterinstancecommunication.

QHowdousersconnecttodatabaseinanOracleRACenvironment?

UserscanaccessaRACdatabaseusingaclient/serverconfigurationorthroughoneormoremiddletiers,with
orwithoutconnectionpooling.Userscanuseoracleservicesfeaturetoconnecttodatabase.

QWhatistheuseofaserviceinOracleRACenvironment?

ApplicationsshouldusetheservicesfeaturetoconnecttotheOracledatabase.Servicesenableustodefine
rulesandcharacteristicstocontrolhowusersandapplicationsconnecttodatabaseinstances.

QWhatarethecharacteristicscontrolledbyOracleservicesfeature?

Thecharateristicsincludeauniquename,workloadbalancingandfailoveroptions,andhighavailability
characteristics.

QWhatenablestheloadbalancingofapplicationsinRAC?

OracleNetServicesenabletheloadbalancingofapplicationconnectionsacrossalloftheinstancesinan
OracleRACdatabase.

QWhatisavirtualIPaddressorVIP?

AvirtlIPaddressorVIPisanalternateIPaddressthattheclientconnectinsuseinsteadofthestandardpublic
IPaddress.ToconfigureVIPaddress,weneedtoreserveaspareIPaddressforeachnode,andtheIP
addressesmustusethesamesubnetasthepublicnetwork.

QWhatistheuseofVIP?
Ifanodefails,thenthenode'sVIPaddressfailsovertoanothernodeonwhichtheVIPaddresscanacceptTCP
connectionsbutitcannotacceptOracleconnections.

QGivesituationsunderwhichVIPaddressfailoverhappens

VIPaddressesfailoverhappenswhenthenodeonwhichtheVIPaddressrunsfails,allinterfacesfortheVIP
addressfails,allinterfacesfortheVIPaddressaredisconnectedfromthenetwork.

QWhatisthesignificanceofVIPaddressfailover?

WhenaVIPaddressfailoverhappens,ClientsthatattempttoconnecttotheVIPaddressreceivearapid
connectionrefusederror.Theydon'thavetowaitforTCPconnectiontimeoutmessages.

QWhataretheadministrativetoolsusedforOracleRACenvironments?

OracleRACclustercanbeadministeredasasingleimageusingOEM(Enterprise
Manager),SQL*PLUS,Servercontrol(SRVCTL),clusterverificationutility(cvu),DBCA,NETCA

QHowdoweverifythatRACinstancesarerunning?

IssuethefollowingqueryfromanyonenodeconnectingthroughSQL*PLUS.
$connectsys/sysassysdba
SQL>select*fromV$ACTIVE_INSTANCES
ThequerygivestheinstancenumberunderINST_NUMBERcolumn,host_:instancenameunderINST_NAME
column.

QWhatisFAN?

FastapplicationNotificationasitabbreviatestoFANrelatestotheeventsrelatedtoinstances,servicesand
nodes.ThisisanotificationmechanismthatOracleRAcusestonotifyotherprocessesabouttheconfiguration
andservicelevelinformationthatincludesservicestatuschangessuchas,UPorDOWNevents.Applications
canrespondtoFANeventsandtakeimmediateaction.

QWherecanweapplyFANUPandDOWNevents?

FANUPandFANDOWNeventscanbeappliedtoinstances,servicesandnodes.
StatetheuseofFANeventsincaseofaclusterconfigurationchange?
Duringtimesofclusterconfigurationchanges,OracleRAChighavailabilityframeworkpublishesaFANevent
immediatelywhenastatechangeoccursinthecluster.SoapplicationscanreceiveFANeventsandreact
immediately.Thispreventsapplicationsfrompollingdatabaseanddetectingaproblemaftersuchastate
change.

QWhyshouldwehaveseperatehomesforASminstance?

ItisagoodpracticetohaveASMhomeseperatefromthedatabasehom(ORACLE_HOME).Thishelpsin
upgradingandpatchingASMandtheOracledatabasesoftwareindependentofeachother.Also,wecan
deinstalltheOracledatabasesoftwareindependentoftheASMinstance.

QWhatistheadvantageofusingASM?
HavingASMistheOraclerecommendedstorageoptionforRACdatabasesastheASMmaximizes
performancebymanagingthestorageconfigurationacrossthedisks.ASMdoesthisbydistributingthe
databasefileacrossalloftheavailablestoragewithinourclusterdatabaseenvironment.

QWhatisrollingupgrade?

ItisanewASMfeaturefromDatabase11g.ASMinstancesinOracledatabase11grelease(from11.1)canbe
upgradedorpatchedusingrollingupgradefeature.ThisenablesustopatchorupgradeASMnodesina
clusteredenvironmentwithoutaffectingdatabaseavailability.Duringarollingupgradewecanmaintaina
functionalclusterwhileoneormoreofthenodesintheclusterarerunningindifferentsoftwareversions.

QCanrollingupgradebeusedtoupgradefrom10gto11gdatabase?

No,itcanbeusedonlyforOracledatabase11greleases(from11.1).

QStatetheinitializationparametersthatmusthavesamevalueforeveryinstanceinanOracleRAC
database

Someinitializationparametersarecriticalatthedatabasecreationtimeandmusthavesamevalues.Theirvalue
mustbespecifiedinSPFILEorPFILEforeveryinstance.Thelistofparametersthatmustbeidenticalonevery
instancearegivenbelow:
ACTIVE_INSTANCE_COUNT
ARCHIVE_LAG_TARGET
COMPATIBLE
CLUSTER_DATABASE
CLUSTER_DATABASE_INSTANCE
CONTROL_FILES
DB_BLOCK_SIZE
DB_DOMAIN
DB_FILES
DB_NAME
DB_RECOVERY_FILE_DEST
DB_RECOVERY_FILE_DEST_SIZE
DB_UNIQUE_NAME
INSTANCE_TYPE(RDBMSorASM)
PARALLEL_MAX_SERVERS
REMOTE_LOGIN_passWORD_FILE
UNDO_MANAGEMENT

QWhatisORA00603:ORACLEserversessionterminatedbyfatalerrororORA29702:erroroccurred
inClusterGroupServiceoperation?
RACnodenamewaslistedintheloopbackaddress...

QCantheDML_LOCKSandRESULT_CACHE_MAX_SIZEbeidenticalonallinstances?

Theseparameterscanbeidenticalonallinstancesonlyiftheseparametervaluesaresettozero.
WhattwoparametersmustbesetatthetimeofstartingupanASMinstanceinaRACenvironment?The
parametersCLUSTER_DATABASEandINSTANCE_TYPEmustbeset.

QMentionthecomponentsofOracleclusterware

OracleclusterwareismadeupofcomponentslikevotingdiskandOracleClusterRegistry(OCR).

QWhatisaCRSresource?

Oracleclusterwareisusedtomanagehighavailabilityoperationsinacluster.AnythingthatOracleClusterware
managesisknownasaCRSresource.SomeexamplesofCRSresourcesaredatabase,aninstance,aservice,a
listener,aVIPaddress,anapplicationprocessetc.

QWhatistheuseofOCR?

OracleclusterwaremanagesCRSresourcesbasedontheconfigurationinformationofCRSresourcesstoredin
OCR(OracleClusterRegistry).

QHowdoesaOracleClusterwaremanageCRSresources?

OracleclusterwaremanagesCRSresourcesbasedontheconfigurationinformationofCRSresourcesstoredin
OCR(OracleClusterRegistry).

QNamesomeOracleclusterwaretoolsandtheiruses?

OIFCFGallocatinganddeallocatingnetworkinterfaces
OCRCONFIGCommandlinetoolformanagingOracleClusterRegistry
OCRDUMPIdentifytheinterconnectbeingused
CVUClusterverificationutilitytogetstatusofCRSresources

QWhatarethemodesofdeletinginstancesfromORacleRealApplicationclusterDatabases?

WecandeleteinstancesusingsilentmodeorinteractivemodeusingDBCA(DatabaseConfigurationAssistant).

QHowdoweremoveASMfromaOracleRACenvironment?

Weneedtostopanddeletetheinstanceinthenodefirstininteractiveorsilentmode.Afterthatasmcanbe
removedusingsrvctltoolasfollows:
srvctlstopasmnnode_name
srvctlremoveasmnnode_name
WecanverifyifASMhasbeenremovedbyissuingthefollowingcommand:
srvctlconfigasmnnode_name
QHowdoweverifythataninstancehasbeenremovedfromOCRafterdeletinganinstance?

Issuethefollowingsrvctlcommand:
srvctlconfigdatabaseddatabase_name
cdCRS_HOME/bin
./crs_stat

QHowdoweverifyanexistingcurrentbackupofOCR?

WecanverifythecurrentbackupofOCRusingthefollowingcommand:ocrconfigshowbackup
WhataretheperformanceviewsinanOracleRACenvironment?
Wehavev$viewsthatareinstancespecific.InadditionwehaveGV$viewscalledasglobalviewsthathasan
INST_IDcolumnofnumericdatatype.GV$viewsobtaininformationfromindividualV$views.
Whatarethetypesofconnectionloadbalancing?
Therearetwotypesofconnectionloadbalancing:serversideloadbalancingandclientsideloadbalancing.

QWhatisthedifferencebetweenserversideandclientsideconnectionloadbalancing?

Clientsidebalancinghappensatclientsidewhereloadbalancingisdoneusinglistener.Incaseofserverside
loadbalancinglistenerusesaloadbalancingadvisorytoredirectconnectionstotheinstanceprovidingbest
service.

QWhatarethethreegreatestbenefitsthatRACprovides??

Thethreemainbenefitsareavailability,scalability,andtheabilitytouselowcostcommodityhardware.RAC
allowsanapplicationtoscalevertically,byaddingCPU,diskandmemoryresourcestoanindividualserver.But
RACalsoprovideshorizontalscalability,whichisachievedbyaddingnewnodesintothecluster.RACalso
allowsanorganizationtobringtheseresourcesonlineastheyareneeded.Thiscansaveasmallormidsize
organizationalotofmoneyintheearlystagesofaproject.

InaRACenvironment,ifanodeintheclusterfails,theapplicationcontinuestorunonthesurvivingnodes
containedinthecluster.Ifyourapplicationisconfiguredcorrectly,mostuserswon'tevenknowthatthenode
theywererunningonbecameunavailable.

QWhatarethemajorRACwaitevents?

InaRACenvironmentthebuffercacheisglobalacrossallinstancesintheclusterandhencethe
processingdiffers.Themostcommonwaiteventsrelatedtothisaregccrrequestandgcbufferbusy

GCCRrequest:thetimeittakestoretrievethedatafromtheremotecache

Reason:RACTrafficUsingSlowConnectionorInefficientqueries(poorlytunedquerieswillincreasethe
amountofdatablocks
requestedbyanOraclesession.Themoreblocksrequestedtypicallymeansthemoreoftenablockwillneedto
bereadfromaremoteinstanceviatheinterconnect.)
GCBUFFERBUSY:Itisthetimetheremoteinstancelocallyspendsaccessingtherequesteddatablock.

QWhatarethedifferentnetworkcomponentsinOracle10gRAC?

Wehavepublic,private,andVIPcomponents.Privateinterfacesisforintranodecommunication.VIPisall
aboutavailabilityofapplication.WhenanodefailsthentheVIPcomponentwillfailovertosomeothernode,this
isthereasonthatallapplicationsshouldbebasedonVIPcomponents.Thismeansthattnsentriesshould
haveVIPentryinthehostlist.

QTunethefollowingRACDATABASE(DBNAME=PROD)whichis3nodeRAC.

PROD1PROD2PROD3
CPU8CPU15CPU8
32GBRAM12GBRAM16GBRAM

Whatareyoulookingforhere?Whattuninginformationdoyouexpect?
Itisa3nodeclusterwithdifferenthardwareconfigurationrunningRAC.
Iwouldput20%ofthememoryforOracleineachnode.SothatwouldmeanthattheSGAisdifferentineachof
thenodes.
AlsosincetheCPU'saredifferentPROD2canhavemorenumberofmaxnumberofprocessesascomparedto
therestofthem.
ButasIsaidthisisjustconfiguration,thisisnottuning.Questionisnotclear.

QWriteasamplescriptforRMANfortherecoveryifalltheinstancearedown.(Firstexplainthe
procedurehowyouwillrestore)

Bringallnodesdown.
StartoneNode
Restorealldatafilesandarchivelogs.
Recover1Node.
Openthedatabase.
bringothernodesup.
Confirmthatallnodesareoperational.

Clientsareperformingsomeoperationandsuddenlyoneofthedatafileisexperiencingproblemwhat
doyoudo?Theclusterisatwonodeone.

1.Bringthedatafileofflinerecoverthedatafile.

1.HowcanyouconnecttoaspecificnodeinaRACenvironment?
2.tnsnames.oraensurethatyouhaveINSTANCE_NAMEspecifiedinit.

QHowtomoveOCRandVotingdisktonewstoragedevice?

MovingOCR
==========
Youmustbeloggedinastherootuser,becauserootownstheOCRfiles.Alsoanocrmirrormustbeinplace
beforetryingtoreplacetheOCRdevice.

MakesurethereisarecentbackupoftheOCRfilebeforemakinganychanges:

ocrconfigshowbackup

IfthereisnotarecentbackupcopyoftheOCRfile,anexportcanbetakenforthecurrentOCRfile.Usethe
followingcommandtogenerateanexportoftheonlineOCRfile:

In10.2

#ocrconfigexportsonline

In11g

#ocrconfigmanualbackup

ThenewOCRdiskmustbeownedbyroot,mustbeintheoinstallgroup,andmusthavepermissionssetto
640.Provideatleast100MBdiskspacefortheOCR.

Ononenodeasrootrun:

#ocrconfigreplaceocr
#ocrconfigreplaceocrmirror

NowrunocrchecktoverifyiftheOCRispointingtothenewfile

MovingVotingDisk
==================

Note:crsctlvotediskcommandsmustberunasroot

ShutdowntheOracleClusterware(crsctlstopcrsasroot)onallnodesbeforemakinganymodificationtothe
votingdisk.Determinethecurrentvotingdisklocationusing:

crsctlquerycssvotedisk

Takeabackupofallvotingdisk:

ddif=voting_disk_nameof=backup_file_name
TomoveaVotingDisk,providethefullpathincludingfilename:

crsctldeletecssvotediskforce
crsctladdcssvotediskforce

Aftermodifyingthevotingdisk,starttheOracleClusterwarestackonallnodes

#crsctlstartcrs

Verifythevotingdisklocationusing

crsctlquerycssvotedisk

QWhatisrunfixup.shscriptinOracleClusterware11grelease2installation

WithOracleClusterware11grelease2,OracleUniversalInstaller(OUI)detectswhentheminimum
requirementsforaninstallationarenotmet,andcreatesshellscripts,calledfixupscripts,tofinishincomplete
systemconfigurationsteps.IfOUIdetectsanincompletetask,thenitgeneratesfixupscripts(runfixup.sh).You
canrunthefixupscriptafteryouclicktheFixandCheckAgainButton.

TheFixupscriptdoesthefollowing:

Ifnecessarysetskernelparameterstovaluesrequiredforsuccessfulinstallation,

including:

Sharedmemoryparameters.

OpenfiledescriptorandUDPsend/receiveparameters.

SetspermissionsontheOracleInventory(centralinventory)directory.

Reconfiguresprimaryandsecondarygroupmembershipsfortheinstallation

owner,ifnecessary,fortheOracleInventorydirectoryandtheoperatingsystem

privilegesgroups.

Setsshelllimitsifnecessarytorequiredvalues.

QWhenexactlyduringtheinstallationprocessareclusterwarecomponentscreated?

Afterfulfillingthepreinstallationrequirements,thebasicinstallationstepstofolloware:

1.InvoketheOracleUniversalInstaller(OUI)
2.Enterthedifferentinformationforsomecomponentslike:
nameofthecluster
publicandprivatenodenames
locationforOCRandVotingDisks
networkinterfacesusedforRACinstances
etc.

3.AftertheSummaryscreen,OUIwillstartcopyingunderthe$CRS_HOME(thisisthe$ORACLE_HOMEfor
OracleClusterware)inthelocalnodethelibrariesandexecutables.
herewewillhavethedaemonsandscriptsinit.*createdandconfiguredproperly.

OracleClusterwareisformedofseveraldaemons,eachoneofwhichhaveaspecialfunctioninsidethestack.
Daemonsareexecutedviatheinit.*scripts(init.cssd,init.crsdandinit.evmd).

notethatforCRSonlysomeclientlibrariesarerecreated,butnotalltheexecutables(asfortheRDBMS).

4.LaterthesoftwareispropagatedtotherestofthenodesintheclusterandtheoraInventoryisupdated.

5.Theinstallerwillasktoexecuteroot.shoneachnode.UntilthisstepthesoftwareforOracleClusterwareis
insidethe$CRS_HOME.

Runningroot.shwillcreateseveralcomponentsoutsidethe$CRS_HOME:

OCRandVDwillbeformated.

controlfiles(orSCLS_SRCfiles)willbecreatedwiththecorrectcontentstostartOracleClusterware.

ThesefilesareusedtocontrolsomeaspectsofOracleClusterwarelike:
enable/disableprocessesfromtheCSSDfamily(Eg.oprocd,oslsvmon)
stopthedaemons(ocssd.bin,crsd.bin,etc).
preventOracleClusterwarefrombeingstartedwhenthemachineboots.
etc.

/etc/inittabwillbeupdatedandtheinitprocessisnotified.

InordertostarttheOracleClusterwaredaemons,theinit.*scriptsfirstneedtoberun.Thesescriptsare
executedbythedaemoninit.Toaccomplishthissomeentriesmustbecreatedinthefile/etc/inittab.

thedifferentprocessesinit.*(init.cssd,init.crsd,etc)willstartthedaemons(ocssd.bin,crsd.bin,etc).Whenall
thedaemonsarerunningthenwecansaythattheinstallationwassuccessful
On10.2andlater,runningroot.shonthelastnodeintheclusteralsowillcreatethenodeapps(VIP,GSDand
ONS).On10.1,VIPCAisexecutedaspartoftheRACinstallation.

6.Afterrunningroot.shoneachnode,weneedtocontinuewiththeOUIsession.Afterpressingthe'OK'button
OUIwillincludetheinformationforthepublicandcluster_interconnectinterfaces.AlsoCVU(ClusterVerification
Utility)willbeexecuted.

QWhatareOracleClusterwareprocessesfor10gonUnixandLinux

ClusterSynchronizationServices(ocssd)Managesclusternodemembershipandrunsastheoracleuser
failureofthisprocessresultsinclusterrestart.

ClusterReadyServices(crsd)Thecrsprocessmanagesclusterresources(whichcouldbeadatabase,an
instance,aservice,aListener,avirtualIP(VIP)address,anapplicationprocess,andsoon)basedonthe
resource'sconfigurationinformationthatisstoredintheOCR.Thisincludesstart,stop,monitorandfailover
operations.Thisprocessrunsastherootuser

Eventmanagerdaemon(evmd)Abackgroundprocessthatpublisheseventsthatcrscreates.

ProcessMonitorDaemon(OPROCD)ThisprocessmonitortheclusterandprovideI/Ofencing.OPROCD
performsitscheck,stopsrunning,andifthewakeupisbeyondtheexpectedtime,thenOPROCDresetsthe
processorandrebootsthenode.AnOPROCDfailureresultsinOracleClusterwarerestartingthenode.
OPROCDusesthehangchecktimeronLinuxplatforms.

RACG(racgmain,racgimon)ExtendsclusterwaretosupportOraclespecificrequirementsandcomplex
resources.RunsservercalloutscriptswhenFANeventsoccur.

QWhatareOracledatabasebackgroundprocessesspecifictoRAC

LMSGlobalCacheServiceProcess

LMDGlobalEnqueueServiceDaemon

LMONGlobalEnqueueServiceMonitor

LCK0InstanceEnqueueProcess

ToensurethateachOracleRACdatabaseinstanceobtainstheblockthatitneedstosatisfyaqueryor
transaction,OracleRACinstancesusetwoprocesses,theGlobalCacheService(GCS)andtheGlobal
EnqueueService(GES).TheGCSandGESmaintainrecordsofthestatusesofeachdatafileandeachcached
blockusingaGlobalResourceDirectory(GRD).TheGRDcontentsaredistributedacrossalloftheactive
instances.

QWhatareOracleClusterwareComponents

VotingDiskOracleRACusesthevotingdisktomanageclustermembershipbywayofahealthcheckand
arbitratesclusterownershipamongtheinstancesincaseofnetworkfailures.Thevotingdiskmustresideon
shareddisk.

OracleClusterRegistry(OCR)Maintainsclusterconfigurationinformationaswellasconfiguration
informationaboutanyclusterdatabasewithinthecluster.TheOCRmustresideonshareddiskthatis
accessiblebyallofthenodesinyourcluster

QHowdoyoutroubleshootnodereboot

Pleasecheckmetalink...

Note265769.1TroubleshootingCRSReboots
Note.559365.1UsingDiagwaitasadiagnostictogetmoreinformationfordiagnosingOracleClusterwareNode
evictions.

QHowdoyoubackuptheOCR

ThereisanautomaticbackupmechanismforOCR.Thedefaultlocationis:
$ORA_CRS_HOME\cdata\"clustername"\

Todisplaybackups:
#ocrconfigshowbackup
Torestoreabackup:
#ocrconfigrestore

WithOracleRAC10gRelease2orlater,youcanalsousetheexportcommand:
#ocrconfigexportsonline,anduseimportoptiontorestorethecontentsback.
WithOracleRAC11gRelease1,youcandoamanaualbackupoftheOCRwiththecommand:
#ocrconfigmanualbackup

QHowdoyoubackupvotingdisk

#ddif=voting_disk_nameof=backup_file_name

QHowdoIidentifythevotingdisklocation

#crsctlquerycssvotedisk

QHowdoIidentifytheOCRfilelocation
check/var/opt/oracle/ocr.locor/etc/ocr.loc(dependsuponplatform)
or
#ocrcheck

QIssshrequiredfornormalOracleRACoperation?

"ssh"arenotrequiredfornormalOracleRACoperation.However"ssh"shouldbeenabledforOracleRACand
patchsetinstallation.

QWhatisSCAN?

SingleClientAccessName(SCAN)issanewOracleRealApplicationClusters(RAC)11gRelease2feature
thatprovidesasinglenameforclientstoaccessanOracleDatabaserunninginacluster.Thebenefitisclients
usingSCANdonotneedtochangeifyouaddorremovenodesinthecluster.

QWhatisthepurposeofPrivateInterconnect?

Clusterwareusestheprivateinterconnectforclustersynchronization(networkheartbeat)anddaemon
communicationbetweenthetheclusterednodes.ThiscommunicationisbasedontheTCPprotocol.
RACusestheinterconnectforcachefusion(UDP)andinterprocesscommunication(TCP).CacheFusionis
theremotememorymappingofOraclebuffers,sharedbetweenthecachesofparticipatingnodesinthe
cluster.

QWhydowehaveaVirtualIP(VIP)inOracleRAC?

WithoutusingVIPsorFAN,clientsconnectedtoanodethatdiedwilloftenwaitforaTCPtimeoutperiod(which
canbeupto10min)beforegettinganerror.Asaresult,youdon'treallyhaveagoodHAsolutionwithoutusing
VIPs.
Whenanodefails,theVIPassociatedwithitisautomaticallyfailedovertosomeothernodeandnewnodere
arpstheworldindicatinganewMACaddressfortheIP.SubsequentpacketssenttotheVIPgotothenew
node,whichwillsenderrorRSTpacketsbacktotheclients.Thisresultsintheclientsgettingerrorsimmediately

QWhatdoyoudoifyouseeGCCRBLOCKLOSTintop5TimedEventsinAWRReport?

Thisismostlikelyduetoafaultininterconnectnetwork.
Checknetstats
ifyousee"fragmentsdropped"or"packetreassembliesfailed",Workwithyoursystemadministratorfindthe
faultwithnetwork.

QHowmanynodesaresupportedinaRACDatabase?

10gRelease2,support100nodesinaclusterusingOracleClusterware,and100instancesinaRACdatabase.
QSrvctlcannotstartinstance,IgetthefollowingerrorPRKP1001CRS0215,howeversqlpluscan
startitonbothnodes?Howdoyouidentifytheproblem?

SettheenvironmentalvariableSRVM_TRACEtotrue..Andstarttheinstancewithsrvctl.Nowyouwillget
detailederrorstack.

QwhatisthepurposeoftheONSdaemon?

TheOracleNotificationService(ONS)daemonisandaemonstartedbytheCRSclusterwareaspartofthe
nodeapps.Thereisoneonsdaemonstartedperclusterednode.
TheOracleNotificationServicedaemonreceiveasubsetofpublishedclusterwareeventsviathelocalevmd
andracgimonclusterwaredaemonsandforwardthoseeventstoapplicationsubscribersandtothelocal
listeners.

Thisinordertofacilitate:

a.theFANorFastApplicationNotificationfeatureorallowingapplicationstorespondtodatabasestate
changes.
b.the10gR2LoadBalancingAdvisory,thefeaturethatpermitloadbalancingaccrossdifferentracnodes
dependentoftheloadonthedifferentnodes.TherdbmsMMONiscreatinganadvisoryfordistributionofwork
every30secondsandforwarditviaracgimonandONStolistenersandapplications.

QHowdousersconnecttodatabaseinanOracleRACenvironment?

UserscanaccessaRACdatabaseusingaclient/serverconfigurationorthroughoneormoremiddletiers,with
orwithoutconnectionpooling.Userscanuseoracleservicesfeaturetoconnecttodatabase.

QWhatistheuseofaserviceinOracleRACenvironment?

ApplicationsshouldusetheservicesfeaturetoconnecttotheOracledatabase.Servicesenableustodefine
rulesandcharacteristicstocontrolhowusersandapplicationsconnecttodatabaseinstances.

QWhatarethecharacteristicscontrolledbyOracleservicesfeature?

Thecharacteristicsincludeauniquename,workloadbalancingandfailoveroptions,andhighavailability
characteristics.

QWhatisavotingdisk?

Avotingdiskisafilethatmanagesinformationaboutnodemembership.

QWhataretheadministrativetasksinvolvedwithvotingdisk?

Followingadministrativetasksareperformedwiththevotingdisk:

1)Backingupvotingdisks
2)RecoveringVotingdisks

3)Addingvotingdisks

4)Deletingvotingdisks

5)Movingvotingdisks

QHowdowebackupvotingdisks?

1)Oraclerecommendsthatyoubackupyourvotingdiskaftertheinitialclustercreationandafterwecomplete
anynodeadditionordeletionprocedures.

2)First,asrootuser,stopOracleClusterware(withthecrsctlstopcrscommand)onallnodes.Then,determine
thecurrentvotingdiskbyissuingthefollowingcommand:

crsctlqueryvotediskcss

3)Then,issuetheddorocopycommandtobackupavotingdisk,asappropriate.

Givethesyntaxofbackingupvotingdisks:

OnLinuxorUNIXsystems:

ddif=voting_disk_nameof=backup_file_name

where,

voting_disk_nameisthenameoftheactivevotingdisk

backup_file_nameisthenameofthefiletowhichwewanttobackupthevotingdiskcontents

OnWindowssystems,usetheocopycommand:

ocopyvoting_disk_namebackup_file_name

QWhatistheOracleRecommendationforbackingupvotingdisk?

Oraclerecommendsustousetheddcommandtobackupthevotingdiskwithaminimumblocksizeof4KB.

QHowdoyourestoreavotingdisk?

Torestorethebackupofyourvotingdisk,issuetheddorocopycommandforLinuxandUNIXsystemsor
ocopyforWindowssystemsrespectively.

OnLinuxorUNIXsystems:

ddif=backup_file_nameof=voting_disk_name
OnWindowssystems,usetheocopycommand:

ocopybackup_file_namevoting_disk_name

where,

backup_file_nameisthenameofthevotingdiskbackupfile

voting_disk_nameisthenameoftheactivevotingdisk

QHowcanweaddandremovemultiplevotingdisks?

Ifwehavemultiplevotingdisks,thenwecanremovethevotingdisksandaddthembackintoourenvironment
usingthefollowingcommands,wherepathisthecompletepathofthelocationwherethevotingdiskresides:

crsctldeletecssvotediskpathcrsctladdcssvotediskpath
QHowdowestopOracleClusterware?Whendowestopit?

Beforemakinganymodificationtothevotingdisk,asrootuser,stopOracleClusterwareusingthecrsctlstop
crscommandonallnodes.

QHowdoweaddvotingdisk?

Toaddavotingdisk,issuethefollowingcommandastherootuser,replacingthepathvariablewiththefully
qualifiedpathnameforthevotingdiskwewanttoadd:

crsctladdcssvotediskpathforce

QHowdowemovevotingdisks?

Tomoveavotingdisk,issuethefollowingcommandsastherootuser,replacingthepathvariablewiththefully
qualifiedpathnameforthevotingdiskwewanttomove:

crsctldeletecssvotediskpathforce

crsctladdcssvotediskpathforce

QHowdoweremovevotingdisks?

Toremoveavotingdisk,issuethefollowingcommandastherootuser,replacingthepathvariablewiththefully
qualifiedpathnameforthevotingdiskwewanttoremove:

crsctldeletecssvotediskpathforce

QWhatshouldwedoaftermodifyingvotingdisks?

Aftermodifyingthevotingdisk,restartOracleClusterwareusingthecrsctlstartcrscommandonallnodes,and
verifythevotingdisklocationusingthefollowingcommand:
crsctlquerycssvotedisk

QWhencanweuseforceoption?

Ifourclusterisdown,thenwecanincludetheforceoptiontomodifythevotingdiskconfiguration,without
interactingwithactiveOracleClusterwaredaemons.However,usingtheforceoptionwhileanyclusternodeis
activemaycorruptourconfiguration.

Like