1WHATAREFRACTUREDBLOCKS?

AnOracleblockcontainsspecialheaderinformationandtailcheck,whichallows
Oracletodetermineadiscrepancy.Fracturedblockreferstoaconditioninwhichthehead
andthetailofanOracleblockdonotmatch.AfracturedblockcanoccurifOracleis
updatingablockatthesametimethatthecopyutility(OSorstoragebased)isreadingthe
sameblock.Thecopyutilitycouldcaptureanimageofapartiallyupdatedblock.A
fracturedblocksituation,althoughextremelyrareandtimingdependent,canhappenifthe
copyutilityoperatesatanI/OblocksizethatissmallerorisnotamultipleoftheOracle
blocksize.AnOracleblockisthesmallestunitofI/OperformedbytheOracleinstance.The
Oracleblocksizeissetduringdatabasecreationandcanrangefrom2kBto32kB.AnOS
blocksizevariesbyplatform,withmostUNIXandLinuxsystemsadoptinga4kBor8kB
blocksize.Windowshasadefaultsizeof2kBor4kB.Asanexample,assumethatthe
Oracleblocksizeis8kBandtheOSblocksizeis4kB.Oracleiswritingtothedatafilesat
8kBblocksize,whiletheOSbackupcopyisreadingat4kBincrements.Theorderinwhich
the4kBblockisreadisdeterminedbytheOSandiscompletelyindependentofOracle.
TheOScopycouldreadthefirstorsecond4kBhalfoftheOracleblockjustbeforethe
databasewriter(DBWR)updatestheblock.Theblockthengetsupdatedwiththenew
image.Subsequently,thecopyreadstheremainingfirstorsecond4kBhalfoftheblock.
Thecopyimageofthatparticularblocknowcontainstwohalvesthatarefromdifferent
pointsintime,perhapswithdifferentsystemchangenumbers(SCNs).Theresultisa
fractured,orsplit,block.Afracturedblockoccurrenceisexceptionallyuncommonbecauseit
isnotonlytimingdependent;italsorequiresthatcertaingoverningconditionsbepresent.
Infact,thegoverningconditionsmustfirstbepresent.Ifthetimingisalsoright,onlythen
canfracturedblocksoccur.Fracturedblockscanbepreventedduringbackupsbyusing
RecoveryManager(RMAN)andbackupmodeoptions.RMANnevercopiesafractured
block.Itknowsthecorrectordertoreadtheblocksandrereadstheblockuntilitgetsa
consistentimage.Backupmodehascontingentblockimageloggingtorecoverfrom
fracturedblocks.Incontrast,snapshotorbackupcopyofdatabasecreatedwithoutbackup
modedoesnothavetheprovisiontoresolvefracturedblocks.Fracturedblockscanoccur
whenthebackupprocessisnonRMANandthedatabaseisnotplacedinbackupmode.
1.1 CAUSESOFFRACTUREDBLOCKS

Fracturedblockshavenumerouscauses.Theycanresultfrombugsorfailuresin
hardware,software,orfirmware;however,theprincipalcauseofafracturedblockistheOS
orcopyutilityandOracleoperatingatdifferentI/Osizes,whichcouldresultinapartially
copiedOracleblock.Forexample,theOScouldsplitasingleOracleblockwriteinto
multiplesmallerphysicalwrites,oritcouldperformareadofanOracleblockinmultiple
OSreads.SuchoccurrencescanleadtoanOracleblockbeingpartiallywrittenorread
whilethecopyorsnapshotiscreated.SeeFigure1foranillustrationofhowafractured
blockcanoccurwhenOraclewritesaresplit.

Figure1)FracturedblockscanoccurwhenOraclewritesaresplit.
Fracturedblockscanoccurduetooneormoreofthefollowingconditions:
OS,filesystem,orvolumemanagersplitsasingleOracleI/Owriteoperationinto
multipleI/Osatalowerlevel
CopyutilityoftheOS/filesystem/LogicalVolumeManager(LVM)orstoragesystem
performsreadoperationsatablocksizesmallerthanthatoftheOracleblocksize,resulting
inmultiplereadsofasingleOracleblock
PossiblefailuresorbugsonOS,LVM,hardware,hostbusadapters(HBAs),orsystem,
whichcausesanOracleI/Owritetobeinterruptedbeforecompletion

SeeFigure2foranillustrationofafracturedblockresultingfrombackuporcopyutility
readingatsmallerblocksizethantheblocksizeatwhichOracleisreading

Figure2)Fracturedblockscanoccurwhenbackup/copyutilityreadsatsmaller
blocksizethanthatofOracle.
2AVOIDINGFRACTUREDBLOCKS
OnewaytoavoidfracturedblocksistomaintainatomicityofanOracleblock
read/write.ThismeansmakingsureasingleOracleblockread/writegetsprocessedasa
singleatomicI/OoperationatthelowerlevelsandisneversplitintomultipleI/Os.The
maximumI/OsizehandledbytheOS,thefilesystem,orthevolumemanagermustalways
tobeamultipleoftheOracleblocksize,andtheOracleblocksizemustalwaysbethe
minimumunitofI/OperformedondatafilesbyOracle.Thus,theOS,filesystem,or

volumemanager,whicharecapableofissuingI/OsizesinmultiplesofOracleblocksize,
wouldnotbreakasingleOracleblockI/Ointomultipleoperations.
2.1DETERMININGTHEMAXIMUMI/OSIZEBEYONDTHEORACLELAYER
ThestructureofanI/Ostackisusuallydefinedbymultiplelayersandcomponents.
ANetworkFileSystem(NFS)stacktypicallycomprisestheNFSclient,TCPIPprotocol,
andthenetworkdevicedrivers.Astoragenetworkarea(SAN)environmentwithblock
basedprotocolismorecomplexandmostlikelydependsonadditionallayers,whereasafile
systemimplementationmayshareasubsetofcomponentsseeninaSANconfiguration.See
Figure3

Figure3)TypicalI/OstackconfigurationofSANandNASenvironments.
BecausetheI/Ostackisinterconnectedbydifferentlayers,themaximumI/Osizeor
I/Orequestsizecanbeindependentlydeterminedorrestrictedbysomeofthelayers.This
behaviorismoreevidentinaSANconfiguration,where,forinstance,theOS,thevolume
manager,andthedevicedrivershavetheirownattributesthatcontrolthemaximumI/O
sizeorI/Orequestsize.WhenthefilesystemisusedwithbufferedI/OandnotdirectI/O,
thefilesystemblocksizeorbuffersize,orOSpagesizemaydictatethesizeoftheI/O,
whichtheOSusestowriteorreadfiles.I/OsubsystemscanvarywidelybyOS,platforms,

kernelreleases,protocols,and/orfilesystemtypes.Thefollowingsectionscontaindetailson
thekeycharacteristicsofblockdevice,filesystem,andNFSconfigurations.
2.2RAWDEVICESANDBLOCKDEVICES
Rawdevices,alsoknownascharacterdevices,wereoncecommonlyimplementedin
OracleenvironmentstoimproveI/Operformance.Rawdevicesaretypicallyaddressedas
/dev/rdskor/dev/raw.Rawdeviceisaspecialkindofblockdevicethatallowsapplications
suchasOracletoaccesstheunderlyingstorageordisksdirectlywithoutpassingthrough
theoperatingsystemscacheandbuffers.Thus,thedefaultmodeforOracletoaccessraw
devicesisdirectI/Oandasynchronous.Duetocomplexmanagementandadministration
suchasoffsetandalignment,useofrawdeviceisbecominglesspreferred.Inaddition,raw
devicesupportisscheduledtobedeprecatedinsomeoperatingsystemssuchasLinux.
Blockdevices,bydefinition,transferdatainfixedsizegroupsofbytesblock,
sector,orcluster.BlockdevicesgenerallyoperatewithbufferedI/O,wheretheoperating
systemallocatesdatabufferstoholdblocksforinputandoutput.Readingandwritingto
blockdevicesarebufferedI/Osbydefault.However,sincetheOracle10gRelease2,Oracle
takesadvantageoftheavailabilityoftheO_DIRECTfeaturebyautomaticallyopeningall
blockdevicesusingtheO_DIRECTflag,allowingallI/OstobypasstheOSbuffers.
FACTORSDETERMININGMAXIMUMI/OSIZEANDI/OREQUESTSIZEFOR
RAWDEVICESANDBLOCKDEVICES
WhenoperatingwithrawdevicesandblockdevicesinaSANenvironment,differentlayers
intheI/OstackcanimposealimitonthemaximumI/Osize.Thefollowinglistcontains
descriptionsofthoselayers:
Operatingsystems:SolarisandHPUXusemaxphyskernelvariabletosetthelargest
physicalI/Orequestsize.Themaxphyskernelvariablehasadefaultof128kBonSolaris
and1MBonHPUX.Linuxofferstheaiomaxsizeparameterinthe2.4kernelreleases,
whichspecifiesthemaximumblocksizethatonesingleAIOwrite/readcanperform.
StartingwithRedHatEnterpriseLinux4and5(2.6kernels),thisoptionisnolonger
availableandinsteadisautomatic,basedonthephysicalcapabilitiesofthediskdevice
driver.
Volumemanagers:Somevolumemanagers,suchasVeritasVxVM,havetunablesthat
limitthesizeofI/Orequeststothevolumemanager.TheVxVMtunablevxio:vol_maxio
definesthelargestI/Orequestsizethatcanbemadetothevolumemanagerwithoutthe
systembreakinguptheI/Orequestsintothespecifiedsize.Thedefaultvalueof
vxio:vol_maxioisusually256kB.SolarisVolumeManager/DiskSuitereliesonthe
md_maxphyskernelparametertolimitthemaximumI/Osizeatitslayer.Thedefault
valueisthesameasthemaxphyskernelparameter.

Devicedrivers:MostdevicedrivershavealimitonthemaximumI/Osizethatcanbe
transferred.Forexample,Solarishasdevicedriverssuchassd(SCSIdevicedriver)andssd
(FibreChanneldevicedriver),whichusuallysupportanupperlimitof1MBI/Osize.In
addition,FibreChannelHBAscansupportuptoacertainmaximumI/Orequestsize.The
maximumsize(8to16MB)isbasedonthedirectmemoryaccess(DMA)sizeallowedwithin
thedesignedFibreChannelchipset,whichisusuallymuchhigherthanthelimitsatthe
OS,volumemanager,anddevicedriverlayers.

Note:RawdevicesandblockdeviceswithdirectI/Ousuallyarenotexposedtothe
potentialoffracturedblocks.I/Osperformedonrawdevicesandunbufferedblockdevices
areinherentlydirect.Inaddition,themaximumI/Osizepermittedbyeachlayer(OS,
volumemanagers,devicedrivers)isalmostalwaysmuchlargerthanthelargestOracle
blocksizeof32kB.Thus,singleOracleblockI/Osarenotsubjecttobeingsplitintomultiple
I/Osbythevariouslayers.
2.3FILESYSTEMS
FilesystemisapopularoptionforOracledatabasedeployments.Comparedtoraw
devicesandblockdevices(withouttheuseofOracleAutomatedStorageManagement
[ASM]),afilesystemofferssimplermanagementandadministration,aswellasbackupand
recovery.Mostfilesystemsarebasedonblockdevices,whichareusuallyconstructedon
hardwareRAIDvolumes/LUNsorLVMvolumes.Bydefault,filesystemsemploybuffered
I/Os,wheretheoperatingsystempagecacheisusedtotransferdatabetweenthe
applicationandthestoragedevice.Infilesystems,thereisthelogicalblocksize,which
referstotheblocksizeofthefilesystem,andthephysicalblocksize,whichistheactual
operatingsystemsphysicalblocksize.Thephysicalblocksizeisthesmallestblocksizethat
thediskcontrollercanreadorwrite512bytesonmostoperatingsystems.Thelogical
blocksizeisthesizeoftheblocksthattheUNIXorLinuxkernelusestoreadorwritefiles.
Thelogicalblocksizemayvarybyvendorsimplementationsandplatforms,thoughitis
typicallyamultipleofthephysicalblocksize,withthemaximumsizebeingthekernelpage
size.Thefollowinglistdescribesthevariousfilesystems:
SolarisUFSfilesystem.Supportsblocksizesof4or8kB.Thedefaultlogicalblocksize
isusuallysettothepagesizeofthesystem,4kBonx86andx86_64architectureand8kB
onSparc.
Veritasfilesystem(VxFS).Supportsblocksize/extentof1024,2048,4096,and8192
bytes.
HPUXJFS.IsbasedonVxFS;supportsblocksizeof1024,2048,4096,and8192bytes
withthedefaultbeing1024bytes.
AIXJFS/JFS2.Supportsblocksizesof512,1024,2048,and4096bytes.
FACTORSAFFECTINGMAXIMUMI/OSIZEINFILESYSTEMS

WhenusingbufferedI/O,whichisthedefaultmodeforfilesystems,somefilesystemsmay
breakupI/Osintoacertainsize.Forexample,theSolarisfilesystembreakssynchronous
writesinto8kBbydefault.JFSonHPUXhasabuffersize,max_buf_data_size,of8kB(or
64kB),whichdeterminestheI/Osize.VxFSwiththeVeritasVolumeManager(VxVM),by
default,breaksupI/Orequeststhatarelargerthan256kB.Ingeneral,thefilesystem
buffersizedictatesthemaximumI/OsizethatcanbeperformedwhenusingbufferedI/O
mode.ForOracledatafiles,thenormalrecommendationistoenabledirectI/Owithfile
systems.ThereareseveralreasonsthatdirectI/Oispreferred:
FacilitatebetteroverallI/OperformanceforOracle
AvoidbreakingupI/Osinsomefilesystemimplementations(henceallowinglargerI/Os
tocompleteasasingleoperation)
EliminatedoublecachingofOracledatabyavoidingtheuseoftheoperatingsystem
buffercacheandmemoryforcachingdata
ReduceCPUoverheadasiteliminatestheneedtocopydatatoandfromthefilesystem
buffercache
ReducelockingcontentionassociatedwithmanagingOSmemory
WithdirectI/O,I/Osizeallowedbythefilesystemsisnolongerrestrictedbythebuffer
size.ByenablingdirectI/O,SolarisUNIXFileSystem(UFS)doesnotbreakwritesinto
8kBsize;insteaditallowslargewritestocompleteasasingleI/O.JFShasthemaximum
directI/Osizeof256kBversusthemaximumbufferedI/Osizeof64kB.VxFShasthe
maximumsizeofadirectI/Orequestdefinedbymax_direct_iosztunableparameter.Ifa
largerI/Orequestcomesin,itgetsbrokenupintomax_direct_ioszchunks.Mostofthe
operatingsystemsandfilesystemsnowofferdirectI/Oimplementation.Thewayinwhich
directI/Oisactivatedcanvarybyoperatingsystemsandfilesystems.DirectI/Ocan
sometimesbeenabledforanentirefilesystemthroughthefilesystemmountoptionor
directI/OisinvokedatthefilehandlerlevelusingdirectI/Ooptionduringfileaccess
systemcalls.SomeexamplesofenablingdirectI/OforOracleundervariousoperating
systemsandfilesystems:
Solaris:UFSsupportsdirectI/Othroughtheforcedirectiomountoption.Thismount
optionisappropriateforenablingdirectI/OifyouareusingOracle8i.Beginningwith
Oracle9i,Oraclecanuseapplicationprogramminginterface(API)callstoenabledirectI/O
whenopeningfiles.SettingtheOracleinitparameter,filesystemio_options=setall,isthe
preferredwayforenablingdirectI/Oin9iorlater.
HPUX:DirectI/OonJFSisenabledbysettingtheconvosync=directmountoption.

AIX:StartingwithAIX5.1,directI/Oisenabledusingthediooptiononthemount
command.ThisoptionworksforbothJFSandJFS2filesystems.DirectI/Ocanalsobe
enabledatthefilelevelinOracleDatabase10gandlater.Thisisachievedbysettingthe
OracleFILESYSTEMIO_OPTIONSparametertosetallordirectio.
Linux:DirectI/OunderLinuxisavailableatthefilelevel.Oraclesinitparameter,
filesystemio_options,shouldbesettosetall,whichimplementsbothdirectI/Oand
asynchronousI/O.
VeritasVxFS:Mountoptionconvosync=directisusedtoenabledirectI/O.Thisoptionis
applicablewhenusingVxFSonSolaris,HPUX,orAIX.VeritasalsooffersVeritasQuick
I/O,whichovercomesthelimitationofsinglewriterlockimplementationseenbyPOSIX
complaintdirectI/O.VeritasODM(OracleDiskManager)isanotherproductwithdirect
I/Ofeature.
Note:Ingeneral,tominimizetheriskoffracturedblocksoccurringwithfilesystems,avoid
usingbufferedI/OandusedirectI/Oinstead.Inaddition,configurethefilesystemblock
sizeinmultiplesoftheOracleblocksizeorthesameastheOracleblocksizewhenpossible.
OLTPdatabasesgenerallyuse8kBOracleblocksizeandmostfilesystemssupport8kB
blocksize.

2.4NETWORKFILESYSTEM
InNFS,thersizeandwsizemountoptionsspecifythesizeofthedatachunksthatare
exchangedbetweentheclientandtheserver.Ifnorsizeandwsizeoptionsarespecified,the
defaultvariesbetweenversionsofNFS.NFSv2usuallyhasadefaultof4kBwiththe
maximumsizeof8kBforread/writeoperations.ForNFSv3overTCP,mostplatformshave
adefaultof32kBreadorwriteoperation.NetAppplatformssupport64kBforwsizeand
rsize.WhenusingOracle,mountingafilesystemwithawsizevaluethatissmallerthan
theOracleblocksizecancauseOracledatablockwritestobesplitintomultipledata
chunks,whichcanresultinfracturedblockwrites.Settingthevalueofthersizemount
optionsmallerthanthedatabaseblocksizecanalsoresultinreadingablockimagethatis
partiallyupdatedandis,therefore,fractured.Toavoidthepotentialforfracturedblocks,
OraclerequiresthatNFSfilesystemsbemountedwithrsizeandwsizethataregreater
thanorequaltothedatabaseblocksizesupportedbyOracle.Oraclerecommendsasetting
of32768foreitherrsizeorwsize.(ThesourceforthesesettingsisMetalinkNote359515.1
MountOptionsforOraclefileswhenusedwithNASdevices.)AsofOracle10.2,theI/O
interfacelayerinOraclehasbuiltincheckstoverifythatthemountoptionsforNFS
mountedfilesystemsaresetcorrectly.TheexactchecksonNFSfilesystemsusedbyOracle
mayvaryamongplatforms,butthebasicchecksinclude:
Themounttable(forexample,/etc/mnttab)canbereadtocheckthemountoptions.
TheNFSfilesystemismountedwiththehardoption.

Themountoptionsincludersize>=32768andwsize>=32768.
ForRACenvironments,whereNFSdisksaresupported,thenoacmountoptionis
specified.
Note:ToavoidfracturedblockswithNFS,usedirectI/Obyspecifyingmountoptions
forcedirectioonSolarisandHPUX,orcioonAIX.SettheOracleinitparameter,
filesystemio_options,tosetallordirectio.Inaddition,setrsizeandwsizeinmultiplesof
Oracleblocksize,startingwithaminimumvalueof32768.
2.5DIRECTNFS(BYORACLE)
DirectNFSistheOracleNFSclient,availableinOracle11g.Itisdesignedtointegratethe
NFSclientfunctionalitywithintheOracleRelationalDatabaseManagementSystem
(RDBMS)servercode.DirectNFSdeliversseveralcorebenefitsoverthestandardkernel
NFSclientsimplicity,scalability,highavailability,loadbalancing,andI/Operformance.
DirectNFSclientusestheNFSODMlibrarytoperformI/Os,whichavoidsthekernelNFS
stackandeliminateswriteorderinglocks.Bydesign,DirectNFSiscapableofexecuting
concurrentdirectI/OandasynchronousI/O.DirectNFSisOraclesowndesignand
architecture.UnlikekernelNFS,thereisnolongeraconcernformakingsurethatthe
appropriateI/Osizeisset.DirectNFShasknowledgeoftheOracleblocksizeusedbythe
database;thus,itguaranteesthatI/OsareperformedinmultiplesoftheOracleblocksize.
3NETAPPSTORAGEANDI/OATOMICITY
Databasetechnologiesareusuallydesignedtodeliverthestrictestlevelofdataintegrity
andconsistency.Inparticular,transactionaldatabasestendtoenforcemaximum
considerationontransactionreliability.Tomakesurethattransactionsaremanagedand
processedreliably,databasedesignaimstoadheretoasetoffundamentalpropertieswell
referencedinthedatabaseindustry,namelyatomicity,consistency,isolation,and
durability(ACID).

Atomicity:Eitherallofthetasksinatransactionareperformedornoneiscommitted.
Thispropertydemandsanallornothingapproach.Thetransactionmustbecompleted.If
partofthetransactionfails,theentiretransactionmustberolledback.
Consistency:Thedatabasemustremaininaconsistentstatebeforethestartofthe
transactionandafterthetransaction.Everytransactionmustpreservethedatabases
consistencyrules.
Isolation:Notransactionhasknowledgeofotheractivetransactionsandnotransaction
canaccessanothertransactionsintermediatedata.
Durability:Oncecompleted,atransactionanditsresultmustpersist.Completed
transactionscannotbeabortedatalaterstage.

Inthecontextoffracturedblocks,atomicityreferstothepremisethatadatabaseblock
writemustbeeithercompletelypresentondiskorcompletelyabsentfromdisk.Partially
writtenblocksarenottoleratedandusuallyrequirerecovery.Oraclehastheabilityto
detectfracturedblocksbywayofchecksummechanismorheaderandtailcheck.Assuming
thattheOSandthehostlayercanpreserveatomicityofanOracleblockwrite,that
atomicityismaintainedandguaranteedbyNetAppstoragesystemsasthedataispassedto
thestoragesystemsandiscommittedtodisk.NetAppstoragesystemsinternallyoperate
on64kBchunks.Regardlessofprotocolsandtransportmethods(FibreChannel,NFS,or
iSCSI),allreadandwriteoperationsupto64kBareprocessedatomicallybyWAFL.I/O
operationslargerthan64kBarebrokenupatthe64kBboundary.BecauseanOracleblock
sizemustbeapoweroftwoandrangesupto32kB,anyOracleblockreadandwriteis
atomic.Evenareadorwritefailureisguaranteedtobeatomicto64kB.WhileNetApp
storagesystemprocessesdatain64kBchunks,theunderlyingWAFLfilesystemis
designedtomanagedataat4kBblocksize;however,the4kBblocksizedoesnotrepresent
theatomicboundary.Atomicityisstillguaranteedupto64kB.Anyhostwritesupto64kB
eithersucceedsorfails;dataassociatedwiththewritesareeitherfullycommittedondisk
orcompletelyabsent.ThekeypointintherelationshipbetweenDataONTAPandWAFL
astheyrelatetothepotentialoffracturedblocksonOracleisthatNetAppstoragesystem
guaranteesthatI/Ooperationsupto64kBareatomic.AllOracleblockreadsorwritesare
managedcorrectlybyNetAppstoragesystems.Fracturedblockoccurrenceisneverthe
resultofNetAppstoragesystem.
WhenconsideringthepotentialoffracturedblocksonOracle,itisessentialtorecognize
thatNetAppstoragesystemguaranteesI/Ooperationsupto64kBtobeatomic.NetApp
storagesystemscorrectlymanageallOracleblockreadsorwrites;fracturedblock
occurrenceisnevertheresultofNetAppstoragesystem.Fordetailedinformationonthe
internaloperationofDataONTAPandWAFL,seetheindependenttechnicalreport,On
theTechnicalSuitabilityofHostingDatabasesonNetworkApplianceStorageSystemsby
SteveDanielandJeffKimmel(http://media.netapp.com/documents/wp_3443.pdf).
4HARDWAREASSISTEDRESILIENTDATAANDNETAPPSNAPVALIDATOR
Frombothoperationalandbackupstandpoints,generalblockcorruptionandfractured
blocksareachallengeinanOracleenvironment.Severalmechanismsandsolutionsare
madeavailablebyOracletohelpdetectandresolveblockcorruption.DBBlockChecksum
andFORCELOGGINGaredirectfeaturesthatcanbeenabledtopreventblockcorruption
inanactiveinstance.RMANandhotbackupmodearesolutionstominimizingpotential
blockcorruptionrelatedtobackupandrecovery.DBBlockChecksumistheprimary
methodfordetectingcorruptionsthatoccurbeyondtheOraclelayer.ItenablesOracleto
detectblockcorruptionthatmaybecausedbysoftwareorhardwarecomponentsassociated
withtheI/Ostack,suchasfilesystems,volumemanagers,devicedrivers,HBAs,storage
systems,orunderlyingdisks.WhenDBBlockChecksumisenabled(defaultinOracle9i)
foreachblockthatismodified,achecksumiscalculatedandstoredintheblock.Whenthe
blockisaccessedthenexttime,thechecksumisvalidatedtomakesurethattheblockisnot

corrupt.DBBlockChecksumdetectsblockcorruptionatacertainlevel.However,because
detectionistriggeredonlyuponthenextreadoftheblocks,anyblocksthatarecorrupted
afterleavingtheOraclelayerarenotpreventedfrombeingwrittentodisk.Aftertheblocks
arepasseddowntotheI/Ostack(operatingsystem,filesystems,volumemanagers,and
storagecontrollers),Oraclecannolongerguaranteethattheblocksbeingwrittentothe
storagesystemsarestillcorrect.Therefore,Oraclealonecannotpreventcorruptioncaused
outsidetheOracleinstancefrombeingwrittentopermanentdisk.
4.1HARDWAREASSISTEDRESILIENTDATA
ThechallengedescribedintheprevioussectionhasledOracletodevelopaninitiativewith
storagevendorstoprovideexternalvalidationmechanismforOracledata.Theprogramis
knownasHardwareAssistedResilientData(HARD).Itrequiresparticipatingvendorsto
incorporateequivalentchecksumalgorithmsintheirsolutionstoverifyblockintegrity
beforetheblocksarecommittedtodisk,therebypreventingcorruptedblocksfrombeing
writtentodisk.TotakeadvantageofHARD,OracledatamustresideonHARDcompliant
storagesystems,andtheHARDvalidationfeaturemustbeenabled.AllOraclewritesare
validatedwithinthestoragesystem.Ifanyincomingwritesappeartobecorrupt,the
storagesystemrejectsthewritestoprotecttheintegrityofthedata.Asaresult,corrupted
datablocksareneverwrittentodisks.UsingtheHARDprogram,storagevendorscan
preventthefollowingtypesofcorruption:
Writesofcorruptedblocks
Writesofblockstoincorrectlocations
ErroneouswritesbyprogramsotherthanOracletoOracledata
Note:HARDissupportedonlythroughOracle10grelease.Atthetimethatthispaperwas
published,Oracle11greleasewasavailablebutitdidnotsupportHARD.Incollaboration
withEmulex,OracleisworkingonT10DIFDataIntegrityextension,astandardfor
completeendtoenddataintegritycheckingforenterprisestoragesystems,whichhasbeen
acceptedintothe2.6.27Linuxkernel.Itincludesgenericsupportfordataintegrityatthe
blockandfilessystemlayers,aswellassupportfortheT10ProtectionInformationModel
andtheDataIntegrityExtensions.FormoreinformationontheprogressofT10DIF,see
thefollowingOracleWebsite:http://oss.oracle.com/projects/dataintegrity/documentation/.
4.2NETAPPSNAPVALIDATOR

SnapValidatoristheNetAppimplementationoftheOracleHARDinitiative,afeature
designedtoleveragetheOraclechecksumalgorithmstodetectandpreventdatacorruption
beforetheyarriveondisk.ByimplementingOracledatavalidationalgorithmsinside
NetAppstoragedevices,NetAppcanpreventcorrupteddatafrombeingwrittento
permanentstorage.SnapValidator,coupledwithOracleDBChecksum,deliversendtoend
checksumfunctionality.ItisalsotightlyintegratedwiththeOracledatabasearchitecture
andcomplieswiththeOracleHARDinitiative.NetAppSnapValidatoroffersthefollowing
keybenefits:

Increaseddatareliabilitythroughendtoenddataprotection
Improvedoperationalefficiencyandavoidedcosts
Modularaccessibility
Multiprotocolsupport
SnapValidatorforOracleisaDataONTAPWAFLenhancementthatallowsNetApp
storagesystemstovalidateOracledatawhenitisbeingwrittentothestorage.With
SnapValidatorenabled,NetAppstoragesystemscanstopdatacorruptionscausedbyother
componentswithintheI/Opathfrombeingwrittentopermanentstorage.Thefollowinglist
brieflydescribestheinternaloperationofSnapValidator:
OracleblocktobewrittenisencodedwithOraclechecksumbyOracleserver.
DataispassedthroughtheI/OstacktoNetAppstorage.
OncedatareachesNetAppstoragesystemandbeforeitiswrittentodisk,NetAppstorage
systemperformsavalidationbasedonthefollowingsetofactions:
Checksumandverificationagainstthedatabeingsent
VerificationofOracleblocknumberagainsttheblockthedataisgettingwritteninto
Verificationoflengthofwriteandtheintendedwritesize
Oraclemagicnumbercheck