Move OCR, Vote File and ASM spfile to a NORMAL Redundant DiskGroup

Pre-Outage

Pre-Requisite
1. Submit request to Linux Admin to create 3 LUNs of 10G each across 3 failgroups.
2. Submit request to Linux Admin to add /opt/app/grid/product/*/bin/ocrconfig to sudoers file

Once Linux Admin confirms the availability of disks, proceed with these steps.

The disks provided were

/dev/sdx1
/dev/sdy1
/dev/sdz1

[mmathoan@ebizoracsb15 ~]$ sudo oracleasm createdisk VOTE_01 /dev/sdx1

ADPassword:
Writing disk header: done
Instantiating disk: done
[mmathoan@ebizoracsb15 ~]$ sudo oracleasm createdisk VOTE_02 /dev/sdy1
Writing disk header: done
Instantiating disk: done
[mmathoan@ebizoracsb15 ~]$ sudo oracleasm createdisk VOTE_03 /dev/sdz1
Writing disk header: done
Instantiating disk: done

Create new DG VOTING_OCR with these disks

create diskgroup VOTING_OCR normal redundancy disk '/dev/oracleasm/disks/VOTE_01'
,'/dev/oracleasm/disks/VOTE_02' ,'/dev/oracleasm/disks/VOTE_03' attribute
'compatible.rdbms'='12.1.0.0.0', 'compatible.asm'='12.1.0.0.0';

Check on Node 1
SQL> select name, state, type from v$asm_diskgroup;

NAME STATE TYPE

------------------------------ ----------- ------
ARCH_CVSAN MOUNTED EXTERN
CVSAN MOUNTED EXTERN
VOTE_OCR MOUNTED EXTERN
VOTING_OCR MOUNTED NORMAL
Repeat these on remaining Nodes (2 to 8)
New Disks are not visible yet.

[mmathoan@ebizoracsb16 ~]$ sudo oracleasm listdisks

CVSAN_ARCH01
CVSAN_ARCH02
CVSAN_ARCH03
CVSAN_ARCH04
CVSAN_DATA01
CVSAN_DATA02
CVSAN_DATA03
CVSAN_DATA04
CVSAN_DATA05
CVSAN_DATA06
CVSAN_DATA07
CVSAN_DATA08
CVSAN_DATA09
CVSAN_DATA10
CVSAN_DATA11
CVSAN_DATA12
CVSAN_DATA13
VOTE01
VOTE02
VOTE03

[mmathoan@ebizoracsb16 ~]$ sudo oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "VOTE_01"
Instantiating disk "VOTE_02"
Instantiating disk "VOTE_03

New Disks should be visible now.

[mmathoan@ebizoracsb16 ~]$ sudo oracleasm listdisks

CVSAN_ARCH01
CVSAN_ARCH02
CVSAN_ARCH03
CVSAN_ARCH04
CVSAN_DATA01
CVSAN_DATA02
CVSAN_DATA03
CVSAN_DATA04
CVSAN_DATA05
CVSAN_DATA06
CVSAN_DATA07
CVSAN_DATA08
CVSAN_DATA09
CVSAN_DATA10
CVSAN_DATA11
CVSAN_DATA12
CVSAN_DATA13
VOTE01
VOTE02
VOTE03
VOTE_01
VOTE_02
VOTE_03

Connect to GRID user

[grid@ebizoracsb16 ~]$ sqlplus "/ as sysasm"

SQL> select name, state, type from v$asm_diskgroup;
NAME STATE TYPE
------------------------------ ----------- ------
VOTING_OCR DISMOUNTED
ARCH_CVSAN MOUNTED EXTERN
CVSAN MOUNTED EXTERN
VOTE_OCR MOUNTED EXTERN

SQL> alter diskgroup VOTING_OCR mount;

Diskgroup altered.
SQL> select name, state, type from v$asm_diskgroup;
NAME STATE TYPE
------------------------------ ----------- ------
ARCH_CVSAN MOUNTED EXTERN
CVSAN MOUNTED EXTERN
VOTE_OCR MOUNTED EXTERN
VOTING_OCR MOUNTED NORMAL

Migrate OCR file to new DG

Current state

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/crsctl query css votedisk

## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 28ed3f3dbba84fc3bf20257e5f32f732 (/dev/oracleasm/disks/VOTE03) [VOTE_OCR]
Located 1 voting disk(s).
[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 901284
Used space (kbytes) : 85852
Available space (kbytes) : 815432
ID : 1064728878
Device/File Name : +VOTE_OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user

Validate new DG can be seen in listing

ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files
Name

MOUNTED EXTERN N 512 512 4096 1048576 614396 592288 0 592288 0 N ARCH_CVSAN/
MOUNTED EXTERN N 512 512 4096 1048576 27262963 4331356 0 4331356 0 N CVSAN/
MOUNTED EXTERN N 512 512 4096 4194304 30708 30040 0 30040 0 Y VOTE_OCR/
MOUNTED NORMAL N 512 512 4096 1048576 30717 29640 10239 9700 0 N VOTING_OCR/

Add OCR file to new DG.

sudo /opt/app/grid/product/19.3.0.0/bin/ocrconfig -add +VOTING_OCR

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/ocrcheck

Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 901284
Used space (kbytes) : 85852
Available space (kbytes) : 815432
ID : 1064728878
Device/File Name : +VOTE_OCR
Device/File integrity check succeeded
Device/File Name : +VOTING_OCR
Device/File integrity check succeeded

Device/File not configured

Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
Outage (shutdown Oracle Database)
Migrate VOTE file to new DG
Current state

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/crsctl query css votedisk

## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 28ed3f3dbba84fc3bf20257e5f32f732 (/dev/oracleasm/disks/VOTE03) [VOTE_OCR]
Located 1 voting disk(s).

Add VOTE file to new DG

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/crsctl replace votedisk +VOTING_OCR
Successful addition of voting disk f1bb616384e34ff8bf4fd41e90c97a1f.
Successful addition of voting disk daee31f946e74fdfbf2cbdec79e056d1.
Successful addition of voting disk aa23ef3b0ea44fdabfc9bfe6ddd6f4ad.
Successful deletion of voting disk 28ed3f3dbba84fc3bf20257e5f32f732.
Successfully replaced voting disk group with +VOTING_OCR.
CRS-4266: Voting file(s) successfully replaced
[grid@ebizoracsb15 ~]$

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/crsctl query css votedisk

## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE f1bb616384e34ff8bf4fd41e90c97a1f (/dev/oracleasm/disks/VOTE_01) [VOTING_OCR]
2. ONLINE daee31f946e74fdfbf2cbdec79e056d1 (/dev/oracleasm/disks/VOTE_02) [VOTING_OCR]
3. ONLINE aa23ef3b0ea44fdabfc9bfe6ddd6f4ad (/dev/oracleasm/disks/VOTE_03) [VOTING_OCR]
Located 3 voting disk(s).

Migrate ASM spfile to new DG

Get/Note current location of ASM spfile

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/gpnptool get

Warning: some command line parameters were defaulted. Resulting command line:
/opt/app/grid/product/19.3.0.0/bin/gpnptool.bin get -o-

<?xml version="1.0" encoding="UTF-8"?><gpnp:GPnP-Profile Version="1.0" xmlns="http://www.grid-

pnp.org/2005/11/gpnp-profile" xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-
pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" ProfileSequence="6"
ClusterUId="de335f6f281cefbeffa5596964b12fb3" ClusterName="ebizscansb15"
PALocation=""><gpnp:Network-Profile><gpnp:HostNetwork id="gen" HostName="*"><gpnp:Network
id="net1" IP="172.16.109.0" Adapter="ens192" Use="public"/><gpnp:Network id="net2"
IP="192.168.1.0" Adapter="ens193"
Use="asm,cluster_interconnect"/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile
id="css" DiscoveryString="+asm" LeaseDuration="400"/><orcl:ASM-Profile id="asm"
DiscoveryString="/dev/oracleasm/disks/*"
SPFile="+VOTE_OCR/ebizscansb15/ASMPARAMETERFILE/registry.253.1119540709" Mode="remote"
Extended="false"/><ds:Signature
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod
Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod
Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference
URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-
signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
<InclusiveNamespaces xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/></ds:Transform></ds:Transforms><ds:DigestMethod
Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>IuheY0i7TNzx54DQpxbpFdQ
DsA4=</ds:DigestValue></ds:Reference></
ds:SignedInfo><ds:SignatureValue>cNkjs9lUtzf1QM5LIeruwyYTGtYNUbcTW0M1E2jG2Inh1UmlL4AnIbYD
mOJT6GwErbpnF4ccKhoa6S3l2U7bqXdzwHJyaGmY/
usznCB7CR8X10eaxbRyenBD5yJbRc2AOiU1sX6oC7rgywGOU/T/
x2IeXp3NbvoZBG0jG2rYZvD0cjTy+KXwuLhCFr9DSoLQ0oYtJMk+83LyI9GBQRD9ghGDw2YjUURW6FiHT4VT
SkWXnpm2sGpIf66+VOg862AqmHwL8gPXv/thSmGicTR8xi17hspnWtG8kSQyx/
seA+CNc5ybR5toCcCb2Lhyf+3jVRd4N4JvlYN0uES1GaMCkw==</ds:SignatureValue></ds:Signature></
gpnp:GPnP-Profile>
Success.

Connect as sysasm

SQL> create pfile='/tmp/cvsan_asm.ora' from spfile;

File created.

SQL> create spfile='+VOTING_OCR' from pfile='/tmp/cvsan_asm.ora';

File created.

Validate spfile is present in new DG

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/gpnptool get

Warning: some command line parameters were defaulted. Resulting command line:

/opt/app/grid/product/19.3.0.0/bin/gpnptool.bin get -o-

<?xml version="1.0" encoding="UTF-8"?><gpnp:GPnP-Profile Version="1.0" xmlns="http://www.grid-

pnp.org/2005/11/gpnp-profile" xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-
pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" ProfileSequence="7"
ClusterUId="de335f6f281cefbeffa5596964b12fb3" ClusterName="ebizscansb15"
PALocation=""><gpnp:Network-Profile><gpnp:HostNetwork id="gen" HostName="*"><gpnp:Network
id="net1" IP="172.16.109.0" Adapter="ens192" Use="public"/><gpnp:Network id="net2"
IP="192.168.1.0" Adapter="ens193"
Use="asm,cluster_interconnect"/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile
id="css" DiscoveryString="+asm" LeaseDuration="400"/><orcl:ASM-Profile id="asm"
DiscoveryString="/dev/oracleasm/disks/*"
SPFile="+VOTING_OCR/ebizscansb15/ASMPARAMETERFILE/registry.253.1186663869" Mode="remote"
Extended="false"/><ds:Signature
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod
Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod
Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference
URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-
signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
<InclusiveNamespaces xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/></ds:Transform></ds:Transforms><ds:DigestMethod
Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>yibV7PBlJKC7jgMGlLjen+mU
NBs=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>NxbQLGd8c8W0cRv2//
BPWG2tRyntLp+rrOzPxAeNSQDRGmUWM78O7gLrp/76GuXeUc3/
XBBhDnr5kGMd765xPBsHmg+wHtHrqtXIvKHB0LlHgM4VNuwRsrhUEwC4h5bnQG5/
OuszBhLPasC3a1Y81WjwS3wUR3LPOXupj1PT3G8u2Nk4ZAiez+ndxJOil72C

jU8AOHcYyBFQAZPgL5qEaLNLsBXt/91Eu3dZiHWrKPNiP2b15LDY3/
X1C2pMz9eTtq8uIqRdlHo1fs3X5m9IYeFNdCzABFSM23BdRI3zzl0H6TdXzxLLGRJZ4ZEFoqVDukHZtvWrWg7
2Vsr0AZvsHg==</ds:SignatureValue></ds:Signature></gpnp:GPnP-Profile>

Success.

Migrate ASM password & backup password file to New DG

Current state

[grid@ebizoracsb15 ~]$ srvctl config asm

ASM home: <CRS home>
Password file: +VOTE_OCR/orapwASM
Backup of Password file: +VOTE_OCR/orapwASM_backup
ASM listener: LISTENER
ASM instance count: ALL
Cluster ASM listener: ASMNET1LSNR_ASM

ASM Password Backup file.

[grid@ebizoracsb15 ~]$ asmcmd

ASMCMD> pwcopy +VOTE_OCR/orapwASM_backup +VOTING_OCR/orapwASM_backup
copying +VOTE_OCR/orapwASM_backup -> +VOTING_OCR/orapwASM_backup
ASMCMD> exit
[grid@ebizoracsb15 ~]$ srvctl modify asm -pwfilebackup +VOTING_OCR/orapwASM_backup

[grid@ebizoracsb15 ~]$ srvctl config asm

ASM home: <CRS home>
Password file: +VOTE_OCR/orapwASM
Backup of Password file: +VOTING_OCR/orapwASM_backup
ASM listener: LISTENER
ASM instance count: ALL
Cluster ASM listener: ASMNET1LSNR_ASM

ASM Password file.

[grid@ebizoracsb15 ~]$ asmcmd

ASMCMD> pwcopy +VOTE_OCR/orapwASM +VOTING_OCR/orapwASM
copying +VOTE_OCR/orapwASM -> +VOTING_OCR/orapwASM

[grid@ebizoracsb15 ~]$ srvctl modify asm -pwfile +VOTING_OCR/orapwASM

[grid@ebizoracsb15 ~]$ srvctl config asm

ASM home: <CRS home>
Password file: +VOTING_OCR/orapwASM
Backup of Password file: +VOTING_OCR/orapwASM_backup
ASM listener: LISTENER
ASM instance count: ALL
Cluster ASM listener: ASMNET1LSNR_ASM

Restart Clusterware
Restart Clustrware to validate files are read from update location.

crsctl stop cluster -all

crsctl start cluster -all

Note: Validate all applicable resources are ONLINE

Delete OCR file from OLD DG

mmathoan@ebizoracsb15 ~]$ sudo /opt/app/grid/product/19.3.0.0/bin/ocrconfig -delete +VOTE_OCR

Validate OCR file is showing in only new DG now.

[grid@ebizoracsb15 ~]$ $ORACLE_HOME/bin/ocrcheck

Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 901284
 Used space (kbytes) : 85852
Available space (kbytes) : 815432
ID : 1064728878
Device/File Name : +VOTING_OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user

Take manual backup of OCR file for validation

Change OCR backup location to new DG

[mmathoan@ebizoracsb15 ~]$ sudo /opt/app/grid/product/*/bin/ocrconfig -backuploc +VOTING_OCR

[grid@ebizoracsb15 ~]$ ocrconfig -showbackuploc

The Oracle Cluster Registry backup location is [+VOTING_OCR].
[mmathoan@ebizoracsb15 ~]$ sudo /opt/app/grid/product/*/bin/ocrconfig -manualbackup
ebizoracsb21 2024/12/02 13:38:10
+VOTING_OCR:/ebizscansb15/OCRBACKUP/backup_20241202_133810.ocr.258.1186666691 3844574830
ebizoracsb15 2024/04/11 11:45:36
+VOTE_OCR:/ebizscansb15/OCRBACKUP/backup_20240411_114536.ocr.265.1166010337 3844574830