Installing Oracle Database 11g and Grid ASM 18c

I. Install 2 nodes server

Install 2 nodes server on Oracle VM VirtualBox and put IP address:

192.168.56.105 dbtest01
192.168.56.106 dbtest02

II. Configure network parameters

1. Change hostname server, use small letters (on both nodes)

Node1: name don’t set underscore (_) point(.) (-) both nodes

# vi /etc/sysctl.conf

kernel.hostname=dbtest01

# vi /etc/hostname

dbtest01

 Save and restart network service to make change affect

# systemctl restart network.service

Node2

# vi /etc/sysctl.conf

kernel.hostname=dbtest02

# vi /etc/hostname

dbtest02

 Save and restart network service to make change affect

# systemctl restart network.service

2. Configure network interface (on both nodes)

- Note: on virtualbox need add network 2 NIC for config heartbeat.

# service network status

Prepared by SUOS Dara 12/2022

 Configured devices:

lo enp0s3,enp0s8,enp0s9

Currently active devices:

lo enp0s3,enp0s8,enp0s9

# ifconfig -a

- Configure heartbeat (Note: in case of network cable for heartbeat already plug-in but interface
not up, use command ifup interface_name (Ex: ifup ifcfg-enp0s8))
- Disable virtual interface unnecessary to use(virbr0)

systemctl status libvirtd.service

systemctl stop libvirtd.service
systemctl disable libvirtd.service

Node1

# vi /etc/sysconfig/network-scripts/ifcfg-enp0s8

TYPE=Ethernet
BOOTPROTO=static
NAME=enp0s8
DEVICE=enp0s8
ONBOOT=yes
IPADDR=192.168.222.1
NETMASK=255.255.255.252

# vi /etc/sysconfig/network-scripts/ifcfg-enp0s9

TYPE=Ethernet
BOOTPROTO=static
NAME=enp0s9
DEVICE=enp0s9
ONBOOT=yes
IPADDR=192.168.223.1
NETMASK=255.255.255.252

 Save change and restart network service

# systemctl restart network.service

Prepared by SUOS Dara 12/2022

 Node2

# vi /etc/sysconfig/network-scripts/ifcfg-enp0s8

TYPE=Ethernet
BOOTPROTO=static
NAME=enp0s8
DEVICE=enp0s8
ONBOOT=yes
IPADDR=192.168.222.2
NETMASK=255.255.255.252

# vi /etc/sysconfig/network-scripts/ifcfg-enp0s9

TYPE=Ethernet
BOOTPROTO=static
NAME=enp0s9
DEVICE=enp0s9
ONBOOT=yes
IPADDR=192.168.223.2
NETMASK=255.255.255.252

 Save change and restart network service

# systemctl restart network.service

3. Add more information IP into hosts file (on both nodes)

# vi /etc/hosts

Prepared by SUOS Dara 12/2022

 #public

192.168.56.105 dbtest01 # Public Network on enp0s3

192.168.56.106 dbtest02 # Public Network on enp0s3

#Private

192.168.222.1 dbtest01-priv1 # Private InterConnect

192.168.222.2 dbtest02-priv1 # Private InterConnect

192.168.223.1 dbtest01-priv2 # Private InterConnect

192.168.223.2 dbtest02-priv2 # Private InterConnect

#Virtual

192.168.56.107 dbtest01-vip # VIP on enp0s3

192.168.56.108 dbtest02-vip # VIP on enp0s3

#Scan

192.168.56.109 dbtest12-scan

 Save and test to ping connection between both nodes

4. Install some packages for database

# yum install binutils-* -y
# yum install compat-libstdc++-* -y
# yum install elfutils-libelf-* -y
# yum install libaio-* -y
# yum install libaio-devel-* -y
# yum install sysstat-* -y
# yum install glibc-* -y
# yum install glibc-common-* -y
# yum install glibc-devel-* -y
# yum install glibc-headers-* -y
# yum install ksh-* -y
# yum install make-* -y
# yum install libgcc-* -y
# yum install libstdc++-* -y
# yum install libstdc++-* -y
# yum install libstdc++-devel-* -y
# yum install gcc-* -y
# yum install gcc-c++-* -y
# yum install elfutils-libelf-* -y
# yum install elfutils-libelf-devel-* -y
# yum install elfutils-libelf-* -y
# yum install elfutils-libelf-devel-* -y
Prepared by SUOS Dara 12/2022
 # yum install libtool-ltdl* -y
# yum install ncurses* -y
# yum install readline* -y
# yum install unixODBC* -y

5. Install X11 environment

# yum install gdm* -y
# yum groupinstall "X Window System" "GNOME Desktop Environment"
# yum install gnome* -y
# yum install xterm -y

6. Disable SELinux Permanently

# vi /etc/sysconfig/selinux

SELINUX=disabled

# sestatus

If its status not yet disable => reboot server.

III. Configure users profile (on both nodes)

1. Create users and groups
# groupadd --gid 54321 oinstall
# groupadd --gid 54322 dba
# groupadd --gid 54323 asmdba
# groupadd --gid 54324 asmoper
# groupadd --gid 54325 asmadmin
# useradd --uid 54321 --gid oinstall --groups dba,asmdba,asmoper oracle
# echo "qaz123@A" | passwd --stdin oracle
# useradd --uid 54322 --gid oinstall --groups dba,asmadmin,asmdba,asmoper grid
# echo "qaz123@A" | passwd --stdin grid
2. Configure parameters for users (both nodes)
3.1 Configure parameter kennel all nodes

vi /etc/sysctl.conf

##Oracle setting
vm.swappiness = 1
vm.dirty_background_ratio = 3
vm.dirty_ratio = 80
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
kernel.shmmax = 4398046511104
kernel.shmall = 1073741824
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
# fs.file-max needs to be set to at least 6815744 for Oracle Installation.
fs.file-max = 6815744

Prepared by SUOS Dara 12/2022

 fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
kernel.panic_on_oops = 1

To apply new configure

#/sbin/sysctl -p

3.2 Install package preinstall

yum install oracle-database-preinstall-19c*

3.3 Set profile for Oracle user

Login as oracle user and add more the following line setting into bash_profile

$ vi ~/.bash_profile

Prepared by SUOS Dara 12/2022

##notes: Change name export ORACLE_SID=dbtest2 on node2 and node1 change dbtest1

# Oracle Settings

export ORACLE_BASE=/u01/app/oracle;

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1;

export ORACLE_SID=dbtest2;

export ORACLE_TERM=xterm;

export GRID_HOME=/u01/18.0.0/grid

export PATH=$ORACLE_HOME/bin:$GRID_HOME/bin:$PATH:/usr/sbin;

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

export TNS_ADMIN=$ORACLE_HOME/network/admin

if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

fi

 Save and exit

3.4 Set profile for Grid user

Login as grid user and add more the following line setting into bash_profile

##notes: Change name ORACLE_SID=+ASM2 on node2 and node1 change +ASM1

$ vi ~/.bash_profile

Prepared by SUOS Dara 12/2022

 # Grid setting

export TMP=/tmp;

export TMPDIR=$TMP;

export GRID_BASE=/u01/app/grid

export ORACLE_BASE=$GRID_BASE;

export GRID_HOME=/u01/18.0.0/grid

export ORACLE_HOME=$GRID_HOME;

export ORACLE_SID=+ASM2
 Save and exit
3.export PATH=$GRID_HOME/bin:$PATH:/usr/sbin;
Create Directories both nodes
Login:
export root
LD_LIBRARY_PATH=$GRID_HOME/lib;
mkdir -p /u01/app/oraInventory
chown -R grid:oinstall /u01/app/oraInventory
chmod -R 775 /u01/app/oraInventory

mkdir -p /u01/app/grid
mkdir -p /u01/app/grid/cfgtoollogs
mkdir -p /u01/18.0.0/grid
mkdir -p /u01/app/grid/cfgtoollogs
chown -R grid:oinstall /u01
chmod -R 775 /u01/app/grid
chown -R grid:oinstall /u01/app/grid
chmod -R 775 /u01/app/grid
chown -R grid:oinstall /u01/18.0.0/grid
chmod -R 775 /u01/18.0.0/grid

mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/cfgtoollogs
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle

mkdir -p /u01/app/oracle/product/11.2.0/db_1
chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/db_1
chmod -R 775 /u01/app/oracle/product/11.2.0/db_1

chmod -R 775 /u01/app/oraInventory

chmod -R 755 /u01/app/
chmod -R 775 /u01/18.0.0/grid
chmod 755 /u01/18.0.0/
chown root:oinstall /u01/18.0.0/

4. Configure ssh between two nodes (not to ask for password when using ssh)
5.1 Login as oracle user
$ mkdir ~/.ssh
Prepared by SUOS Dara 12/2022
 $ chmod 755 ~/.ssh
$ /usr/bin/ssh-keygen -t rsa
 type enter until finish

$ /usr/bin/ssh-keygen -t dsa

 type enter until finish

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

$ chmod 644 ~/.ssh/authorized_keys

 Copy contents in file authorized_keys (more ~/.ssh/authorized_keys) on node1 and add into
node2. After that, before copy file authorized_keys from node 2 into node 1 pls run script
(node2) by exsample below and then copy authorized_keys

Exsample: Run script on node2

$ /usr/bin/ssh-keygen -t rsa
 type enter until finish

$ /usr/bin/ssh-keygen -t dsa

 type enter until finish

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

$ scp ~/.ssh/authorized_keys [email protected]:/home/oracle/.ssh

5.2 Login as grid user

 Run all steps the same as 5.1 and copy file authorized_keys to node 1 by grid user
$ scp ~/.ssh/authorized_keys [email protected]:/home/grid/.ssh

##Test ssh both nodes

# ssh dbtest01 (user:oracle,grid on node 1) itself
[oracle@dbtest01]$ ssh dbtest01
[grid@dbtest01]$ ssh dbtest01

# ssh dbtest02 (user:oracle,grid on node 2) itself

[oracle@dbtest02]$ ssh dbtest02
[grid@dbtest02]$ ssh dbtest02

# ssh dbtest01 (user:oracle,grid on node 2)

# ssh dbtest02 (user:oracle,grid on node 1)

[oracle@dbtest01]$ ssh dbtest02

Prepared by SUOS Dara 12/2022

 [grid@dbtest01]$ ssh dbtest02
[oracle@dbtest02]$ ssh dbtest01
[grid@dbtest02]$ ssh dbtest01

Confirm yes all nodes test ssh.

5.3 Configure user Equivalence (by root user)

# exec /usr/bin/ssh-agent $SHELL
# /usr/bin/ssh-add

IV. Create disks shareable (on both nodes) (by root user)
Planning for Grid Infrastructure 18c
Refer this link
https://www.dell.com/support/kbdoc/en-ae/000179544/how-to-deploy-oracle-18c-grid-and-
standalone-database-on-red-hat-enterprise-linux-7-x

1. file >> Virtual Media manager>>create

Ex: Create CRS disk >> choose VDI (Virtualbox disk image) >> Fixed size >> create

Prepared by SUOS Dara 12/2022

 Note: another disks create the same step by step (ex: archived,redolog,data……)

2. add disk after create virtual disks

- add disk devices >> select node1 >> right click setting

>> choose add disk select disks name (EX: redolog01…..)

Note: choose one by one disks virtual.

Node2 add disk do the same node1

Prepared by SUOS Dara 12/2022

 3. Format disks partition which mounted from Storage (only on node1)
- Check disk
fdisk -l
- Format disk partition
# fdisk /dev/sdc
Command (m for help): p

Command (m for help): n

Select (default p): p

Partition number (1-4, default 1): 1

Command (m for help): w

The partition table has been altered!

*Follow the same cycle steps for other disk devices

# fdisk /dev/sdc
# fdisk /dev/sdd
# fdisk /dev/sde
# fdisk /dev/sdf
# fdisk /dev/sdg
# fdisk /dev/sdh
# fdisk /dev/sdi

4. Configure Oracle ASM (by root user) both servers

- Install oracleasm package
# yum -y install oracleasm-support-*
# yum -y install kmod-oracleasm-*

- Configure Oracle ASM library driver

# oracleasm configure -i

Prepared by SUOS Dara 12/2022

 Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting <ENTER> without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid

Default group to own the driver interface []: asmdba

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

- Check ASM status

# oracleasm status
Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

*If its status shown “no”, run the below command and check its status again.
# oracleasm init

5. Create ASM disk (create only on node1)

*Check each size with mpath name carefully before create

# /usr/sbin/oracleasm createdisk CRS001 /dev/sdc1

# /usr/sbin/oracleasm createdisk CRS002 /dev/sdd1

# /usr/sbin/oracleasm createdisk CRS003 /dev/sde1

# /usr/sbin/oracleasm createdisk REDOLOG01 /dev/sdf1

# /usr/sbin/oracleasm createdisk REDOLOG02 /dev/sdg1

# /usr/sbin/oracleasm createdisk ARCHIVED /dev/sdh1

# /usr/sbin/oracleasm createdisk DATA_01 /dev/sdi1

Note: After fdisk not show /dev/sdx1 => create disk fail. Run command below to refresh
disk partition.

# partprobe /dev/sdx*

- Check disk after created

# /usr/sbin/oracleasm listdisks
Prepared by SUOS Dara 12/2022
 ARCHIVED

CRS001

CRS002

CRS003

DATA

REDOLOG01

REDOLOG02

- Scan disk on node2 (after create disk successfully on node1)

# /usr/sbin/oracleasm scandisks
# /usr/sbin/oracleasm listdisks
V. Installing grid infrastructure
1. Allow root login direct
vi /etc/ssh/sshd_config
PermitRootLogin yes
AllowUsers oracle grid root

#restart service to apply new configuration

service sshd restart

2. Copy software file for installing

login user grid
copy soft to home of grid
sftp> cd /u01/18.0.0/grid
sftp> lcd "E:\Oracle Database 18c Grid"
put GI_V978971-01.zip
unzip

3. Install package
login root

cd /u01/18.0.0/grid/cv/rpm

rpm -ivh cvuqdisk-1.0.10-1.rpm

copy to node2 install

[grid@oradb01 rpm]$ scp cvuqdisk-1.0.10-1.rpm grid@dbtest02:/home/grid

node 2 login root

cd /home/grid

rpm -ivh cvuqdisk-1.0.10-1.rpm

4. Install grid software

Prepared by SUOS Dara 12/2022

 - Before installing (on both nodes):
 make sure that service iptables and firewall are off
 synchronize time on server with its gateway
# /bin/systemctl stop firewalld.service
# /bin/systemctl disable firewalld.service
# systemctl stop firewalld
# systemctl stop iptables

- login as grid user by interface mode (install only on node1) by x Brower manager
$ cd /u01/18.0.0/grid
$ ./gridSetup.sh &

 On the select configuration option => choose Configure Oracle Grid Infrastructure for a New
Cluster

Prepared by SUOS Dara 12/2022

  choose select Configure an Oracle Standalone Cluster

 Fill Cluster Name:

Uncheck on Configure GNS => Next

 Click add node

Prepared by SUOS Dara 12/2022

  After click add node, fill node name(node2)

 Click SSH connectivity

Prepared by SUOS Dara 12/2022

  Fill password Grid=> click Setup

 Click OK

Prepared by SUOS Dara 12/2022

  Click Next Step

 Choose Specify Network Interface Usage:

Public: For interface public IP.
ASM & Private: For interface interconnect(one or two depends on actual configuration).
Click Next

Prepared by SUOS Dara 12/2022

  Choose Configure ASM using block devices => Next

 GIMR data: No => Next

Prepared by SUOS Dara 12/2022

  Fill diskgroup name: CRS
Select: Normal
Change Discovery Path: /dev/oracleasm/disks
Choose 3 ASM disk

 ASM Password: Use same passwords for these accounts

Ora123qaz

Prepared by SUOS Dara 12/2022

  Confirm YES(Notice password not standard)

Prepared by SUOS Dara 12/2022

  Select Do Not use Intelligent Platform Management Interface (IPMI)

 Skip register with EM later

 Privilege OS group

Prepared by SUOS Dara 12/2022

  Specify installation location
Correct path we already defined in bash_profile grid

 Create inventory

Prepared by SUOS Dara 12/2022

  Root script execution configuration: fill password root. Click Next

 Prerequisite Checks

Prepared by SUOS Dara 12/2022

  Notification about Physical Memory

 Summary

Prepared by SUOS Dara 12/2022

 Click Install
 Installing progress

 Confirm root scripts execution. Click YES

Prepared by SUOS Dara 12/2022

  Installing product 69%

 Install finish

Prepared by SUOS Dara 12/2022

 VI. Create ASM disk groups
Login as grid user by interface mode by x Brower manager
$ asmca
 Select Disk Groups

 Create… =>Disk Group Name: ARCHIVED => External => Select Disk Path
/dev/oralceasm/disks/ARCHIVED => OK

Prepared by SUOS Dara 12/2022

  Create the same with DATA, REDOLOG01,REDOLOG02…
VII. Install Oracle software
Login SSH as oracle copy soft intall to /u01/soft. Run extract all.
Login as oracle user by interface mode (only on node1)
$ cd /u01/soft/database
$ ./runInstaller

 Can leave the mail blank and uncheck I wish to receive security update via…… => Next

 Skip software updates => Next

Prepared by SUOS Dara 12/2022

  Install database software only => Next

 Choose Oracle Real Aplication Clusters databse installation => Next

Prepared by SUOS Dara 12/2022

  Choose English => Next

 Choose Enterprise Edition => Next

Prepared by SUOS Dara 12/2022

  Check/browse its path => Next

Prepared by SUOS Dara 12/2022

  Change group of Database Operator to oinstall => Next

 While processing check,

o if any Fixable is Yes => click on Fix & Check Again
o If No => tick on Ignore All => Next

Prepared by SUOS Dara 12/2022

  Install

erpbccs01, erpbccs02

 While installing in progress

o User oracle If pop-up window error appeared, edit parameter in ins_emagent.mk file
(only nodes 1)

$ vi $ORACLE_HOME/sysman/lib/ins_emagent.mk

Prepared by SUOS Dara 12/2022

 change : $(MK_EMAGENT_NMECTL)

to: $(MK_EMAGENT_NMECTL) -lnnz11

=> Save files then Retry

o Execute script as in pop-up window appeared by root user (on both nodes)
# /u01/app/oracle/product/11.2.0/db_1/root.sh
Note: run script on node1 then node2.
=> OK

 Wait to finish and Close

Prepared by SUOS Dara 12/2022

 VIII. Create Database
Login as oracle
$dbca

Prepared by SUOS Dara 12/2022

Prepared by SUOS Dara 12/2022
Prepared by SUOS Dara 12/2022
Prepared by SUOS Dara 12/2022
Prepared by SUOS Dara 12/2022
Prepared by SUOS Dara 12/2022