RAC12c Theory Guide v8.0
RAC12c Theory Guide v8.0
Theory Guide
www.ucliditschool.com
Uclid IT School RAC Theory Guide
Disclaimer
This document is proprietary of Uclid IT School. You may use this document solely for your own use. The
document may not be modified or altered in any way. Except where your use constitutes "fair use" under
copyright law, you may not use, share, download, upload, copy, print, display, perform, reproduce,
publish, license, post, transmit, or distribute this document in whole or in part without the express
authorization of Uclid IT School.
The information contained in this document is subject to change without notice. If you find any errors in
this document, please report them in writing to [email protected] . This document is not warranted
to be error-free.
Oracle, Java are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be
trademarks of their respective owners.
.
Contents
1. INTRODUCTION TO ORACLE RAC ........................................................................................4
2. ORACLE CLUSTERWARE CONCEPTS ................................................................................ 12
3. ASM CONCEPTS ....................................................................................................................... 21
4. NEW FEATURES IN RAC 12C ............................................................................................... 32
5. PREPARING FOR GRID INFRASTRUCTURE INSTALLATION ..................................... 38
6. ADMINISTERING ORACLE CLUSTERWARE .................................................................... 51
7. ADMINISTERING ASM INSTANCES .................................................................................... 67
8. ADMINISTERING ASM DISK GROUPS ............................................................................... 79
9. ASM COMMAND LINE UTILITY ........................................................................................... 91
10.ORACLE RAC ARCHITECTURE ............................................................................................ 97
11.ADMINISTERING RAC INSTANCES & DATABASE ...................................................... 102
12.WORKLOAD MANAGEMENT WITH SERVICES ............................................................ 115
13.PATCHING RAC ENVIRONMENTS ................................................................................... 122
14.MIGRATING NON-ASM DATABASE TO ASM ................................................................ 127
15.CONVERTING SINGLE INSTANCE DATABASE TO RAC DATABASE ...................... 130
A common cluster architecture for data-intensive transactions and computations is built around
shared disk storage. Shared-nothing clusters use an alternative architecture where storage is
not shared and data must be either replicated or segmented across the cluster. Shared-nothing
clusters are commonly used for workloads that can be easily and predictably divided into small
units that can be spread across the cluster in parallel. Shared disk clusters can perform these
tasks but also offer increased flexibility for varying workloads.
Load balancing clusters allow a single application to balance its workload across the cluster.
Alternatively, in a failover cluster, some nodes can be designated as the primary host for an
application, whereas others act as the primary host for different applications. In a failover
cluster, the failure of a node requires that the applications it supports be moved to a surviving
node.
Load balancing clusters can provide failover capabilities but they can also run a single
application across multiple nodes providing greater flexibility for different workload
requirements. Oracle supports a shared disk cluster architecture providing load balancing and
failover capabilities. In an Oracle cluster, all nodes must share the same processor architecture
and run the same operating system.
l.
ai
gm
About Clusterware
7@
Clusterware is a term used to describe software that provides interfaces and services that
enable and support a cluster. Different cluster architectures require clusterware that delivers
71
different services.
n2
wa
For example, in a simple failover cluster the clusterware may monitor the availability of
applications and perform a failover operation if a cluster node becomes unavailable. In a load
ha
balancing cluster, different services are required to support workload concurrency and
id
coordination.
it
What is RAC?
Real Application Clusters (RAC) is a software that enables you to use clustered hardware by
running multiple Instances against the same database. The database files are stored on disks
that are either physically or logically connected to each node, so that every active instance can
read from or write to them.
RAC software manages data access, so that changes are coordinated between the instances
and each instance sees a consistent image of the database. The cluster interconnect enables
instances to pass coordination information and data images between each other.
Oracle Real Application Clusters (Oracle RAC) enables an Oracle database to run across a
cluster of servers, providing fault tolerance, performance, and scalability with no application
changes necessary. Oracle RAC provides high availability for applications by removing the
single point of failure with a single server.
Oracle RAC enables you to combine smaller commodity servers into a cluster to create scalable
environments that support mission critical business applications. Applications that you deploy on
Oracle RAC databases can operate without code changes. In an Oracle RAC database, multiple
instances access a single set of database files (i.e. single database) . Having multiple database
instances accessing a single set of data files prevents the server from single point of failure. If a
node in the cluster fails, then the Oracle database continues running on the remaining nodes.
Individual nodes can be shut down for maintenance while application users continue to work. An
Oracle RAC database can have up to 100 instances, all of which access one database.
Oracle RAC extends Oracle Database so that you can store, update, and efficiently retrieve
data using multiple database instances on different servers at the same time. Oracle RAC
provides the software that manages multiple servers and instances as a single group. The data
files that comprise the database must reside on shared storage that is accessible from all
servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.
A big difference between Oracle RAC and OPS is the addition of Cache Fusion. With OPS a
request for data from one instance to another required the data to be written to disk first, then
the requesting instance can read that data (after acquiring the required locks). This process was
called disk pinging. With cache fusion, data is passed along a high-speed interconnect using a
sophisticated locking algorithm.
Not all database clustering solutions use shared storage. Some vendors use an approach
known as a Federated Cluster, in which data is spread across several machines rather than
shared by all. With Oracle RAC, however, multiple instances use the same set of disks for
storing data. Oracle's approach to clustering leverages the collective processing power of all the
nodes in the cluster and at the same time provides failover security.
Oracle Clusterware supports NFS, iSCSI, Direct Attached Storage (DAS), Storage Area
Network (SAN) storage, and Network Attached Storage (NAS).
All data files (including an undo tablespace for each instance) and redo log files (at least two for
each instance) for an Oracle RAC database must reside on shared storage. Oracle
recommends that you use Oracle ASM to store these files in an Oracle ASM disk group.
Once you have the storage attached to the servers, you have 2 choices on how to setup the
disks
Cluster File System (CFS) - used to hold all the Oracle data files. Can be used by
windows and Linux, its not used widely
Automatic Storage Management (ASM) - Oracle choice of storage management, its a
portable, dedicated and optimized cluster file system
Oracle strongly recommends using Automatic Storage Management for Oracle RAC.
Oracle Clusterware
A cluster comprises of multiple interconnected computers or servers that appear as if they are
one server to end users and applications. The Oracle RAC option with Oracle Database enables
you to cluster Oracle databases. Oracle RAC uses Oracle Clusterware for the infrastructure to
bind multiple servers so they operate as a single system. Oracle Clusterware is a portable
cluster management solution that is integrated with Oracle Database. Oracle Clusterware is a
required component for using Oracle RAC that provides the infrastructure necessary to run
Oracle RAC. Oracle Clusterware provides all of the features required to manage the cluster
database including node membership, group services, global resource management, and high
availability functions. Oracle Clusterware also manages resources, such as Virtual Internet
Protocol (VIP) addresses, databases, listeners, services, and so on. In addition, Oracle
Clusterware enables both non-cluster Oracle databases and Oracle RAC databases to use the
Oracle high-availability infrastructure. Oracle Clusterware along with Oracle Automatic
Storage Management (Oracle ASM) (the two together comprise the Oracle Grid
Infrastructure) enables you to create a clustered pool of storage to be used by any
combination of non-cluster and Oracle RAC databases. Oracle Clusterware is the only
Clusterware that you need for most platforms on which Oracle RAC operates.
When you create an Oracle RAC database using any of the management tools, the database is
registered with and managed by Oracle Clusterware, along with the other required components
such as the VIP address, the Single Client Access Name (SCAN) (which includes the SCAN
VIPs and the SCAN listener), Oracle Notification Service, and the Oracle Net listeners. These
resources are automatically started when the node starts and automatically restart if they fail.
The Oracle Clusterware daemons run on each node.
Anything that Oracle Clusterware manages is known as a CRS resource. A CRS resource can
be a database, an instance, a service, a listener, a VIP address, or an application process.
Oracle Clusterware manages CRS resources based on the resource's configuration information
that is stored in the Oracle Cluster Registry (OCR). You can use SRVCTL commands to
administer any Oracle-defined CRS resources. Oracle Clusterware provides the framework that
enables you to create CRS resources to manage any process running on servers in the cluster
which are not predefined by Oracle. Oracle Clusterware stores the information that describes
the configuration of these components in OCR.
In Oracle 11g Release 2, Oracle Clusterware and ASM are installed into a single home directory
called Oracle Grid Infrastructure. Oracle Clusterware supports up to 100 nodes in a cluster on
configurations running Oracle Database 10g release 2 (10.2) and later releases.
Further, in Oracle 12c Release 1, Oracle has introduced Flex cluster concept. A flex cluster
uses hub and spoke topology. Flex Cluster is designed to scale up to 64 hub nodes and many
more Leaf nodes.
Cluster Interconnect
The cluster interconnect is used to synchronize the resources of the RAC cluster, and also used
to transfer some data from one instance to another. This interconnect should be private, highly
available and fast with low latency. Ideally they should be on a minimum 1GB private network.
In RAC, as more than one instance is accessing the resource, the instances require better
coordination at the resource management level. Each node will have its own set of buffers but
will be able to request and receive data blocks currently held in another instance's cache. The
management of data sharing and exchange is done by the Global Cache Services (GCS).
All the resources in the cluster group form a central repository called the Global Resource
Directory (GRD), which is distributed. Each instance masters some set of resources and
together all instances form the GRD. The resources are equally distributed among the nodes
based on their weight. The GRD is managed by two services called Global Caches Services
(GCS) and Global Enqueue Services (GES), together they form and manage the GRD. When a
node leaves the cluster, the GRD portion of that instance needs to be redistributed to the
surviving nodes, a similar action is performed when a new node joins.
Benefits of RAC
RAC architecture enables users and applications to benefit from the processing power of
multiple machines.
RAC harnesses the power of multiple low-cost computers to serve as single large
computer for database processing and provides the only viable alternative to large scale
symmetric multiprocessing (SMP) for all types of applications. If your application scales
transparently on SMP machines, it is realistic to expect it to scale well on RAC, without
having to make any changes to the application code.
RAC, which is based on shared-disk architecture, can grow or shrink on demand
RAC allows you to perform dynamic provisioning of nodes, storage, CPUs, and memory
to maintain service levels more easily and efficiently.
RAC architecture also achieves redundancy in the case of system failure or
unavailability. The application can still access the database on surviving instances.
RAC eliminates the database instance and/or the node , as a single point of failure and
ensures database integrity in case of such failures
RAC offers automatic workload management for services. Services in RAC enable
continuous, uninterrupted database operations
The combined processing power of the multiple servers can provide greater throughput
and scalability than is available from a single server.
Oracle Universal Installer (OUI): OUI installs the Oracle grid infrastructure software (which
consists of Oracle Clusterware and Oracle ASM) and the Oracle Database software with
Oracle RAC. After you configure the nodes that you want to use in your cluster, OUI installs
the Oracle software on the specified nodes using a network connection
Cluster Verification Utility (CVU): CVU is a command-line tool that you can use to verify a
range of cluster and Oracle RAC components such as shared storage devices, networking
configurations, system requirements, and Oracle Clusterware, as well as operating system
groups and users. You can use CVU for preinstallation as well as post installation checks of
your cluster environment. CVU is especially useful during preinstallation and during
installation of Oracle Clusterware and Oracle RAC components. OUI runs CVU after the
Oracle Clusterware installation to verify your environment.
Oracle Enterprise Manager: Oracle Enterprise Manager Cloud Control GUI interface for
managing both noncluster database and Oracle RAC database environments. Oracle
recommends that you use Oracle Enterprise Manager to perform administrative tasks
whenever feasible. You can use Oracle Enterprise Manager Cloud Control to also manage
Oracle RAC One Node databases.
SQL*Plus: SQL*Plus commands operate on the current instance. The current instance can
be either the local default instance on which you initiated your SQL*Plus session, or it can
be a remote instance to which you connect with Oracle Net Services.
Server Control Utility (SRVCTL): SRVCTL is a command-line interface that you can use to
manage an Oracle RAC database from a single point. You can use SRVCTL to start and
stop the database and instances and to delete or move instances and services. You can
also use SRVCTL to manage configuration information, Oracle Real Application Clusters
One Node (Oracle RAC One Node), Oracle Clusterware, and Oracle ASM.
Cluster Ready Services Control (CRSCTL): CRSCTL is a command-line tool that you can
use to manage Oracle Clusterware daemons. These daemons include Cluster
Synchronization Services (CSS), Cluster-Ready Services (CRS), and Event Manager
(EVM). You can use CRSCTL to start and stop Oracle Clusterware and to determine the
current status of your Oracle Clusterware installation.
DBCA: The recommended utility for creating and initially configuring Oracle RAC and Oracle
noncluster databases.
OCR manages Oracle Clusterware and Oracle RAC database configuration information
OLR resides on every node in the cluster and manages Oracle Clusterware configuration
information for each particular node
Voting disks manage information about node membership. Each voting disk must be
accessible by all nodes in the cluster for nodes to be members of the cluster
You can store OCR and voting disks on Oracle Automatic Storage Management (Oracle ASM),
or a certified cluster file system
Voting Disks
The voting disk (or quorum disk) is a file, where heartbeat information of all nodes will be stored
by CSSD continuously. Oracle Clusterware uses voting disk files to determine which nodes are
members of a cluster. You can configure voting disks on Oracle ASM, or you can configure
voting disks on shared storage. If you configure voting disks on Oracle ASM, then you do not
need to manually configure the voting disks. Depending on the redundancy of your disk group,
an appropriate number of voting disks are created.
When voting disks are placed into an ASM diskgroup, the number of voting disks that are
created depend on the redundancy level of the diskgroup. For external redundancy, 1 voting
disk is created. When running ASM redundancy, voting disks have a higher requirement for the
number of disks. Normal redundancy creates 3 voting disks, and high redundancy creates 5.
Oracle recommends at least 400MB per voting disk file, and 400MB for each copy of the OCR.
If you do not configure voting disks on Oracle ASM, then for high availability, Oracle
recommends that you have a minimum of three voting disks on physically separate storage.
This avoids having a single point of failure. If you configure a single voting disk, then you must
use external mirroring to provide redundancy. You should have at least three voting disks,
unless you have a storage device, such as a disk array that provides external redundancy.
Oracle recommends that you do not use more than five voting disks.
As far as voting disks are concerned, a node must be able to access strictly more than half of
the voting disks at any time. So if you want to be able to tolerate a failure of “n” voting disks, you
must have at least 2n+1 configured. (n=1 means 3 voting disks). You can configure minimum of
1 and maximum 15 copies of voting disks.
Once you configure voting disks on Oracle ASM, you can only make changes to the voting
disks' configuration using the crsctl replace votedisk command. This is true even in
cases where there are no working voting disks. Despite the fact that crsctl query css
votedisk reports zero vote disks in use, Oracle Clusterware remembers the fact that Oracle
ASM was in use and the replace verb is required. Only after you use the replace verb to
move voting disks back to non-Oracle ASM storage are the verbs add css votedisk and
delete css votedisk again usable.
The number of voting files you can store in a particular Oracle ASM disk group depends upon
the redundancy of the disk group.
External redundancy: A disk group with external redundancy can store only 1 voting disk
Normal redundancy: A disk group with normal redundancy stores 3 voting disks
High redundancy: A disk group with high redundancy stores 5 voting disks
Starting from Oracle 11g release 2 (11.2), if voting disks are stored on Oracle ASM with normal
or high redundancy, and the storage hardware in one failure group suffers a failure, then if there
is another disk available in a disk group in an unaffected failure group, Oracle ASM
automatically recovers the voting disk in the unaffected failure group. In previous versions, this
automation did not exist. If a voting disk went offline, the DBA had to manually create a new
voting disk.
Keep in mind that this feature will only take effect if there must be another failgroup available to
place the voting disk in. If you only have 3 failgroups for your normal redundancy (or 5 for your
high redundancy) OCR/voting diskgroup, this automatic recreation will not occur.
Using the crsctl replace votedisk command, you can move a given set of voting
disks from one Oracle ASM disk group into another, or onto a certified file system. If you move
voting disks from one Oracle ASM disk group to another, then you can change the number of
voting disks by placing them in a disk group of a different redundancy level as the former disk
group.
Note:
You cannot directly influence the number of voting disks in ASM disk group.
You cannot use the crsctl add | delete votedisk commands on voting
disks stored in Oracle ASM disk groups because Oracle ASM manages the number of
voting disks according to the redundancy level of the disk group.
node membership information i.e. which nodes are part of the cluster
software active version
location of voting disks
server pools
status for the cluster resources such as RAC databases, listeners, instances, and services
− server up/down
− network up/down
− database up/down
− instance up/down
− listener up/down …
configuration for the cluster resources such as RAC databases, listeners, instances, and
services.
− dependencies
− management policy (automatic/manual)
− callout scripts
− retries
l.
ai
gm
CSSd during cluster setup - to update the status of servers
CSS during node addition/deletion – to add/delete node names
7@
CRSd about status of nodes during failure/reconfiguration
71
Oracle Universal Installer (OUI)
n2
SRVCTL (used to manage clusters and RAC databases/instance)
Cluster control utility – CRSCTL (to manage cluster/local resources)
wa
Each node in the cluster maintains a copy of the OCR in memory for better performance and
each node is responsible for updating the OCR as required. The CRSd process is responsible
for reading and writing to the OCR files as well as refreshing the local OCR cache and the
caches on the other nodes in the cluster. Oracle uses a distributed shared cache architecture
during cluster management to optimize queries against the cluster repository. Each node
maintains a copy of the OCR in memory. Oracle Clusterware uses a background process to
access the OCR cache. Only one CRSd process (designated as the master) in the cluster
performs any disk read/write activity. Once any new information is read by the master CRSd
process, it performs a refresh of the local OCR cache and the OCR cache on other nodes in the
cluster. Since the OCR cache is distributed across all nodes in the cluster, OCR clients (srvctl,
crsctl etc.) communicate directly with the local OCR process on the node to obtain required
information. Clients communicate via the local CRSd process for any updates on the physical
OCR binary file.
The following block diagram shows the architecture of Oracle Cluster Registry.
All SYSTEM level resource or application definitions would require root, or super user,
privileges to start, stop, and execute resources defined at this level.
those defined at the DATABASE level will require dba privileges to execute
Considering the criticality of the OCR contents to the cluster functionality, Oracle strongly
recommends you to multiplex the OCR file. From Oracle 11g Release 2, you can have up to 5
OCR copies.
NOTE: When using ASM Diskgroup for Grid Infrastructure, the following files are generally
stored in ASM Diskgroup:
Multiple processes on each node have simultaneous read and write access to the OLR
particular to the node on which they reside, regardless of whether Oracle Clusterware is running
or fully functional.
OLR is the very first file that is accessed to startup clusterware when OCR is stored on ASM.
OCR should be accessible to find out the resources which need to be started on a node. If OCR
is on ASM, it can’t be read until ASM (which itself is a resource for the node and this information
is stored in OCR) is up. To resolve this problem, information about the resources which need to
be started on a node is stored in an operating system file which is called Oracle Local Registry
or OLR. Since OLR is an operating system file, it can be accessed by various processes on the
node for read/write irrespective of the status of the clusterware (up/down). Hence, when a node
joins the cluster, OLR on that node is read, various resources, including ASM are started on the
node. Once ASM is up, OCR is accessible and is used henceforth to manage all the clusterware
resources. If OLR is missing or corrupted, clusterware can’t be started on that node.
GPnP Profile
The GPnP profile is a small XML file located in GRID_HOME/gpnp/<hostname>/profiles/peer
under the name profile.xml. It is used to establish the correct global personality of a node.
Each node maintains a local copy of the GPnP Profile and is maintained by the GPnP Daemon
(GPnPD).
GPnP Profile is used to store necessary information required for the startup of Oracle
Clusterware like SPFILE location, ASM Disk String etc. It contains various attributes defining
node personality like:
Cluster name
Network classifications (Public/Private)
Storage to be used for CSS
Storage to be used for ASM : SPFILE location, ASM Disk String etc
Digital signature information
Profile is updated whenever changes are made to a cluster with configuration tools like
− oifcfg (Change network)
− crsctl (change location of voting disk)
− asmcmd (change ASM_DISKSTRING, SPfile location) etc.
GPnPd daemon replicates changes to the profile during installation, system boot and when
updated.
Next CRSD needs to read OCR to startup various resources on the node. Since OCR is also on
ASM, location of ASM SPfile should be known.
In cluster environment, the location of SPfile for ASM is read from GPnP profile. Using the
parameters in SPfile, ASM is started. Once ASM is up, OCR is read by CRSD and various
resources on the node are started. Each node reads network information in GPnP profile and
using GNS, negotiates appropriate network identity for itself. Hence, nodes can be dynamically
added/deleted.
least two network interface cards: one for a public network and one for a private network. The
interconnect network is a private network using a
switch (or multiple switches) that only the nodes in the cluster can access.
You can obtain the details of cluster interconnect by using the following query.
Oracle 10g introduced the concept of Virtual IP. With Oracle 10g RAC, each node requires a
VIP. Starting with Oracle RAC 11g Release 2, three (3) additional SCAN VIPs are required for
the cluster. When a node fails, the VIP associated with it is automatically failed over to some
other node.
Virtual IP (VIP) addresses are all about availability of applications when an entire node fails.
When a node fails, the VIP address associated with it automatically fails over to some other
node in the cluster. When this occurs:
The new node indicates the new Media Access Control (MAC) address for the VIP. For
directly connected clients, this usually causes them to see errors on their connections to
the old address.
Subsequent packets sent to the VIP address go to the new node, which will send error
reset (RST) packets back to the clients. This results in the clients getting errors
immediately
The VIPs should be registered in the DNS. The VIP addresses must be on the same subnet as
the public host network addresses.
If you use GNS, and you have DHCP support, then the GNS will assign addresses dynamically
to the SCAN. If you do not use GNS, the SCAN should be defined in the DNS to resolve to the
three addresses assigned to that name. This should be done before you install Oracle Grid
Infrastructure.
The SCAN and its associated IP addresses provide a stable name for clients to use for
connections, independent of the nodes that make up the cluster.
SCANs function like a cluster alias. However, SCANs are resolved on any node in the cluster,
so unlike a VIP address for a node, clients connecting to the SCAN no longer require updated
VIP addresses as nodes are added to or removed from the cluster. Because the SCAN
addresses resolve to the cluster, rather than to a node address in the cluster, nodes can be
added to or removed from the cluster without affecting the SCAN address configuration.
During installation, listeners are created on each node for the SCAN IP addresses. Oracle
Clusterware routes application requests to the cluster SCAN to the least loaded instance
providing the service.
SCAN listeners can run on any node in the cluster. SCANs provide location independence for
the databases so that client configuration does not have to depend on which nodes run a
particular database.
Oracle Database 11g Release 2 and later instances register with SCAN listeners only as remote
listeners. Upgraded databases register with SCAN listeners as remote listeners, and also
continue to register with all other listeners. If you specify a GNS domain during installation, the
SCAN defaults to clustername-scan.GNS_domain. If a GNS domain is not specified at
installation, the SCAN defaults to clustername-scan.current_domain.
During installation, listeners are created on nodes for the SCAN IP addresses. Oracle Net
Services routes application requests to the least loaded instance providing the service. Because
the SCAN addresses resolve to the cluster, rather than to a node address in the cluster, nodes
can be added to or removed from the cluster without affecting the SCAN address configuration
The following diagram shows how SCAN and Node listeners (local listeners) are used in RAC.
The following diagram shows Oracle RAC cluster and various components of RAC cluster
ASM Concepts
This chapter describes Oracle Automatic Storage Management (Oracle ASM) concepts and
provides an overview of Oracle ASM features.
Oracle ASM uses disk groups to store data files; an Oracle ASM disk group is a collection of
disks that Oracle ASM manages as a unit. Within a disk group, Oracle ASM exposes a file
system interface for Oracle database files. The content of files that are stored in a disk group is
evenly distributed to eliminate hot spots and to provide uniform performance across the disks.
The performance is comparable to the
performance of raw devices.
You can add or remove disks from a disk group while a database continues to access files from
the disk group. When you add or remove disks from a disk group, Oracle ASM automatically
redistributes the file contents and eliminates the need for downtime when redistributing the
content.
The Oracle ASM volume manager functionality provides flexible server-based mirroring options.
The Oracle ASM normal and high redundancy disk groups enable two-way and three-way
mirroring respectively. You can use external redundancy to enable a Redundant Array of
Independent Disks (RAID) storage subsystem to perform the mirroring protection function.
Oracle ASM also uses the Oracle Managed Files (OMF) feature to simplify database file
management. OMF automatically creates files in designated locations. OMF also names files
and removes them while relinquishing space when tablespaces or files are deleted.
Oracle ASM reduces the administrative overhead for managing database storage by
consolidating data storage into a small number of disk groups. The smaller number of disk
groups consolidates the storage for multiple databases and provides for improved I/O
performance.
Oracle ASM files can coexist with other storage management options such as raw disks and
third-party file systems. This capability simplifies the integration of Oracle ASM into pre-existing
environments.
Oracle Enterprise Manager includes a wizard that enables you to migrate non-Oracle ASM
database files to Oracle ASM. Oracle ASM also has easy to use management interfaces such
as SQL*Plus, the Oracle ASM Command Line Utility (ASMCMD) command-line interface,
Oracle ASM Configuration Assistant, and Oracle Enterprise Manager.
Oracle ASM is installed in the Oracle Grid Infrastructure home before Oracle Database is
installed in a separate Oracle home. Oracle ASM and database instances require shared
access to the disks in a disk group. Oracle ASM instances manage the metadata of the disk
group and provide file layout information to the database instances.
Oracle ASM metadata is the information that Oracle ASM uses to control a disk group and the
metadata resides within the disk group. Oracle ASM metadata includes the following
information:
The disks that belong to a disk group
The amount of space that is available in a disk group
The filenames of the files in a disk group
The location of disk group data file extents
A redo log that records information about atomically changing metadata blocks
Oracle ADVM volume information
Oracle ASM instances can be clustered using Oracle Clusterware; there is one Oracle ASM
instance for each cluster node. If there are several database instances for different databases
on the same node, then the database instances share the same single Oracle ASM instance on
that node.
If the Oracle ASM instance on a node fails, then all of the database instances on that node also
fail. Unlike a file system driver failure, an Oracle ASM instance failure does not require restarting
the operating system. In an Oracle RAC environment, the Oracle ASM and database instances
on the surviving nodes automatically recover from an Oracle ASM instance failure on a node.
The following figure shows a single node configuration with one Oracle ASM instance and
multiple database instances. The Oracle ASM instance manages the metadata and provides
space allocation for the Oracle ASM files. When a database instance creates or opens an
Oracle ASM file, it communicates those requests to the Oracle ASM instance. In response, the
Oracle ASM instance provides file extent map information to the database instance.
The following figure shows an Oracle ASM cluster in an Oracle RAC environment where Oracle
ASM provides a clustered pool of storage. There is one Oracle ASM instance for each node
serving multiple Oracle RAC or single-instance databases in the cluster. All of the databases
are consolidated and share the same two Oracle ASM disk groups.
To share a disk group among multiple nodes, you must install Oracle Clusterware on all of the
nodes, regardless of whether you install Oracle RAC on the nodes. Oracle ASM instances that
are on separate nodes do not need to be part of an Oracle ASM cluster. However, if the Oracle
ASM instances are not part of an Oracle ASM cluster, they cannot communicate with each
other. Multiple nodes that are not part of an Oracle ASM cluster cannot share a disk group.
Files are allocated from disk groups. Any Oracle ASM file is completely contained within a single
disk group. However, a disk group might contain files belonging to several databases and a
single database can use files from multiple disk groups. For most installations you need only a
small number of disk groups, usually two, and rarely more than three.
You can store the following file types in ASM disk groups:
Control files
Datafiles, temporary datafiles, and datafile copies
SPFILEs
Online redo logs, archive logs, and Flashback logs
RMAN backups
Disaster recovery configurations
Change tracking bitmaps
Data Pump dumpsets
Note: Oracle executables and ASCII files, such as alert logs and trace files, cannot be
stored in ASM disk groups.
The redundancy level controls how many disk failures are tolerated without dismounting the disk
group or losing data. The disk group type determines the mirroring levels with which Oracle
creates files in a disk group.
Oracle ASM mirroring is more flexible than traditional RAID mirroring. For a disk group specified
as NORMAL redundancy, you can specify the redundancy level for each file. For example, two
files can share the same disk group with one file being mirrored while the other is not.
A Failure Group is a subset of the disks in a disk group. Failure groups define disks that share
components, such that if one fails then other disks sharing the component might also fail. An
example of what you might define as a failure group would be a set of SCSI disks sharing the
same SCSI controller. Failure groups are used to determine which Oracle ASM disks to use for
storing redundant data. For example, if two-way mirroring is specified for a file, then redundant
l.
ai
gm
copies of file extents must be stored in separate failure groups. By default, Oracle ASM puts
each voting disk in its own failure group within the disk group.
7@
71
A normal redundancy disk group must contain at least two failure groups but if you are storing
n2
your voting disks on Oracle ASM, then a normal redundancy disk group must contain at least
three failure groups. A high redundancy disk group must contain at least three failure groups.
wa
However, Oracle recommends using several failure groups. A small number of failure groups, or
ha
failure groups of uneven capacity, can create allocation problems that prevent full use of all of
id
When Oracle ASM allocates an extent for a mirrored file, Oracle ASM allocates a primary copy
Pr
and a mirror copy. Oracle ASM chooses the disk on which to store the mirror copy in a different
failure group than the primary copy. Failure groups are used to place mirrored copies of data so
that each copy is on a disk in a different failure group. The simultaneous failure of all disks in a
failure group does not result in data loss.
You define the failure groups for a disk group when you create an Oracle ASM disk group. After
a disk group is created, you cannot alter the redundancy level of the disk group. If you omit the
failure group specification, then Oracle ASM automatically places each disk into its own failure
group. Normal redundancy disk groups require at least two failure groups. High redundancy disk
groups require at least three failure groups. Disk groups with external redundancy do not use
failure groups.
The disks are first taken offline and then automatically dropped. In this case, the disk
group remains mounted and serviceable. In addition, because of mirroring, all of the disk
group data remains accessible. After the disk drop operation, Oracle ASM performs a
rebalance to restore full redundancy for the data on the failed disks.
The entire disk group is automatically dismounted, which means loss of data
accessibility
Each disk in a disk group can belong to only one failure group.
Failure groups should all be of the same size. Failure groups of different sizes may lead
to reduced availability.
Oracle ASM requires at least two failure groups to create a normal redundancy disk
group and at least three failure groups to create a high redundancy disk group.
When you add a disk to a disk group, you can assign an Oracle ASM disk name or Oracle ASM
assigns the Oracle ASM disk name automatically. This name is different from the path name
used by the operating system. In a cluster, a disk may be assigned different operating system
device names on different nodes, but the disk has the same Oracle ASM disk name on all of the
nodes. In a cluster, an Oracle ASM disk must be accessible from all of the instances that share
the disk group.
Oracle ASM spreads the files proportionally across all of the disks in the disk group. This
allocation pattern maintains every disk at the same capacity level and ensures that all of the
disks in a disk group have the same I/O load. Because Oracle ASM load balances among all of
the disks in a disk group, different Oracle ASM disks should not share the same physical drive.
Allocation Units
Every Oracle ASM disk is divided into allocation units (AU). An allocation unit is the fundamental
unit of allocation within a disk group. A file extent consists of one or more allocation units. An
Oracle ASM file consists of one or more file extents.
When you create a disk group, you can set the Oracle ASM allocation unit size with the
AU_SIZE disk group attribute. The values can be 1, 2, 4, 8, 16, 32, or 64 MB, depending on the
specific disk group compatibility level. Larger AU sizes typically provide performance
advantages for data warehouse applications that use large sequential reads.
Control files
Data files, temporary data files, and data file copies
SPFILEs
Online redo logs, archive logs, and Flashback logs
RMAN backups
Disaster recovery configurations
Change tracking bitmaps
Data Pump dumpsets
Oracle ASM automatically generates Oracle ASM file names as part of file creation and
tablespace creation. Oracle ASM file names begin with a plus sign (+) followed by a disk group
name. You can specify user-friendly aliases for Oracle ASM files and create a hierarchical
directory structure for the aliases.
Extents
The contents of Oracle ASM files are stored in a disk group as a set, or collection, of extents
that are stored on individual disks within disk groups. Each extent resides on an individual disk.
Extents consist of one or more allocation units (AU). To accommodate increasingly larger files,
Oracle ASM uses variable size extents.
Variable size extents enable support for larger Oracle ASM data files, reduce SGA memory
requirements for very large databases, and improve performance for file create and open
operations. The initial extent size equals the disk group allocation unit size and it increases by a
factor of 4 or 16 at predefined thresholds. This feature is automatic for newly created and
resized data files when specific disk group compatibility attributes are set to 11.1 or higher.
You can explicitly dismount a disk group. Oracle reports an error if you attempt to dismount a
disk group without the force option when any of the disk group files are open. It is possible to
have disks fail in excess of the Oracle ASM redundancy setting. If this happens, then the disk
group is forcibly dismounted. If the disk group is forcibly dismounted, a database cannot access
files in the disk group.
You can drop a disk from a disk group if it fails or to re-purpose capacity. Use the Oracle ASM
disk name to drop a disk, not the discovery string device name. If an error occurs while writing
to a disk, then Oracle ASM drops the disk automatically.
Rebalancing is performed on all database files on a per file basis. The ASM background
process, RBAL, responsible for rebalancing activity.
The Rebalance process examines each file extent map, and the new extents are re-plotted
on to the new configuration. For example, consider an 8-disk diskgroup, with a datafile with 40
extents (each disk will house 5 extents). When 2 new drives (of same size) are added, that
datafile is rebalanced and spread across 10 drives, with each drive containing 4 extents. Only 8
extents need to move to complete the rebalance.
Rebalancing a disk group moves data between disks to ensure that every file is evenly spread
across all of the disks in a disk group. When all of the files are evenly dispersed, all of the disks
are evenly filled to the same percentage; this ensures load balancing. Rebalancing does not
relocate data based on I/O statistics nor is rebalancing started based on I/O statistics. Oracle
ASM rebalancing operations are controlled by the size of the disks in a disk group.
Rebalancing involves physical movement of file extents. This impact is generally low because
the rebalance is done one extent/AU at a time; therefore, there will be only be one outstanding
I/O at any given time. This should not adversely affect online database activity. However, it is
generally advisable to schedule the rebalance operation during off-peak hours.
The init.ora parameter, asm_power_limit, influences the throughput and speed of the
rebalance operation. The range of values for asm_power_limit are 1 to 11; where a value of 11
is full throttle and a value of 1 is low speed. However, the power value can also be set for a
specific rebalance activity using the alter diskgroup command. This value is only effective for
the specific rebalance task. A power value of 0 indicates that no rebalance should occur
for this rebalance. This setting is particularly important to set when adding or removing
storage (that has redundancy), and then deferring the rebalance to a later time.
Oracle ASM automatically initiates a rebalance after storage configuration changes, such as
when you add, drop, or resize disks. You can manually start a rebalance to change the power
setting of a running rebalance. A rebalance is automatically restarted if the instance on which
the rebalancing is running stops. Databases can remain operational during rebalancing
operations.
Voting files manage information about node membership. OCR is a system that manages
cluster and Oracle Real Application Clusters (Oracle RAC) database configuration information.
For Oracle Clusterware files a minimum of three disk devices or three failure groups is required
with a normal redundancy disk group.
When a database instance creates, opens, or deletes an Oracle ASM file, it communicates
those requests to the Oracle ASM instance. In response, the Oracle ASM instance provides file
extent map information to the database instance. Database instances access contents of ASM
files directly, communicating with an ASM instance only to get information about the layout of
these files. Oracle Database communicates with Oracle ASM in terms of files. ASM and
Database Instances also interact if the storage configuration changes, i.e. when disks are
added, dropped or fail.
The file creation process provides an illustration of the interactions that take place between
Database Instances and ASM. The file creation process occurs as follows:
1. The database requests file creation.
2. An ASM foreground process creates a Continuing Operation Directory (COD) entry and
allocates space for the new file across the disk group.
3. A background database process (ASMB) receives an extent map for the new file.
4. The file is now open and the database process initializes the file directly.
5. After initialization, the database process requests that the file creation be committed.
This causes the ASM foreground process to clear the COD entry and mark the file as
created.
6. Acknowledgement of the file commit implicitly closes the file. The database instance will
need to reopen the file for future I/O.
All ASM management commands are directed through the ASM instance, no ASM commands
are issued to any regular Oracle database instance that is using ASM files.
The number of Hub Nodes in an Oracle Flex Cluster can be as many as 64. The number of Leaf
Nodes can be many more. Hub Nodes and Leaf Nodes can host different types of applications.
Hub Nodes are similar to Oracle Grid Infrastructure nodes in an Oracle Clusterware standard
Cluster configuration: they are tightly connected among themselves via private network and
have direct access to shared storage just like in previous versions of Oracle. These nodes are
the ones that access the Oracle Cluster Registry (OCR) and Voting Disk (VD) directly.
Leaf Nodes are different from standard Oracle Grid Infrastructure nodes, in that they do not
require direct access to shared storage, but instead request data through Hub Nodes.
These nodes are lighter and are not connected among them; neither accesses the shared
storage like the Hub Nodes. Each Leaf Node communicates with the Hub Node that is attached
to, and its connected to the cluster via the Hub Node that is linked to.
Hub Nodes can run in an Oracle Flex Cluster configuration without having any Leaf Nodes as
cluster member nodes, but Leaf Nodes must be members of a cluster that includes at least one
Hub Node.
This topology allows loosely coupled application servers to form a cluster with tightly coupled
database servers. Tightly coupled servers are Hub Servers that share storage for database,
OCR and Voting devices as well as peer-to-peer communication with other Hub Servers in the
cluster. A loosely coupled server is a Leaf Server that has a loose communication association
with a single Hub Server in the cluster and does not require shared storage nor peer-to-peer
communication with other Hub or Leaf Servers in the cluster, except to communicate with the
Hub to which it is associated.
Overall, Flex Clusters enhance the ability of Grid Infrastructure to manage the complete server
environment: application servers as well as database servers.
The following figure shows a typical Oracle Flex Cluster with four Leaf nodes and two Hub
nodes.
The following figure shows the ASM and Database instance association in Oracle versions prior
to Oracle 12c.
With the launch of Oracle 12c the above constraint has been addressed with the feature called
Oracle Flex ASM which primarily has a feature to fail over to another node in the cluster.
Essentially a Hub and Leaf architecture, the connection of a failed node is seamlessly
transferred to another participating node by way of a replacement ASM instance by Oracle
Clusterware. The number of ASM instances running in a given cluster is called ASM cardinality
with a default value of 3.
l.
ai
gm
The following figure shows ASM Instance Failure on Oracle Flex ASM configuration:
7@
71
n2
wa
ha
id
it
Pr
Starting with Oracle Grid Infrastructure 12c Release 1 (12.1), as part of an Oracle Flex Cluster
installation, Oracle ASM is configured within Oracle Grid Infrastructure to provide storage
services. Each Oracle Flex ASM cluster has its own name that is globally unique within the
enterprise.
In Flex ASM, ASM network provides the capability to isolate ASM’s internal network traffic to its
own dedicated private network. The OUI presents the DBA with the choice as to whether a
dedicated network is to be used for ASM. The ASM network is the communication path in which
all the traffic between database instances and ASM instances commence. This traffic is mostly
the metadata such as a particular file’s extent map. If the customer chooses, the ASM private
network can be dedicated for ASM traffic or shared with CSS and a dedicated network is not
required.
There will be a local ASM listener that runs on each node where the flex ASM instance is
running. ASM clients connect to Flex ASM over the ASM network by using highly available
connection endpoints supplied by up to three ASM listeners. Connections are load balanced
across the ASM cluster.
Oracle Flex ASM can use either the same private networks as Oracle Clusterware, or use its
own dedicated private networks. Each network can be classified PUBLIC, PRIVATE, ASM or
ASM & PRIVATE.
Database clients can connect to either IPv4 or IPv6 addresses. The Single Client Access Name
(SCAN) listener automatically redirects client connection requests to the appropriate database
listener for the IP protocol of the client request.
When you are using Oracle RAC, all of the clients must be able to reach the database. This
means that the VIP addresses must be resolved by the clients. This problem is solved by the
addition of the Oracle Grid Naming Service (GNS) to the cluster. GNS is linked to the corporate
domain name service (DNS) so that clients can easily connect to the cluster and the databases
running there. Activating GNS in a cluster requires a DHCP service on the public network.
To implement GNS, you must collaborate with your network administrator to obtain an IP
address on the public network for the GNS VIP. DNS uses the GNS VIP to forward requests for
access to the cluster to GNS. The network administrator must delegate a subdomain in the
network to the cluster. The subdomain forwards all requests for addresses in the subdomain to
the GNS VIP. GNS and the GNS VIP run on one node in the cluster. The GNS daemon listens
on the GNS VIP using port 53 for DNS requests. Oracle Clusterware manages the GNS and
the GNS VIP to ensure that they are always available. If the server on which GNS is running
fails, then Oracle Clusterware fails GNS over, along with the GNS VIP, to another node in the
cluster.
With DHCP on the network, Oracle Clusterware obtains an IP address from the server along
with other network information, such as what gateway to use, what DNS servers to use, what
domain to use, and what NTP server to use. Oracle Clusterware initially obtains the necessary
IP addresses during cluster configuration and it updates the Oracle Clusterware resources with
the correct information obtained from the DHCP server.
For small clusters, you can use a static configuration of IP addresses. For large clusters,
manually maintaining the large number of required IP addresses becomes too cumbersome.
The Oracle Grid Naming Service is used with large clusters to ease administration costs.
In previous releases (up to release 11gR2) , the Grid Naming Service (GNS) was dedicated to
one Oracle Grid Infrastructure-based cluster, providing name resolution only for its own cluster
member nodes.
With 12c release, one Oracle GNS can now manage just the cluster member nodes in its own
cluster, or GNS can provide naming resolution for all nodes across all clusters in the data center
that are delegated to Oracle GNS for resolution.
Using only one Oracle GNS for all nodes that are part of an Oracle Grid Infrastructure cluster in
the data center not only streamlines the naming convention, but also enables a data center
cloud, minimizing day-to-day administration efforts.
An individual password file for Oracle Database or Oracle ASM can reside on a designated
Oracle ASM disk group. Having the password files reside on a single location accessible across
the cluster reduces maintenance costs and situations where passwords become out of sync.
You can use a password file located on a disk group for authentication only if the Oracle ASM
instance is running and the designated disk group is mounted. Otherwise, operating system
authentication must be used to bootstrap the startup of the Oracle ASM instance and stack.
The COMPATIBLE.ASM disk group attribute must be set to at least 12.1 for the disk group
where the password file is to be located. The SYSASM privilege is required to manage the
Oracle ASM password file.
The shared password file in a disk group is managed by ASMCMD commands, the ORAPWD
tool, and SRVCTL commands. ORAPWD supports the creation of password files on an Oracle
ASM disk group. All other password file manipulation is performed with ASMCMD or SRVCTL
commands.
You must specify the disk group location and database unique name when using orapwd to
create a database password file on a disk group.
This chapter has the steps to install Oracle 12c Release 1 (12.1.0.2.0) GI & RAC software on
Oracle Enterprise Linux 6.x (OEL 6.x) 64-bit platform. This guide assumes that you are using
Oracle ASM for shared storage.
In general you need to perform the following tasks before installing Oracle Grid Infrastructure
and Oracle Software for RAC.
make-3.81-19.el6
sysstat-9.0.4-11.el6 (x86_64)
If the current value of any parameter is higher than the value listed in this table, then do not
change the value of that parameter.
To view the current value specified for these kernel parameters, enter commands similar to the
following
For example:
If the value of any kernel parameter is different from the recommended value, then using vi
editor, create or edit the /etc/sysctl.conf file, and add or edit lines similar to the following:
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
After updating the values of kernel parameters in the /etc/sysctl.conf file, run the command
sysctl -p ( or restart the computer ) to make the changes in the /etc/sysctl.conf file available in
the active kernel memory.
# /sbin/sysctl -p
Oracle Cluster Time Synchronization Service is designed for organizations whose cluster
servers are unable to access NTP services. If you use NTP, then the Oracle Cluster Time
Synchronization daemon (ctssd) starts up in observer mode. If you do not have NTP daemons,
then ctssd starts up in active mode and synchronizes time among cluster members without
contacting an external time server
Note: Before starting the installation of Oracle Grid Infrastructure, Oracle recommends that you
ensure the clocks on all nodes are set tothe same time.
If you have NTP daemons on your server but you cannot configure them to synchronize time
with a time server, and you want to use Cluster Time Synchronization Service to provide
synchronization service in the cluster, then deactivate and deinstall the NTP.
To deactivate the NTP service, you must stop the existing ntpd service, disable it from
the initialization sequences and remove the ntp.conf file. To complete these step on
Oracle Linux, and Asianux systems, run the following commands as the root user
When the installer finds that the NTP protocol is not active, the Cluster Time Synchronization
Service is installed in active mode and synchronizes the time across the nodes. If NTP is found
configured, then the Cluster Time Synchronization Service is started in observer mode, and no
active time synchronization is performed by Oracle Clusterware within the cluster.
To confirm that ctssd is active after installation, enter the following command as the Grid
installation owner:
At least 4 GB of RAM for Oracle Grid Infrastructure for a Cluster installations, including
installations where you plan to install Oracle RAC
An amount of swap space that is at least equal to the amount of RAM
Temporary space (at least 1 GB) available in /tmp
Adequate local disk space to install Oracle software:
− At least 8 GB of available disk space for the Grid home directory, which includes
both the binary files for Oracle Clusterware and Oracle Automatic Storage
Management
− Additional disk space (100 GB is recommended) for the Grid home directory for
associated log files and patches.
− At least 3.5 GB of space for the Oracle base of the Oracle Grid Infrastructure
installation owner (Grid user). The Oracle base includes Oracle Clusterware and
Oracle ASM log files.
− At least 5.8 GB of disk space for the Oracle home (the location for the Oracle
Database software binaries).
These Oracle Clusterware components require the following disk space on a shared file system:
Three Oracle Clusterware Registry (OCR) files, a least 400 MB for each, or 1.2 GB total disk
space
Three voting disk files, 300 MB each, or 900 MB total disk space
Voting disk–Manages cluster membership and arbitrates cluster ownership between the nodes
in case of network failures. The voting disk is a file that resides on shared storage. For high
availability, Oracle recommends that you have multiple voting disks, and that you have an odd
number of voting disks. If you define a single voting disk, then use mirroring at the file system
level for redundancy.
Hardware Requirements
A processor type (CPU) that is certified with the release of the Oracle software being
installed
A minimum of 1024 x 786 display resolution, so that Oracle Universal Installer (OUI)
displays correctly
All servers in the cluster have the same chip architecture, for example, all 64-bit processors
To determine if the system architecture can run the software, enter the following command:
# uname -m
For example, you should see the following for a x86-64 bit system:
x86_64
Network Requirements
Each node must have at least two network adapters or network interface cards (NICs): one
for the public network interface, and one for the private network interface (the interconnect).
Interface for Oracle Flex ASM Network
Oracle Flex ASM can use either the same private networks as Oracle Clusterware,
or use its own dedicated private networks. Each network can be classified PUBLIC,
ASM & PRIVATE, PRIVATE, or ASM.
A public IP address for each node
A virtual IP address for each node
A private IP address for each node
A Single Client Access Name (SCAN) for the cluster with three static IP addresses
configured on the domain name server (DNS) or one static IP address configured in
/etc/hosts file.
You must use Grid Naming Service (GNS) with an Oracle Flex Cluster Deployment
You must configure the GNS VIP as a static IP address for Hub Nodes.
Public IP addresses, VIP addresses, and SCAN addresses should be on the same subnet as all
other public IP addresses. Private IP addresses should be configured on a separate, private
network, with its own subnet, that is not resolvable except by other cluster member nodes.
When performing an advanced installation of Oracle Grid Infrastructure for a cluster, you can
choose to use Grid Naming Service (GNS) and Dynamic Host Configuration Protocol
(DHCP) for virtual IPs (VIPs). GNS uses multicast Domain Name Server (mDNS) to enable
the cluster to assign host names and IP addresses dynamically as nodes are added and
removed from the cluster, without requiring additional network address configuration in the
domain name server (DNS).
In this course you will perform a typical installation, which does not use GNS.
inventory_loc=central_inventory_location
inst_group=group
For Oracle Grid Infrastructure installations, the central inventory must be on local storage on the
node. If you have an existing Oracle central inventory, then ensure that you use the same
Oracle Inventory for all Oracle software installations, and ensure that all Oracle software users
you intend to use for installation have permissions to write to this directory.
If the oraInst.loc file does not exist, then create the Oracle Inventory group by entering a
command similar to the following:
Note: Group and user IDs must be identical on all nodes in the cluster. Check to make sure
that the group and user IDs you want to use are available on each cluster member node, and
confirm that the primary group for each Oracle Grid Infrastructure for a cluster installation owner
has the same name and group ID.
Oracle software owners (grid & oracle) must have the Oracle Inventory group (oinstall) as their
primary group, so that each Oracle software installation owner can write to the central inventory
(oraInventory), and so that OCR and Oracle Clusterware resource permissions are set correctly.
The database software owner must also have the OSDBA group (dba) as secondary group.
With Oracle Database job role separation, each Oracle Database installation has separate
operating system groups to provide authentication for system privileges on that Oracle
Database, so multiple databases can be installed on the cluster without sharing operating
system authentication for system privileges.
Job role separation configuration enables database security, and does not restrict user roles in
starting and stopping various Clusterware resources.
With Oracle Grid Infrastructure job role separation, Oracle ASM has separate operating system
groups that provides operating system authentication for Oracle ASM system privileges for
storage tier administration. This operating system authentication is separated from Oracle
Database operating system authentication. In addition, the Oracle Grid Infrastructure installation
owner provides operating system user authentication for modifications to Oracle Grid
Infrastructure binaries.
During the Oracle Database installation, Oracle Universal Installer (OUI) prompts you to specify
the name of the OSDBA, OSOPER, OSBACKUPDBA, OSDGDBA and OSKMDBA groups.
Members of these groups are granted operating system authentication for the set of database
system privileges each group authorizes
Note: Job Role Separation configuration is optional, to restrict user access to Oracle software
by responsibility areas for different administrator users. You can choose to create one
administrative user and one group for operating system authentication for all system privileges
on the storage and database tiers.
If you do not want to use role allocation groups, then Oracle strongly recommends that you use
at least two groups:
A system privileges group whose members are granted administrative system privileges,
including OSDBA, OSASM, and other system privileges groups.
An installation owner group (the oraInventory group) whose members are granted
Oracle installation owner system privileges (the OINSTALL system privilege).
Starting with Oracle Automatic Storage Management 11g release 2(11.2), members of the
database OSDBA group are not granted SYSASM privileges, unless the operating system
group designated as the OSASM group is the same group designated as the OSDBA group.
This privileges grant is removed with Oracle Grid Infrastructure 11g Release 2 (11.2), if different
operating system groups are designated as the OSDBA and OSASM groups.
Members of the OSASM group can use SQL to connect to an Oracle ASM instance as
SYSASM using operating system authentication. The SYSASM privileges permit mounting and
dismounting disk groups, and other storage administration tasks. SYSASM privileges provide no
access privileges on an RDBMS instance.
OSDBA for ASM Database Administrator group for ASM (typically asmdba):
Members of the ASM Database Administrator group (OSDBA for ASM) are granted read and
write access to files managed by Oracle ASM. The Oracle Grid Infrastructure installation owner
and all Oracle Database software owners must be a member of this group, and all users with
OSDBA membership on databases that have access to the files managed by Oracle ASM must
be members of the OSDBA group for ASM.
The following section, shows the commands to create various groups and users for Oracle Grid
Infrastructure and Oracle RAC.
Creating Groups:
l.
ai
# groupadd –g 1101 dba
gm
# groupadd –g 1102 asmdba
7@
# groupadd –g 1103 asmoper
71
# groupadd –g 1104 asmadmin
n2
Create the above mentioned groups and users on each node of the cluster.
Pr
Resource Limits
Also, before you start the installation, for each installation software owner, check the resource
limits for installation, using the following recommended ranges
2. Check the soft and hard limits for the file descriptor setting. Ensure that the result is in the
recommended range. For example:
$ ulimit -Sn
4096
$ ulimit -Hn
65536
3. Check the soft and hard limits for the number of processes available to a user. Ensure
that the result is in the recommended range. For example:
$ ulimit -Su
2047
$ ulimit -Hu
16384
4. Check the soft limit for the stack setting. Ensure that the result is in the recommended
range. For example:
$ ulimit -Ss
10240
$ ulimit -Hs
32768
If necessary, update the resource limits in the /etc/security/limits.conf configuration file for the
installation owner. For example
3. Configuring SSH
SSH is required for Oracle Grid Infrastructure installation. OpenSSH should be included in the
Linux distribution minimal installation.
To install Oracle software, Secure Shell (SSH) connectivity must be set up between all cluster
member nodes. OUI uses the ssh and scp commands during installation to run remote
commands on and copy files to the other cluster nodes. You must configure SSH so that these
commands do not prompt for a password.
You can configure SSH from the Oracle Universal Installer (OUI) interface during installation for
the user account running the installation. The automatic configuration creates passwordless
SSH connectivity between all cluster member nodes. Oracle recommends that you use the
automatic procedure if possible.
If automatic ssh configuration is not possible, you can manually configure user equivalence as
described below. The following steps show how to configure SSH between 2 nodes named
node1 and node2.
Note: If you are using separate operating system accounts for installing Grid Infrastructure and
Oracle RAC software, then you need to repeat the following steps for each linux account.
Node1:
Node2:
Node1:
Test Connectivity
check all ssh combinations as mentioned below
on node1:
$ssh node1
$ssh node2
on node2:
$ssh node1
$ssh node2
Make sure the above tests do not prompt for password. If any of the above tests prompt
for password, then the SSH is not properly configured. In that case you need to fix the
issue and make sure SSH is working correctly.
For Oracle Grid Infrastructure for a cluster installations, note the following restrictions for the
Oracle Grid Infrastructure binary home (Grid home directory for Oracle Grid Infrastructure):
It must not be placed under one of the Oracle Base directories, including the Oracle
base directory of the Oracle Grid Infrastructure installation owner.
It must not be placed in the home directory of an installation owner.
Placing Oracle Grid Infrastructure for a cluster binaries on a cluster file system is not
supported.
Note: On Linux and UNIX systems, you must ensure the Grid home directory is not a
subdirectory of the Oracle base directory. Installing Oracle Clusterware in an Oracle base
directory causes installation
errors.
Oracle recommends that you create Oracle Grid Infrastructure Grid home and Oracle base
homes manually, particularly if you have separate Oracle Grid Infrastructure for a cluster and
Oracle Database software owners, so that you can separate log files for the Oracle Grid
Infrastructure installation owner in a separate Oracle base, and prevent accidental placement of
the Grid home under an Oracle base path.
Use commands similar to the following to create Oracle Home and Oracle Base directories for
Oracle Grid Infrastructure and Oracle Software for RAC.
For example, a voting disk on a device named /dev/sdd owned by the user grid may be on a
device named /dev/sdf owned by root after restarting the server.
If you use ASMLIB, then you do not have to ensure permissions and device path persistency in
udev. If you do not use ASMLIB, then you must create a custom rules file for the shared disks
mounted on each node.
If you use ASMLIB to configure the shared disks, then when you restart the node:
The disk device names do not change
The ownership and group membership for these disk devices remains the same
The following sections describe how to install and configure ASMLIB, and how to use ASMLIB
to configure your shared disk devices:
Installing ASMLIB
The ASMLIB software for various Linux platforms is available from the Oracle Technology
Network. Select the link for your platform on the ASMLIB download page at:
http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html
You should see four to six packages for your Linux platform. The oracleasmlib package
provides the actual Oracle ASM library. The oracleasm-support package provides the utilities
used to get the Oracle ASM driver up and running. Both of these packages must be installed.
The remaining packages provide the kernel driver for the Oracle ASM library. Each package
provides the driver for a different kernel. You must install the appropriate package for the kernel
you run.
Download and install the ASMLIB packages on each node in your cluster.
For Example:
Configuring ASMLIB
After the ASMLIB software is installed, a few steps have to be taken by you to make the Oracle
ASM driver available. The Oracle ASM driver must be loaded, and the driver file system must be
mounted. This is taken care of by the initialization script, /usr/sbin/oracleasm.
# /usr/sbin/oracleasm configure -i
The script prompts you for the default user and group to own the Oracle ASM driver access
point. Specify the Oracle GI software owner (grid) and the OSDBA group (oinstall).
The script also prompts you to specify whether you want to start the ASMLIB driver when the
node is started and whether you want to scan for presence of any Oracle Automatic Storage
Management disks when the node is started. Answer yes for both of these questions.
If the target disk device supports partitioning, then you must first create a single partition that
encompasses the entire disk.
If the target disk device does not support partitioning, then you do not have to create a partition
on the disk.
In this command, disk_name is the name you choose for the Oracle ASM disk. The name you
choose must contain only ASCII capital letters, numbers, or underscores, and the disk name
must start with a letter, for example, DISK1 or VOL1, or RAC_FILE1. The name of the disk
partition to mark as an Oracle ASM disk is the device_partition_name. For example:
If you must unmark a disk that was used in a createdisk command, then you can use the
following syntax:
3. After you have created all the Oracle ASM disks for your cluster, use the listdisks command
to verify their availability:
# /usr/sbin/oracleasm listdisks
DISK1
DISK2
DISK3
4. On all the other nodes in the cluster, use the scandisks command to view the newly created
Oracle ASM disks. You do not have to create the Oracle ASM disks on each node, only on
one node in the cluster.
# /usr/sbin/oracleasm scandisks
5. After scanning for Oracle ASM disks, display the available Oracle ASM disks on each node
to verify their availability:
# /usr/sbin/oracleasm listdisks
DISK1
DISK2
DISK3
Note: At this point, you should restart each node on which you are installing Oracle Grid
Infrastructure for a cluster.
Oracle Real Application Clusters (Oracle RAC) uses Oracle Clusterware as the infrastructure
that binds together multiple nodes that then operate as a single server. In an Oracle RAC
environment, Oracle Clusterware monitors all Oracle components (such as instances and
listeners). If a failure occurs, Oracle Clusterware automatically attempts to restart the failed
component and also redirects operations to a surviving component.
Oracle Clusterware includes a high availability framework for managing any application that
runs on your cluster. Oracle Clusterware manages applications to ensure they start when the
system starts. Oracle Clusterware also monitors the applications to make sure that they are
always available. For example, if an application process fails, then Oracle Clusterware attempts
to restart the process based on scripts that you customize. If a node in the cluster fails, then you
can program application processes that typically run on the failed node to restart on another
node in the cluster.
Oracle Clusterware is managed using CRSCTL and SRVCTL command-line utilities installed
under the Oracle Grid Infrastructure home directory. However, most CRSCTL commands are
available only to Clusterware administrators, whereas SRVCTL commands are available to
other groups such as database administrators.
CRSCTL is used to manage Clusterware related operations like Starting and Stopping Oracle
Clusterware, Enabling and disabling Oracle Clusterware daemons, Registering Cluster
resources.
SRVCTL is used to manage Oracle resource related operations like Starting and Stopping
database instances and services.
Apart from CRSCTL & SRVCTL command-line utilities Oracle Enterprise Manager also provides
browser based graphical interface to manage Oracle Clusterware.
Oracle Clusterware consists of two separate stacks: an upper stack anchored by the Cluster
Ready Services (CRS) daemon (crsd) and a lower stack anchored by the Oracle High
Availability Services daemon (ohasd). These two stacks have several processes that facilitate
cluster operations.
Cluster Ready Services (CRS): The primary program for managing high availability
operations in a cluster.
The CRS daemon (crsd) manages cluster resources based on the configuration information that
is stored in OCR for each resource. This includes start, stop, monitor, and failover operations.
The crsd process generates events when the status of a resource changes. When you have
Oracle RAC installed, the crsd process monitors the Oracle database instance, listener, and so
on, and automatically restarts these components when a failure occurs.
Local resources: A local resource runs on every node in the cluster (no failover) and
can be, for example, a listener, ASM, a disk group, or Oracle Notification Service (ONS).
The cssdagent process monitors the cluster and provides I/O fencing. This service formerly was
provided by Oracle Process Monitor Daemon (oprocd), also known as OraFenceService on
Windows. A cssdagent failure may result in Oracle Clusterware restarting the node.
Event Management (EVM): A background process that publishes events that Oracle
Clusterware creates.
Oracle Notification Service (ONS): A publish and subscribe service for communicating
Fast Application Notification (FAN) events.
Oracle ASM: Provides disk management for Oracle Clusterware and Oracle Database.
Cluster Time Synchronization Service (CTSS): Provides time management in a cluster for
Oracle Clusterware.
Oracle Root Agent (orarootagent): A specialized oraagent process that helps crsd
manage resources owned by root, such as the network, and the Grid virtual IP address.
The Cluster Synchronization Service (CSS), Event Management (EVM), and Oracle Notification
Services (ONS) components communicate with other cluster component layers on other nodes
in the same cluster database environment. These components are also the main
communication links between Oracle database, applications, and the Oracle clusterware high
availability components. In addition, these background processes monitor and manage
database operations.
Cluster Logger Service (ologgerd): Receives information from all the nodes in the cluster
and persists in a CHM repository-based database. This service runs on only one node in a
cluster and an additional service will be started for every 32 nodes.
System Monitor Service (osysmond): The monitoring and operating system metric
collection service that sends the data to the cluster logger service. This service runs on
every node in a cluster.
Grid Plug and Play (GPNPD): Provides access to the Grid Plug and Play profile, and
coordinates updates to the profile among the nodes of the cluster to ensure that all of the
nodes have the most recent profile.
Grid Inter Process Communication (GIPC): A support daemon that enables Redundant
Interconnect Usage.
Multicast Domain Name Service (mDNS): Used by Grid Plug and Play to locate profiles in
the cluster, as well as by GNS to perform name resolution. The mDNS process is a
background process on Linux and UNIX and on Windows.
Oracle Grid Naming Service (GNS): Handles requests sent by external DNS servers,
performing name resolution for names defined by the cluster.
Cluster Kill Daemon (oclskd): handles instance/node eviction requests that have been
escalated to CSS.
Disk Monitory Daemon (diskmon): Monitors and performs input/output fencing for Oracle
Exadata Storage server. As Exadata storage can be added to any RAC node at any point in
time, the diskmon daemon is always started when OCSSD is started
Clusterware processes are organized into several component groups as mentioned in below
table:
The Oracle High Availability Services daemon (ohasd) is responsible for starting in proper order,
monitoring, and restarting other local Oracle daemons, up through the crsd daemon, which
manages cluster wide resources. When init starts ohasd on Clusterware startup, ohasd starts
orarootagent, oraagent and cssdagent. These processes then carry out the following tasks:
− crsd starts another orarootagent process responsible for root-owned CRS resources
including the SCAN VIPS.
• oraagent starts mdnsd, evmd, ASM, ctssd, gpnpd, gsd, ONS, and the listeners.
Some of the high availability daemons will be running under the root user with real-time priority,
and others will be running under the Clusterware owner with user-mode priorities after they are
started. When a command is used to stop Oracle Clusterware, the daemons will be stopped, but
the ohasd process will remain running.
l.
ai
gm
The following diagram depicts the Cluster startup process
7@
71
n2
wa
ha
id
it
Pr
You may have to manually control the Oracle Clusterware stack using the following mentioned
commands while applying patches or during planned outages.
To check the status of the individual daemons like crsd, evmd, and cssd
Each node in a cluster also has its own local OCR, called an Oracle Local Registry (OLR),that is
created when Oracle Clusterware is installed. Multiple processes on each node have
simultaneous read and write access to the OLR particular to the node on which they reside,
whether or not Oracle Clusterware is fully functional. By default, OLR is located at
GRID_HOME/cdata/$HOSTNAME.olr
The Oracle Clusterware installation process creates the voting disk and the OCR on shared
storage. If you select the option for normal redundant copies during the installation process,
then Oracle Clusterware automatically maintains redundant copies of these files to prevent the
files from becoming single points of failure. When you use normal redundancy, Oracle
Clusterware automatically maintains two copies of the OCR file and three copies of the voting
disk file.
$cat /etc/oracle/ocr.loc
To check the integrity of Voting disks, examine ocssd.log. Errors with voting disks appear in the
log.
or
$ocrcheck
1.To add one or more voting disks to non-ASM storage, run the following command as root:
3.To replace voting disk A with voting disk B on non-ASM storage, first add voting disk B and
then delete voting disk A
You do not have to delete any voting disks from ASM using this command
For example, if you have five voting disks configured, then a node must be able to access at
least three of the voting disks at any time. If a node cannot access the minimum required
number of voting disks it is evicted, or removed, from the cluster. After the cause of the failure
has been corrected and access to the voting disks has been restored, you can instruct Oracle
Clusterware to recover the failed node and
restore it to the cluster.
If you lose all copies of the voting disk and do not have a backup, the only safe way to re-create
the voting disk is to reinstall Oracle Clusterware. Having a backup of the voting disk can
drastically simplify the recovery of your system.
Starting with Oracle Clusterware 11g Release 2, backing up and restoring a voting disk using
dd command may result in the loss of the voting disk, so this procedure is not supported.
Backing up voting disks manually is no longer required because voting disk data is backed up
automatically in the OCR as part of any configuration change and voting disk data is
automatically restored to any added voting disks.
The voting disk files are backed up automatically by Oracle Clusterware if the contents of the
files have changed in the following ways:
Backing up OCR
The information stored in OCR is much more dynamic in nature than the voting disk. Oracle
Clusterware automatically performs routine backups of the OCR file. These are Physical
backups. Only one node has the responsibility to perform these backups, but that responsibility
can transfer to any other node in the cluster when outages occur. The default target location of
each automatically generated OCR backup file is <Grid_Home>/cdata/<clustername>
directory.
Oracle Clusterware automatically creates OCR backups (auto backups) every 4 hours (called 4
hourly backups), daily at the end of the day (daily backups), every week (weekly backup). At
any one time, Oracle Clusterware retains the latest 3 copies of 4 hourly backup, last 2 copies of
daily and weekly backups.
You cannot customize the backup frequencies or the number of files that Oracle Clusterware
retains. You can use any backup software to copy the automatically generated backup files at
least once daily to a different device from where the primary OCR file resides.
The backup file names cannot be changed and are named as follows:
backup00.ocr, backup01.ocr, backup02.ocr, day.ocr, day_.ocr, week.ocr and week_.ocr
$ocrconfig –showbackup
The new backup location will be used for both automatic and manual backups.
#ocrconfig –manualbackup
Note: you are not allowed to specify the name used for the manual backup. A file will be created
with the name backup_<date>_<time>.ocr and placed into the default backup location.
Note: the export will create a binary file containing a logical backup of the OCR keys and values.
Note: The operations in this section affect the OCR for the entire cluster. However, the
ocrconfig command cannot modify OCR configuration information for nodes that are shut down
or for nodes on which Oracle Clusterware is not running. Avoid shutting down nodes while
modifying the OCR using the ocrconfig command.
As the root user, enter the following command to add a new OCR file:
This command updates the OCR configuration on all the nodes on which Oracle Clusterware is
running.
For Example:
Replacing an OCR
If you need to change the location of an existing OCR, or change the location of a failed OCR to
the location of a working one, you can use the following procedure as long as one OCR file
remains online.
1. Use the OCRCHECK utility to verify that a copy of the OCR other than the one you are going
to replace is online, using the following command:
$ocrcheck
Note: The OCR that you are replacing can be either online or offline.
2. Use the following command to verify that Oracle Clusterware is running on the node on which
the you are going to perform the replace operation:
3. As the root user, enter the following command to designate a new location for the specified
OCR file:
This command updates the OCR configuration on all the nodes on which Oracle Clusterware is
running.
For example:
4. Use the OCRCHECK utility to verify that OCR replacement file is online:
$ocrcheck
Removing an OCR
To remove an OCR file, at least one copy of the OCR must be online. You can remove an OCR
location to reduce OCR-related overhead or to stop mirroring your OCR because you moved the
OCR to a redundant storage system, such as a redundant array of independent disks (RAID).
1. Use the OCRCHECK utility to ensure that at least one OCR other than the OCR that you are
removing is online.
$ocrcheck
2. As the root user, run the following command on any node in the cluster to remove a specific
OCR file:
For example:
The OLR provides various Oracle Clusterware processes with access to key configuration
information even when Oracle Clusterware is not running on the node. One of its functions is to
facilitate the Oracle Clusterware startup process in situations where the ASM stores the OCR
and Voting disks. During the startup process, the OLR is referenced to determine the exact
location of the voting disks. This enables to node to join the cluster. After this initial phase, ASM
is started. After ASM is started, processes that require the full OCR can start and the
Clusterware startup process completes.
You can manage the OLR using ocrcheck, ocrdump and ocrconfig utilities with the –local
option.
You can check the status of OLR on the local node using the OCRCHECK utility:
#ocrcheck –local
You can display the content of OLR on the local node to the text terminal that initiated the
program using the OCRDUMP utility:
You can perform administrative tasks on OLR on the local node using the OCRCONFIG
utility.
CHM Services
CHM consists of the following services:
System Monitor Service
Cluster Logger Service
Oracle Grid Infrastructure Management Repository
Syntax
Use the diagcollection.pl script with the following syntax:
diagcollection.pl {--collect [--crs | --acfs | -all] [--chmos [--
incidenttime time [--incidentduration time]]]
[--adr location [--aftertime time [--beforetime time]]]
[--crshome path | --clean | --coreanalyze}]
To limit the amount of data collected, use the following command syntax:
# Grid_home/bin/diagcollection.pl --collect --chmos
--incidenttime time --incidentduration duration
OCLUMON
The OCLUMON command-line tool is included with CHM and you can use it to query the CHM
repository to display node-specific metrics for a specified time period. You can also use
OCLUMON to perform miscellaneous administrative tasks, such as changing the debug levels,
querying the version of CHM, and changing the metrics database size.
l.
ai
gm
Oracle Clusterware Diagnostic and Alert Log files
7@
Oracle Clusterware uses Oracle Database fault diagnosability infrastructure to manage
diagnostic data and its alert log. As a result, most diagnostic data resides in the
71
Automatic Diagnostic Repository (ADR), a collection of directories and files located under a
n2
base directory that you specify during installation. This section describes clusterware-specific
wa
ADR files reside in an ADR home directory. The ADR home for Oracle Clusterware running on a
given host always has this structure:
ORACLE_BASE/diag/crs/host_name/crs
Under the ADR home are various directories for specific types of ADR data. The directories of
greatest interest are trace and incident. The trace directory contains all normal (non-incident)
trace files written by Oracle Clusterware daemons and command-line programs as well as the
simple text version of the Oracle Clusterware alert log. This organization differs significantly
from versions prior to Oracle Clusterware 12c release 1 (12.1.0.2), where diagnostic log files
were written under distinct directories per daemon.
In addition to alert messages, incidents also cause the affected program to produce a special,
separate trace file containing diagnostic data related to the error. These incident-specific trace
files are collected in the incident subdirectory of the ADR home rather than the trace
subdirectory. Both the normal trace files and incident trace files are collected and submitted to
Oracle when reporting the error.
The core directory is where Oracle Clusterware daemon core files are written when the normal
ADR location used for core files is not available (for example, before ADR services are
initialized in a program).
The output directory is where Oracle Clusterware daemons redirect their C standard output and
standard error files. These files generally use a name structure consisting of the executable
name with the characters OUT appended to a .trc file extension (like trace files). For example,
the redirected standard output from the Cluster Time Synchronization Service daemon is named
octssdOUT.trc. Typically, daemons write very little to these files, but in certain failure scenarios
important data may be written there.
Syntax
Use the diagcollection.pl script with the following syntax:
diagcollection.pl {--collect [--crs | --acfs | -all] [--chmos [--
incidenttime time [--incidentduration time]]]
[--adr location [--aftertime time [--beforetime time]]]
[--crshome path | --clean | --coreanalyze}]
Oracle ASM is installed in the Oracle Grid Infrastructure home separate from the Oracle
Database home. Only one Oracle ASM instance is supported on a server. When managing an
Oracle ASM instance, the administration activity must be performed in the Oracle Grid
Infrastructure home.
Also, unlike standard database instance, the ASM instance does not contain any physical files
such as Control files, Redo log file and Data files and only require a few init.ora parameters for
startup.
Upon startup, an ASM instance will spawn all the basic background processes, plus some new
one background processes that are specific to the operation of ASM. All the ASM instance
background processes names begin with asm, as opposed to the database instance, whose
process names begin with ora.
Oracle ASM instances mount disk groups to make Oracle ASM files available to database
instances.
Oracle ASM and database instances require shared access to the disks in a disk group. Oracle
ASM instances manage the metadata of the disk group and provide file layout information to the
database instances.
Oracle ASM instances can be clustered using Oracle Clusterware; there is one Oracle ASM
instance for each cluster node. If there are several database instances for different databases
on the same node, then the database instances share the same single Oracle ASM instance on
that node.
If the Oracle ASM instance on a node fails, then all of the database instances on that node also
fail. Unlike a file system driver failure, an Oracle ASM instance failure does not require restarting
the operating system. In an Oracle RAC environment, the Oracle ASM and database instances
on the surviving nodes automatically recover from an Oracle ASM instance failure on a node.
Each database instance using ASM has two new background processes called ASMB and
RBAL. RABL performs global opens of the disks in the disk groups. A global open means that
more than one database instance can be accessing the ASM disks at a time. At database
instance startup, ASMB connects as a foreground process into the ASM instance. All
communication between the database and ASM instances is performed via this bridge. This
includes physical file changes such as data file creation and deletion. Over this connection,
periodic messages are exchanged to update statistics and to verify that both instances are
healthy
ASM Instance
RBAL coordinates the rebalancing when a new disk is add or removed
(rebalance master)
ARB[1-9] actually does the work requested by the RBAL process (upto 9 of these)
(rebalance)
RDBMS Instance
RBAL performs global opens of all the disks in the disk groups
ASM registers its name and disks with the RDBMS via the cluster synchronization service
(CSS). This is why the oracle cluster services must be running, even if the node and instance is
not clustered.
When installing Oracle ASM in a clustered Oracle ASM environment where the Oracle Grid
Infrastructure home is shared among all of the nodes, OUI creates a single SPFILE for Oracle
ASM in a disk group. In a clustered environment without a shared Oracle Grid Infrastructure
home, OUI creates an SPFILE in a disk group for the Oracle ASM instance on each node.
You can use an SPFILE or a text-based initialization parameter file (PFILE) as the Oracle ASM
instance parameter file. If you use an SPFILE in a clustered Oracle ASM environment, then you
must place the SPFILE in a disk group, on a shared raw device, or on a cluster file system.
Oracle recommends that the Oracle ASM SPFILE is placed in a disk group.
If you do not use a shared Oracle Grid Infrastructure home, then the Oracle ASM instance can
use a PFILE. The same rules for file name, default location, and search order that apply to
database initialization parameter files also apply to Oracle ASM initialization parameter files.
You can administer Oracle ASM initialization parameter files with SQL*Plus, Oracle Enterprise
Manager, ASMCA, and ASMCMD commands.
During startup, Oracle searches for the spfile in the order below:
1. In a RAC environment the path is retrieved from the Grid Plug and Play profile (GPnP). GPnP
is an XML file that stores the configuration information related to Grid Infrastructure like virtual
ips, public ips etc. It also stores the spfile path and disk discovery path, which are used during
startup. GPnP is identical in all nodes of a cluster and it is stored under
"<Grid_Home>/gpnp/<Node_Name>/profiles/peer/profile.xml"
If it is a single instance configuration, the path of spfile is retrieved from the OLR (Oracle Local
Registry). OLR is a binary file. It contains similar configuration data as GPnP profile. The path of
OLR is kept in "/etc/oracle/olr.loc" file.
2. If spfile path is not set in GPnP profile or OLR, then it is searched under GI home. The path of
spfile should be "$ORACLE_HOME/dbs/spfile+ASM.ora"
3. If no spfile exists then pfile is searched under GI home. The path of pfile should be
"$ORACLE_HOME/dbs/init+ASM.ora"
Backing Up, Copying, and Moving an Oracle ASM Initialization Parameter File
You can back up, copy, or move an Oracle ASM SPFILE with the ASMCMD spbackup,
spcopy or spmove commands. You can also use the SQL CREATE SPFILE to create an
Oracle ASM SPFILE when connected to the Oracle ASM instance. You can copy and move an
Oracle ASM PFILE with the commands available on the specific platform, such as cp for Linux.
If the COMPATIBLE.ASM disk group attribute is set to 11.2 or greater for a disk group, you can
create, copy, or move an Oracle ASM SPFILE into the disk group.
For example, create an Oracle ASM SPFILE from the existing PFILE.
The CREATE SPFILE statement also updates the Grid Plug and Play (GPnP) profile. You can
check the location of the Oracle ASM SPFILE in the GPnP profile with the ASMCMD spget
command.
some parameters after database creation using Oracle Enterprise Manager or SQL ALTER
SYSTEM or ALTER SESSION statements.
The INSTANCE_TYPE initialization parameter is the only required parameter in the Oracle ASM
instance parameter file. The Oracle ASM* parameters use suitable defaults for most
environments. You cannot use parameters with names that are prefixed with Oracle ASM* in
database instance parameter files
Some database initialization parameters are also valid for an Oracle ASM instance initialization
file. In general, Oracle ASM selects the appropriate defaults for database parameters that are
relevant to an Oracle ASM instance
The following section contains information about some important parameters for Oracle ASM.
INSTANCE_TYPE
The INSTANCE_TYPE initialization parameter must be set to Oracle ASM for an Oracle ASM
instance. This parameter is optional for an Oracle ASM instance in an Oracle Grid Infrastructure
home.
The following is an example of the INSTANCE_TYPE parameter in the initialization file:
INSTANCE_TYPE = ASM
ASM_DISKGROUPS
The ASM_DISKGROUPS initialization parameter specifies a list of the names of disk groups
that an Oracle ASM instance mounts at startup.
ASM_DISKSTRING
The ASM_DISKSTRING initialization parameter specifies a comma-delimited list of strings that
limits the set of disks that an Oracle ASM instance discovers. The discovery strings can include
wildcard characters. Only disks that match one of the strings are discovered.
ASM_POWER_LIMIT
The ASM_POWER_LIMIT initialization parameter specifies the default power for disk
rebalancing in a disk group. The range of values is 0 to 1024. The default value is 1. A value of
0 disables rebalancing. Higher numeric values enable the rebalancing operation to complete
more quickly, but might result in higher I/O overhead and more rebalancing processes.
For disk groups that have the disk group ASM compatibility set to 11.2.0.2 or greater (for
example, COMPATIBLE.ASM = 11.2.0.2), the operational range of values is 0 to 1024
for the rebalance power.
For disk groups that have the disk group ASM compatibility set to less than 11.2.0.2, the
operational range of values is 0 to 11 inclusive. If the value for ASM_POWER_LIMIT is
larger than 11, a value of 11 is used for these disk groups
MEMORY_TARGET
This parameter enables Automatic Memory Management for Oracle ASM Instance. Oracle
strongly recommends that you use automatic memory management for Oracle ASM. Automatic
memory management is enabled by default on an Oracle ASM instance, even when the
MEMORY_TARGET parameter is not explicitly set. The default value used for
MEMORY_TARGET is acceptable for most environments
The minimum MEMORY_TARGET for Oracle ASM is 256 MB. If you set MEMORY_TARGET to
100 MB, then Oracle increases the value for MEMORY_TARGET to 256 MB automatically.
DB_CACHE_SIZE
This buffer cache is used to store metadata blocks .You do not have to set a value for the
DB_CACHE_SIZE initialization parameter if you use automatic memory management. The
setting for the DB_CACHE_SIZE parameter determines the size of the buffer cache. The default
value for this parameter is suitable for most environments.
SHARED_POOL_SIZE
Used for standard memory usage (like control structures) to manage the instance. Also used to
store extent maps. You do not have to set a value for the SHARED_POOL_SIZE initialization
parameter if you use automatic memory management. The setting for the
SHARED_POOL_SIZE parameter determines the amount of memory required to manage the
instance. The default value for this parameter is suitable for most environments.
LARGE_POOL_SIZE
Used for large allocations. You do not have to set a value for the LARGE_POOL_SIZE
initialization parameter if you use automatic memory management. The default value for this
parameter is suitable for most environments.
DIAGNOSTIC_DEST
The DIAGNOSTIC_DEST initialization parameter specifies the directory where diagnostics for
an instance are located. The default value for an Oracle ASM instance is the $ORACLE_BASE
directory for the Oracle Grid Infrastructure installation.
PROCESSES
The PROCESSES initialization parameter affects Oracle ASM, but the default value is usually
suitable. However, if multiple database instances are connected to an Oracle ASM instance,
you can use the following formula:
PROCESSES = 50 + 50*n
where n is the number database instances connecting to the Oracle ASM instance
REMOTE_LOGIN_PASSWORDFILE
The REMOTE_LOGIN_PASSWORDFILE initialization parameter specifies whether the Oracle
ASM instance checks for a password file. This parameter operates the same for Oracle ASM
and database instances.
You can use ALTER SYSTEM command to adjust ASM instance parameter settings in an
SPFILE. You can add an optional SID clause to specify that the setting is instance specific or
use the default SID= ‘*’ to explicitly state that the parameter applies to all instances.
For example:
To adjust your spfile so that the +ASM1 instance uses a specific power-limit setting
To adjust your spfile so that all the ASM instances use the same setting
Whether you create separate operating system privilege groups or use one group to provide
operating system authentication for all system privileges, you should use SYSASM to
administer an Oracle ASM instance. The SYSDBA privilege cannot be used to administer an
Oracle ASM instance. If you use the SYSDBA privilege to run administrative commands on an
Oracle ASM instance, the operation results in an error. The SYSDBA privilege is intended to be
used by the database to access disk groups.
Oracle also recommends the use of a less privileged user, such as ASMSNMP with SYSDBA
privileges that is created during installation, for monitoring the Oracle ASM instance.
OSASM group
This group is granted the SYSASM privilege, which provides full administrative privileges for the
Oracle ASM instance. For example, the group could be asmadmin.
You can connect as SYSDBA to use SQL*Plus or ASMCMD commands to manage Oracle ASM
components associated with the database. When running SQL or ASMCMD operations with the
SYSDBA privilege, connect to the database instance rather than the Oracle ASM instance.
Connecting as SYSDBA to the database instance has a limited set of Oracle ASM privileges.
For example, you cannot create a disk group when connected with the SYSDBA privilege.
When connected as SYSDBA to the database instance, the Oracle ASM operations are
limited to:
When you implement separate Oracle ASM and database administrator duties, this
configuration requires different group and different software owners. Implicitly this
implementation requires that the OSASM and OSDBA are different groups. For this
configuration, you must create an OSDBA for Oracle ASM group and a database instance must
be a member of that group to access the Oracle ASM instance.
In an installation that has been configured as Oracle Grid Infrastructure, the Oracle ASM user,
such as grid, does not have to be a member of the Oracle Database OSDBA group, such as
dba1 or dba2, because the Oracle Clusterware database agent runs as the database owner and
can use SYSDBA to connect to the database. Additionally, the owner of the operating system
disk devices should be the same as the owner of the Oracle ASM software.
Oracle also allows you to separate OSASM group from OSDBA group. Members of OSASM
group will have SYSASM privilege while members of OSDBA group will have SYSDBA
privilege.
Note: These commands update the password file for the local Oracle ASM instance only.
For example:
To connect to a local Oracle ASM instance with SQL*Plus, set the ORACLE_SID
environment variable to the Oracle ASM system identifier (SID).
The default Oracle ASM SID for a single-instance database is +ASM, and the default SID for
Oracle ASM for an Oracle RAC node is +ASMnode_number where node_number is the
number of the node. The ORACLE_HOME environment variable must be set to the Grid
Infrastructure home where Oracle ASM was installed.
Note: Oracle recommends that you do not change the default Oracle ASM SID name.
l.
ai
gm
This parameter indicates that an Oracle ASM instance, not a database instance, is starting.
7@
71
When you run the STARTUP command, rather than trying to mount and open a database,
n2
this command attempts to mount Oracle ASM disk groups.
The associated Oracle database instance does not have to be running when you start the
wa
You can start & stop ASM instances either using SRVCTL or SQL*PLUS.
it
The following commands shows how to start ASM instances using SRVCTL
Pr
$sqlplus / as sysasm
The following list describes how Oracle ASM interprets SQL*Plus STARTUP command
parameters.
FORCE Parameter
Issues a SHUTDOWN ABORT to the Oracle ASM instance before restarting it.
Mounts the disk groups specified in the ASM_DISKGROUPS initialization parameter. This is
the default if no command parameter is specified.
NOMOUNT Parameter
Starts up the Oracle ASM instance without mounting any disk groups.
RESTRICT Parameter
Starts up an instance in restricted mode that enables access only to users with both the
CREATE SESSION and RESTRICTED SESSION system privileges. You can use the
RESTRICT clause in combination with the MOUNT, NOMOUNT, and OPEN clauses.
In restricted mode, database instances cannot use the disk groups. In other words, databases
cannot open files that are in that disk group. Also, the disk group cannot be mounted by any
other instance in the cluster. Mounting the disk group in restricted mode enables only one
Oracle ASM instance to mount the disk group. This mode is useful to mount the disk group for
repairing configuration issues.
At startup, the Oracle ASM instance attempts to mount the following disk groups:
Disk groups specified in the ASM_DISKGROUPS initialization parameter
Disk group used by Cluster Synchronization Services (CSS) for voting files
Disk groups used by Oracle Clusterware for Oracle Cluster Registry (OCR)
Disk group used by the Oracle ASM instance to store the ASM server parameter file
(SPFILE)
If no disk groups are found in the previous list, then the Oracle ASM instance does not mount
any disk groups at startup. After the Oracle ASM instance has started, you can mount disk
groups with the ALTER DISKGROUP...MOUNT command
To shut down an Oracle ASM instance using SQL*PLUS, perform the following steps:
$sqlplus / as sysasm
SQL> SHUTDOWN NORMAL
The following list describes the SHUTDOWN modes and the behavior of the Oracle ASM
instance in each mode.
NORMAL Clause
Oracle ASM waits for any in-progress SQL to complete before performing an orderly
dismount of all of the disk groups and shutting down the Oracle ASM instance. Before the
instance is shut down, Oracle ASM waits for all of the currently connected users to
disconnect from the instance. If any database instances are connected to the Oracle ASM
instance, then the SHUTDOWN command returns an error and leaves the Oracle ASM
instance running. NORMAL is the default shutdown mode.
Oracle ASM waits for any in-progress SQL to complete before performing an orderly
dismount of all of the disk groups and shutting down the Oracle ASM instance. Oracle ASM
does not wait for users currently connected to the instance to disconnect. If any database
instances are connected to the Oracle ASM instance, then the SHUTDOWN command
returns an error and leaves the Oracle ASM instance running. Because the Oracle ASM
instance does not contain any transactions, the TRANSACTIONAL mode behaves the same
as IMMEDIATE mode.
ABORT Clause
The Oracle ASM instance immediately shuts down without the orderly dismount of disk
groups. This causes recovery to occur upon the next Oracle ASM startup. If any database
instance is connected to the Oracle ASM instance, then the database instance aborts.
Oracle strongly recommends that you shut down all database instances that use the Oracle
ASM instance before attempting to shut down the Oracle ASM instance. If Oracle Cluster
Registry (OCR) or voting files are stored in a disk group, the disk group can only be dismounted
by shutting down the Oracle ASM instance as part of shutting down the clusterware on a node.
You can alternatively use the Server Control Utility (SRVCTL) to start and stop the ASM listener
as below:
The lsnrctl and srvctl utilities exist in both the grid_home and RDBMS home directories.
Which one you use depends on where the listener configuration files reside. The Grid
Infrastructure installation will start a listener with the configuration files in the grid_home. By
default, the database installation will use that listener. In this case, set the ORACLE_HOME and
PATH environment variables to use the grid_home and run the utilities.
If you create a new listener with configuration files in the RDBMS home, set the
ORACLE_HOME and PATH environment variables to use the RDBMS home and run the
utilities.
On the Create ASM page, you enter the SYS password for the administrator of the Oracle ASM
instance and the password for the ASMSNMP user that can monitor this Oracle ASM instance.
Note that ASMSNMP is a less privileged user that is primarily used by Oracle Enterprise
Manager to monitor Oracle ASM instances. You can select the listener with which to register
this Oracle ASM instance. Click ASM Parameters to update the initialization parameters for the
Oracle ASM instance.
You can administer ASM Disk groups both with SQL commands and Oracle ASM Configuration
Assistant (ASMCA) graphical tool.
The following section describes some of the important disk group attributes.
AU_SIZE
This attribute can only be set when creating a disk group.
COMPATIBLE.ASM
The value for the disk group COMPATIBLE.ASM attribute determines the minimum software
version for an Oracle ASM instance that can use the disk group.
For Oracle ASM in Oracle Database 11g, 10.1 is the default setting for the COMPATIBLE.ASM
attribute when using the SQL CREATE DISKGROUP statement. When creating a disk group
with ASMCA, the default setting is 11.2.
COMPATIBLE.RDBMS
The value for the disk group COMPATIBLE.RDBMS attribute determines the minimum
COMPATIBLE database initialization parameter setting for any database instance that is
allowed to use the disk group.
Before advancing the COMPATIBLE.RDBMS attribute, ensure that the values for the
COMPATIBLE initialization parameter for all of the databases that access the disk group are set
to at least the value of the new setting for COMPATIBLE.RDBMS.
For example, if the COMPATIBLE initialization parameters of the databases are set to either
11.1 or 11.2, then COMPATIBLE.RDBMS can be set to any value between 10.1 and 11.1
inclusively.
For Oracle ASM in Oracle Database 11g, 10.1 is the default setting for the
COMPATIBLE.RDBMS attribute when using the SQL CREATE DISKGROUP statement as well
as ASMCA
Note: The database initialization parameter COMPATIBLE enables you to use a new release of
Oracle Database, while at the same time guaranteeing backward compatibility with an earlier
release.
DISK_REPAIR_TIME
Specifies the amount of time ASM should wait before an OFFLINE disk is dropped and start
disk rebalance operation. By default, Oracle ASM drops a disk in 3.6 hours after it is taken
offline.
The time can be specified in units of minutes (m or M) or hours (h or H). If you omit the unit,
then the default unit is hours. The REPAIR_TIMER column of V$ASM_DISK shows the amount
of time left (in seconds) before an offline disk is dropped.
You can display disk group attributes with the V$ASM_ATTRIBUTE view
FAILGROUP_REPAIR_TIME
This attribute specifies a default repair time for the failure groups in the disk group. The failure
group repair time is used if Oracle ASM determines that an entire failure group has failed. The
default value is 24 hours (24h). If there is a repair time specified for a disk, such as with the
DROP AFTER clause of the SQL ALTER DISKGROUP OFFLINE DISK statement, that disk
repair time overrides the failure group repair time.
This attribute can only be set when altering a disk group and is only applicable to normal and
high redundancy disk groups.
Disk group name is not case sensitive and is always converted to uppercase when stored
internally.
Specify the redundancy level (normal, high, external) of the disk group.
After a disk group is created, you cannot alter the redundancy level of the disk group. To
change the redundancy level, you must create another disk group with the appropriate
redundancy and then move the files to the new disk group.
Specify the disks that are to be formatted as Oracle ASM disks belonging to the
disk group.
The disks can be specified using operating system dependent wildcard characters in search
strings that Oracle ASM then uses to find the disks. You can specify names for the disks with
the NAME clause or use the system-generated names. System-generated names are in
the form diskgroup_nnnn, where nnnn is the disk number for the disk in the disk group.
For ASMLIB disks, the disk name defaults to the ASMLIB name that is the user label of the disk;
for example, mydisk is the default Oracle ASM disk name for ORCL:mydisk.
Oracle recommends that you create failure groups of equal size to maintain space balance and
even distribution of mirror data.
Optionally specify disk group attributes, such as software compatibility or allocation unit size.
The Oracle ASM instance ensures that any disk in a newly created disk group is addressable
and is not currently a member of another disk group. You must use FORCE only when adding a
disk that was dropped with FORCE. If a disk is dropped with NOFORCE, then you can add it
with NOFORCE.
For example, a disk might have failed and was dropped from its disk group. After the disk is
repaired, it is no longer part of any disk group, but Oracle ASM still recognizes that the disk had
been a member of a disk group. You must use the FORCE flag to include the disk in a new disk
group. In addition, the disk must be addressable, and the original disk group must not be
mounted. Otherwise, the operation fails.
Note: Use caution when using the FORCE option to add a previously used disk to a disk group;
you might cause another disk group to become unusable.
Disks with a valid header status of candidate, former, or provisioned are the only ones
allowed to be diskgroup members. This check and validation prevents ASM from destroying any
in-use data devices.
The CREATE DISKGROUP statement mounts the disk group for the first time, and adds the disk
group name to the ASM_DISKGROUPS initialization parameter if a server parameter file is being
used. If a text initialization parameter file is being used and you want the disk group to be
automatically mounted at instance startup, then you must remember to add the disk group name
to the ASM_DISKGROUPS initialization parameter before you shut down and restart the Oracle
ASM instance.
The following SQL statement creates disk group named fra with the default disk group
attribute values.
The following SQL statement creates a disk group named data with normal redundancy
consisting of two failure groups controller1 or controller2 with four disks in each failure
group.
Oracle ASM automatically rebalances when the configuration of a disk group changes. By
default, the ALTER DISKGROUP statement does not wait until the operation is complete before
returning. Query the V$ASM_OPERATION view to monitor the status of this operation.
Use the REBALANCE WAIT clause to cause the ALTER DISKGROUP statement processing to
wait until the rebalance operation is complete before returning. This is especially useful in
scripts.
You can interrupt a rebalance running in wait mode by typing CTRL-C on most platforms. This
causes the statement to return immediately with the message ORA-01013: user requested
cancel of current operation, and then to continue the operation asynchronously.
Typing CTRL-C does not cancel the rebalance operation or any disk add, drop, or resize
operations.
To control the speed and resource consumption of the rebalance operation, you can include the
REBALANCE POWER clause in statements that add, drop, or resize disks.
The following statement runs with a rebalance power of 5, and does not return until the
rebalance operation is complete
ALTER DISKGROUP data1 ADD DISK
'/devices/diskd*'
REBALANCE POWER 5 WAIT;
If a disk was previously member of a disk group that no longer exists, then you could use the
FORCE option to add the disk as a member of another disk group. For example, the following
example uses the FORCE clause to enable /devices/diskc3 to be added to data2, even
though it is a current member of data3. For this statement to succeed, data3 cannot be
mounted.
When a disk is dropped, the disk group is rebalanced by moving all of the file extents from the
dropped disk to other disks in the disk group. A drop disk operation might fail if not enough
space is available on the other disks. The best approach is to perform both the add and drop
operation with the same ALTER DISKGROUP statement. This has the benefit of rebalancing
data extents only one time and ensuring that there is enough space for the rebalance operation
to succeed.
Caution: The ALTER DISKGROUP...DROP DISK SQL statement returns to SQL prompt
before the drop and rebalance operations are complete. Do not reuse, remove, or disconnect
the dropped disk until the HEADER_STATUS column for this disk in the V$ASM_DISK view
changes to FORMER. You can query the V$ASM_OPERATION view to determine the amount of
time remaining for the drop/rebalance operation to complete.
If you specify the FORCE clause for the drop operation, the disk is dropped even if Oracle ASM
cannot read or write to the disk. You cannot use the FORCE flag when dropping a disk from an
external redundancy disk group.
The below example drops diska5 from disk group data1, and also illustrates how multiple
actions are possible with one ALTER DISKGROUP statement.
The following example cancels the dropping of disks from disk group data1:
When a disk group is mounted, a disk group number is chosen. This number may change
across disk group mounts. A disk group number is not recorded in any persistent structure, but
the current value can be viewed in the GROUP_NUMBER column of the V$ASM views.
When you want to mount or dismount disk groups manually, use the ALTER DISKGROUP
...MOUNT or ALTER DISKGROUP...DISMOUNT statement. You can mount or dismount disk
groups by name, or specify ALL.
If you try to dismount a disk group that contains open files, the statement fails, unless you also
specify the FORCE clause.
The following SQL statement dismounts all disk groups that are currently mounted to the Oracle
ASM instance:
l.
ai
The following SQL statement mounts disk group data1:
gm
7@
ALTER DISKGROUP data1 MOUNT;
71
n2
Mounting Disk Groups Using the FORCE Option
Oracle ASM provides MOUNT FORCE option with ALTER DISKGROUP to enable Oracle ASM
wa
disk groups to be mounted in normal or high redundancy modes even though some Oracle ASM
ha
disks may be unavailable to the disk group at mount time. The default behavior without the
id
FORCE option is to fail to mount a disk group that has damaged or missing disks.
it
The MOUNT FORCE option is useful in situations where a disk is temporarily unavailable and
Pr
you want to mount the disk group with reduced redundancy while you correct the situation that
caused the outage.
To successfully mount with the MOUNT FORCE option, Oracle ASM must be able to find at least
one copy of the extents for all of the files in the disk group. In this case, Oracle ASM can
successfully mount the disk group, but with potentially reduced redundancy.
The disks that Oracle ASM cannot access are placed in an offline mode. Oracle ASM then
begins timing the period that these disks are in an offline mode. If the disk offline time period
exceeds the timer threshold set by DISK_REPAIR_TIME disk group attribute, then those disks
are permanently dropped from the disk group.
The following example shows how to use the FORCE option to force the mount of the data1
disk group:
By default, the CHECK DISK GROUP clause verifies all of the metadata directories. Oracle ASM
displays summary errors and writes the details about the errors in an alert log. The CHECK
keyword performs the following operations:
The REPAIR | NOREPAIR clause specifies whether Oracle ASM should attempt to repair
errors that are found during the check. The default is NOREPAIR. Use the NOREPAIR clause to
receive alerts about inconsistencies and to suppress Oracle ASM from resolving the errors
automatically.
The following example statement checks for consistency in the metadata for all disks in the
data1 disk group:
The Oracle ASM instance must be started and the disk group must be mounted with none of the
disk group files open, in order for the DROP DISKGROUP statement to succeed. The statement
does not return until the disk group has been dropped.
When you drop a disk group, Oracle ASM dismounts the disk group and removes the disk group
name from the ASM_DISKGROUPS initialization parameter if a server parameter file is being
used. If a text initialization parameter file is being used, and the disk group is mentioned in the
ASM_DISKGROUPS initialization parameter, then you must remove the disk group name from
the ASM_DISKGROUPS initialization parameter before the next time that you shut down and
restart the Oracle ASM instance.
After ensuring that none of the files contained in data1 are open, Oracle ASM rewrites the
header of each disk in the disk group to remove Oracle ASM formatting information.
If you cannot mount a disk group but must drop it, you can use the FORCE option of the DROP
DISKGROUP statement. This command enables you to remove the headers on disks that
belong to a disk group that cannot be mounted by any Oracle ASM instances as shown in the
following example:
DROP DISKGROUP data1 FORCE;
The disk group on which you perform this operation should not be mounted anywhere in the
cluster. When you use the FORCE option, the Oracle ASM instance does not attempt to verify
that a disk group is being used by another Oracle ASM instance in the same storage
subsystem.
Oracle ASM Configuration Assistant enables you to configure or create Oracle ASM disk groups
with the Configure ASM Disk Groups tab, as shown in the below figure. The disk group tab
displays selected details about the disk group, such as name, size, free space, usable space,
redundancy level, and state of the disk group.
You can right click a selected disk group in the screen to display the configuration options
menu. The menu options enable you to:
Add disks to the disk group
Edit the disk group attributes
Manage templates for the disk group
Create an Oracle ACFS file system on the disk group for a database home
Dismount and mount the disk group
Drop the disk group
There are additional options for creating a disk group, mounting all disk groups, and
dismounting all disk groups. Oracle ASM Configuration Assistant enables you to create an
Oracle ASM disk group by clicking Create on the Oracle ASM Configuration Assistant disk
group page. The page for creating a disk group is shown in the below figure.
On this page, you can enter a name for the disk group, select the redundancy level, change the
disk discovery path to locate disks in a different directory, identify the disks to be in the disk
group.
You can choose to show advanced options when creating a disk group, as shown in the below
figure. The advanced options page enables you to set additional options, such as disk group
compatibility settings and allocation unit (AU) size. When you are finished with the create disk
group specifications, click OK.
511 disk groups in a storage system for Oracle Database 12c Release 1 or later
10,000 Oracle ASM disks in a storage system
1 million files for each disk group
Without any Oracle Exadata Storage, Oracle ASM has the following storage limits if the
COMPATIBLE.ASM or COMPATIBLE.RDBMS disk group attribute is set to less than 12.1:
2 terabytes (TB) maximum storage for each Oracle ASM disk
20 petabytes (PB) maximum for the storage system
Without any Oracle Exadata Storage, Oracle ASM has the following storage limits if
the COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes are set to 12.1
or greater:
4 PB maximum storage for each Oracle ASM disk with the allocation unit (AU) size
equal to 1 MB
8 PB maximum storage for each Oracle ASM disk with the AU size equal to 2 MB
16 PB maximum storage for each Oracle ASM disk with the AU size equal to 4 MB
32 PB maximum storage for each Oracle ASM disk with the AU size equal to 8 MB
320 exabytes (EB) maximum for the storage system
With all Oracle Exadata Storage, Oracle ASM has the following storage limits:
4 PB maximum storage for each Oracle ASM disk with the AU size equal to 1 MB
8 PB maximum storage for each Oracle ASM disk with the AU size equal to 2 MB
16 PB maximum storage for each Oracle ASM disk with the AU size equal to 4 MB
32 PB maximum storage for each Oracle ASM disk with the AU size equal to 8 MB
320 EB maximum for the storage system
The maximum size limit of a disk group equals the maximum disk size multiplied by the
maximum number of disks in a disk group (10,000). The maximum number of disks across all
disk groups is 10,000. The 10,000 disks can be in one disk group or distributed across a
maximum of 511 disk groups.
File size limits are dependent on the value of the disk group compatibility attributes. Oracle ASM
supports file sizes greater than 128 TB in any redundancy mode when the
COMPATIBLE.RDBMS disk group attribute is set greater than10.1.
If COMPATIBLE.RDBMS is set to 10.1, the file size limits are less. For example, with
COMPATIBLE.RDBMS equal to 10.1 and the AU size equal to 1 MB, Oracle ASM filesize limits
are:
External redundancy: 16 TB
Normal redundancy: 5.8 TB
High redundancy: 3.9 TB
Note: Oracle Database supports data file sizes up to 128 TB depending on the file system. In
addition, Oracle Database has a file size limit that is dependent on the DB_BLOCK_SIZE
initialization parameter.
Category Commands
ASMCMD Instance Management Commands dsget, dsset, lsct, lsop, lspwusr, orapwusr,
shutdown, spbackup, spcopy, spget, spmove,
spset, startup
ASMCMD File Management Commands cd, cp, du, find, ls, lsof, mkalias, pwd, rm, rmalias
ASMCMD Disk Group Management Commands chdg, chkdg, dropdg, iostat, lsattr, lsdg, lsdsk, lsod,
md_
backup, md_restore, mkdg, mount, offline, online,
rebal,
remap, setattr, umount
ASMCMD Template Management Commands chtmpl, lstmpl, mktmpl, rmtmpl
ASMCMD File Access Control Commands chgrp, chmod, chown, groups, grpmod, lsgrp, lsusr,
mkgrp, mkusr, passwd, rmgrp, rmusr
ASMCMD Volume Management Commands volcreate, voldelete, voldisable, volenable, volinfo,
volresize, volset, volstat
In the previous fully qualified filename, data is the disk group name, orcl is the database name,
CONTROLFILE is the file type, and so on.
Only the slash (/) is supported by ASMCMD. Filenames are not case sensitive, but are
case retentive. If you type a path name as lowercase, ASMCMD retains the lowercase.
Directory
As in other file systems, an Oracle ASM directory is a container for files, and an Oracle ASM
directory can be part of a tree structure of other directories. The fully qualified filename
represents a hierarchy of directories in which the plus sign (+) represents the root directory. In
each disk group, Oracle ASM automatically creates a directory hierarchy that corresponds to the
structure of the fully qualified filenames in the disk group. The directories in this hierarchy are
known as system-generated directories.
ASMCMD enables you to move up and down in this directory hierarchy with the cd (change
directory) command. The ASMCMD ls (list directory) command lists the contents of the current
directory, while the pwd command prints the name of the current directory.
When you start ASMCMD, the current directory is set to root (+). For an Oracle ASM instance
with two disk groups, for example, data and fra, entering an ls command with the root
directory as the current directory produces the following output:
ASMCMD> ls
data/
fra/
The following example demonstrates navigating the Oracle ASM directory tree (refer to the fully
qualified filename shown previously):
ASMCMD> cd +data/orcl/CONTROLFILE
ASMCMD> ls
Current.256.541956473
Current.257.541956475
You can create your own directories as subdirectories of the system-generated directories using
the ASMCMD mkdir command. The directories that you create can have subdirectories, and
you can navigate the hierarchy of both system-generated directories and user-created
directories with the cd command.
The following example creates the directory mydir under orcl in the disk group data:
If you start ASMCMD with the -p flag, then ASMCMD shows the current directory as part of its
prompt
Alias
Aliases are filenames that are references or pointers to system-generated filenames. However,
aliases are user-friendly names. Aliases are similar to symbolic links in UNIX or Linux
computers. You can create aliases to simplify Oracle ASM filename administration.
You can create aliases with the mkalias ASMCMD command or a SQL ALTER DISKGROUP
command. An alias has at a minimum the disk group name as part of its complete path. You can
create aliases at the disk group level or in any system-generated or user-created subdirectory.
If you run the ASMCMD ls (list directory) with the -l flag, each alias is listed with the system-
generated file to which the alias refers.
An absolute path refers to the full path of a file or directory. An absolute path begins with a plus
sign (+) followed by a disk group name, followed by subsequent directories in the directory tree.
The absolute path includes directories until the file or directory of interest is reached. A
complete system-generated filename, otherwise known as the fully qualified filename, is an
example of an absolute path to a file.
Using an absolute path enables the command to access the file or directory regardless of where
the current directory is set. The following rm command uses an absolute path for the filename:
A relative path includes only the part of the filename or directory name that is not part of the
current directory. That is, the path to the file or directory is relative to the current directory.
In addition, you can use the pseudo-directories "." and ".." rather than a directory name. The "."
pseudo-directory is the current directory. The ".." pseudo-directory is the parent directory of the
current directory.
The following example demonstrates how to use relative directory paths and pseudo-directories:
Wildcard Characters
The wildcard characters * and % match zero or more characters anywhere within an absolute or
relative path, which saves typing of the full directory or file name. The two wildcard characters
behave identically. There are various ASMCMD commands that accept wildcards. These
include cd, du, find, ls, lsattr, lsdg, lsdsk, lsgrp, lsusr, and rm.
• Log in to the host which contains the Oracle ASM instance that you plan to administer.
You must log in as a user that has SYSASM or SYSDBA privileges through operating
system authentication. The SYSASM privilege is the required connection to administer the
Oracle ASM instance.
• To connect to the Oracle ASM instance, run ASMCMD that is located in the bin
subdirectory of the Oracle Grid Infrastructure home (Oracle ASM home).
Ensure that the ORACLE_HOME and ORACLE_SID environment variables to refer to the Oracle
ASM instance.
The default value of the Oracle ASM SID for a single-instance database is +ASM. In Oracle Real
Application Clusters environments, the default value of the Oracle ASM SID on any node is
+ASMnode#.
• You can connect to the database instance as SYSDBA by running ASMCMD that is located
in the bin directory of the Oracle Database home to manage Oracle ASM components
associated with the database.
Ensure that the ORACLE_HOME and ORACLE_SID environment variables to refer to the
database instance. With this connection, there is a limited set of operations that can be run. For
example, you cannot create a disk group when connected with the SYSDBA privilege.
When connected as SYSDBA to the database instance, the Oracle ASM operations are limited
to:
• Create and delete files, aliases, directories, and templates
• Examine various Oracle ASM instance views
• Operate on files that were created by this user or only access files to which another user
had explicitly granted access
• Granting Oracle ASM File Access Control to other users
l.
ai
You can specify the -a option to choose the type of connection, either SYSASM or SYSDBA.
gm
The default value is SYSASM and is used when administering an Oracle ASM instance. Connect
7@
as SYSDBA when connecting to the database instance.
71
n2
For example:
$ asmcmd -a sysasm
wa
ha
You can specify the -p option with the asmcmd command to include the current directory in the
id
$ asmcmd -p
Pr
You can specify the -V option when starting asmcmd to display the asmcmd version number.
After displaying the version number, asmcmd immediately exits.
For example:
$ asmcmd -V
asmcmd version 12.1.0.2.0
Getting Help
Type help at the ASMCMD prompt to view general information about ASMCMD and a list of
available ASMCMD commands.
You can type help command to display help text for a specific command, including usage
information about how to run the command with its options.
The following example uses dsget to retrieve the current discovery diskstring value from the
GPnP profile and the ASM_DISKSTRING parameter.
A resource can be local or global, if it is local it can be acted upon without consulting other
instances, if it is global it cannot be acted upon without consulting or informing remote
instances. GCS is used as a messaging agent to coordinate manipulation of a global resource.
A global resource is a resource that is visible to all the nodes within the cluster. Data buffer
cache blocks are the most obvious and most heavily used global resource, transaction
enqueues and database data structures are other examples.
A local resource is a resource that is accessible to only one instance within the cluster.
Each instance maintains a part of the GRD in its System Global Area. GCS and GES nominate
one instance to manage all information about a particular resource. This instance is called the
resource master. Also, each instance knows which instance masters which resource.
Maintaining cache coherency is an important part of RAC activity. Cache Coherency is the
technique of keeping multiple copies of a block consistent between different oracle instances.
GCS implements cache coherency by using Cache Fusion algorithm. The GCS processes
manage the routing of inter-instance traffic among Oracle RAC instances.
latest version of the data block (including instance A modifications) and then exclusively
locks it on instance B behalf.
3. At any one point in time, only one instance has the current copy of the block, thus
keeping the integrity of the block.
GCS maintains data coherency and coordination by keeping track of all lock status of each
block that can be read/written to by any nodes in the RAC. GCS is an in memory database that
contains information about current locks on blocks and instances waiting to acquire locks. This
is known as Parallel Cache Management (PCM).
GES manages all non-cache fusion inter instance resource operations and tracks the status of
all Oracle enqueuing mechanisms. The primary resources that GES controls are dictionary
cache locks and library cache locks. GES also performs deadlock detection to all deadlock-
sensitive enqueues and resources.
GCS handle data buffer cache blocks and GES handle all the non-data block resources.
The Cache Fusion architecture overcomes this fundamental weakness in traditional shared disk
systems by utilizing the collective caches of all the nodes in the cluster to satisfy database
requests. The Global Cache Service manages the status and transfer of data blocks across the
buffer caches of the instances.
The Global Cache Service is tightly integrated with the buffer cache manager to enable fast
lookup of resource information in the Global Resource Directory. This directory is distributed
across all instances and maintains the status information about resources including any data
blocks that require global coordination. Query requests can now be satisfied by the local cache
or any of the other caches. This reduces disk I/O. Update operations do not require disk I/O for
synchronization since the local node can obtain the needed block directly from any of the cluster
database node caches. Expensive disk I/Os are only performed when none of the collective
caches contain the necessary data and when an update transaction performs a COMMIT
operation that requires disk write guarantees. This implementation effectively expands the
working set of the database cache and reduces disk I/O to dramatically speed up database
operation.
Let’s look, however, at the specific types of cross-node contention for data and how the Cache
Fusion architecture addresses each of them. There are four situations that warrant
consideration when multiple Oracle instances access the same data. For reasons of simplicity,
the examples refer to a two-node cluster with “node 1” and “node 2.”
read/read — User on node 1 wants to read a block that user on node 2 has recently read.
read/write — User on node 1 wants to read a block that user on node 2 has recently updated.
write/read — User on node 1 wants to update a block that user on node 2 has recently read.
write/write — User on node 1 wants to update a block that user on node 2 has recently
updated.
The read/read case typically requires little or no coordination, depending the on specific
database implementation. In a traditional shared disk implementation the request by the user on
node 1 will be satisfied either via local cache access or by way of disk read operations. In the
Cache Fusion implementation, the read request may be served by any of the caches in the
cluster database where the order of access preference is local cache, remote cache, and finally
disk I/O. If the query request is served by a remote cache, the block is transferred across the
high speed cluster interconnect from one node’s cache to another and expensive disk I/O is
avoided.
Both the read/write and write/write cases, in which a user on node 2 updates the block,
coordination between the instances becomes necessary so that the block being read is a read
consistent image (for read/write) and the block being updated preserves data integrity (for
write/write). In both cases, the node that holds the initially updated data block ships the block to
the requesting node across the high speed cluster interconnect and avoids expensive disk I/O
for read. It does this with full recovery capabilities.
In the case of the write/read scenario, node 1 wants to update a block that’s already read and
cached by node 2. An update operation typically involves reading the relevant block into
memory and then writing the updated block back to disk. In this scenario disk I/O for read is
avoided and performance is increased as the block is shipped from the cache of node 2 into the
cache of node 1.
This “fusing of the caches,” to provide users an extended database cache for queries and
updates, makes the Cache Fusion architecture a promising one.
Database in RAC
The primary difference between RAC storage and storage for non-RAC (single instance)
database is that all data files in RAC must reside on shared devices to be shared by all
instances that access the same database. You must also create at least two redo log groups for
each instance, and all the redo log groups must also be stored on shared devices for instance
or crash recovery purpose. Each instance’s online redo log groups are called an instance’s
thread of online redo. You must also create one undo tablespace for each instance. Each
instance’s undo tablespace must be shared by all other instances for recovery purposes.
You can place Archive log files on a local file system or shared storage that accessible by all
nodes of the cluster. If you store archive log files on local file system of each node, then you are
always forced to make archive logs available to the other cluster members at the time of
recovery. For this reason Oracle recommends to place archive logs on shared storage. If you
are using the recommended Fast Recovery Area (FRA) feature, then it must be created on
shared storage so that all instances can access it.
For each of the V$ views available, there is a corresponding GV$ view, with a few exceptions.
You can query GV$ views from any started instance.
Most administration tasks are the same for Oracle single-instance and Oracle RAC databases.
This chapter provides additional instructions for database administration tasks specific to Oracle
RAC, and recommendations for managing Oracle RAC databases.
When you use SRVCTL to perform configuration operations on your cluster, SRVCTL stores
configuration data in the Oracle Cluster Registry (OCR) in a cluster. SRVCTL performs other
operations, such as starting and stopping instances, by configuring and managing Oracle
Clusterware resources, which define agents that perform database startup and shutdown
operations using Oracle Call Interface APIs.
In Oracle Enterprise Manager, Oracle RAC-specific administrative tasks generally focus on two
levels: tasks that affect an entire cluster database and tasks that affect specific instances. For
example, you can use Oracle Enterprise Manager to start, stop, and monitor databases, cluster
database instances, and their listeners, and to schedule jobs or set up alert thresholds for
metrics. Or you can perform instance-specific commands such as setting parameters or creating
resource plans. You can also use Oracle Enterprise Manager to manage schemas, security,
and cluster database storage features.
Starting a SQL*Plus session and connecting to the database without specifying an instance
directs all SQL*Plus commands to the local instance. In this case, the default instance is also
the current instance. To connect to a different instance in SQL*Plus, issue a new CONNECT
command and specify a remote instance net service name, as shown in the following example:
CONNECT user_name@net_service_name
Enter password:
To start an Oracle RAC database instance on a node in the cluster, you must first start the
Oracle Grid Infrastructure stack on the node. An Oracle RAC database instance will not start on
a server on which the Oracle Grid Infrastructure stack is not running.
The procedure for shutting down Oracle RAC instances is identical to shutting down instances
in noncluster Oracle databases, with the following exceptions:
In Oracle RAC, shutting down one instance does not interfere with the operation of other
running instances.
To shut down an Oracle RAC database completely, shut down every instance that has
the database open or mounted
Using the SHUTDOWN TRANSACTIONAL command with the LOCAL option is useful
to shut down a particular Oracle RAC database instance. Transactions on other
instances do not block this operation. If you omit the LOCAL option, then this operation
waits until transactions on all other instances that started before you ran the
SHUTDOWN command either commit or rollback, which is a valid approach, if you
intend to shut down all instances of an Oracle RAC database.
To start or stop your entire cluster database, that is, all of the instances and its enabled
services, enter the following SRVCTL commands:
The following SRVCTL command, for example, mounts all of the non-running instances of
an Oracle RAC database:
l.
ai
gm
$ srvctl start instance -db db_unique_name -node node_name [-startoption start_ options]
7@
71
Note that this command also starts all enabled and non-running services that have
AUTOMATIC management policy, and for which the database role matches one of the
n2
service's roles.
wa
ha
To stop one or more instances, enter the following SRVCTL syntax from the command line:
id
You can enter either a comma-delimited list of instance names to stop several instances or
you can enter a node name to stop one instance.
This command also stops the services related to the terminated instances on the nodes
where the instances were running.
As an example, the following command shuts down the two instances, orcl3 and orcl4,
on the orcl database using the immediate stop option:
To start or shutdown your local instance, initiate a SQL*Plus session and connect with the
SYSDBA or SYSOPER privilege and then issue the required command. For example:
You can start multiple instances from a single SQL*Plus session on one node using Oracle
Net Services. Connect to each instance in turn by using a Net Services connection string,
typically an instance-specific alias from your tnsnames.ora file.
For example, you can use a SQL*Plus session on a local node to perform a transactional
shutdown for two instances on remote nodes by connecting to each in turn using the
instance's individual alias name.
It is not possible to start or stop multiple instances, simultaneously, with SQL*Plus, so you
cannot start or stop all of the instances for a cluster database with a single SQL*Plus
command.
NOTE: Oracle recommends that you do not use SQL*Plus to manage Oracle ASM
instances in an Oracle RAC environment. Oracle Clusterware automatically manages Oracle
ASM instances, as required. If manual intervention is necessary, then use respective
SRVCTL commands.
Using SQL*Plus
1. On any node, from a SQL*Plus prompt, connect to a database instance by using a Net
Services connection string, typically an instance-specific alias from your tnsnames.ora
file.
INST_NUMBER INST_NAME
--------------------- -----------------
1 uclid-rac:yahoo1
2 uclid-rac:yahoo2
3 uclid-rac:yahoo3
Terminating Sessions
You can use the ALTER SYSTEM KILL SESSION statement to terminate a session on a
specific instance. When a session is terminated, any active transactions of the session are
rolled back, and resources held by the session (such as locks and memory areas) are
immediately released and available to other sessions.
1. Query the value of the INST_ID column in the GV$SESSION dynamic performance view to
identify which session to terminate.
2. Issue the ALTER SYSTEM KILL SESSION and specify the session index number (SID)
and serial number of a session that you identified with the GV$SESSION dynamic
performance view.
To use this statement, your instance must have the database open, and your session and
the session to be terminated must be on the same instance unless you specify integer3.
If the session is performing some activity that must be completed, such as waiting for a reply
from a remote database or rolling back a transaction, then Oracle Database waits for this
activity to complete, marks the session as terminated, and then returns control to you. If the
waiting lasts a minute, then Oracle Database marks the session to be terminated and
returns control to you with a message that the session is marked to be terminated. The
PMON background process then marks the session as terminated when the activity is
complete.
System altered.
Oracle RAC uses a traditional PFILE only if an SPFILE does not exist or if you specify
PFILE in your STARTUP command. Oracle recommends that you use an SPFILE to
simplify administration, to maintain parameter setting consistency, and to guarantee
parameter setting persistence across database shutdown and startup events. In addition,
you can configure RMAN to back up your SPFILE.
All instances in the cluster database use the same SPFILE at startup.
The value before the period (.) in an SPFILE entry identifies the instance to which the
particular parameter value belongs. When an asterisk (*) precedes the period, the value is
applied to all instances that do not have a subsequent, individual value listed in the SPFILE.
For example, assume that you start an instance with an SPFILE containing the following
entries:
*.OPEN_CURSORS=500
prod1.OPEN_CURSORS=1000
For the instance with the Oracle system identifier (SID) prod1, the OPEN_CURSORS
parameter is set to 1000 even though it has a database-wide setting of 500. Parameter file
entries that have the asterisk (*) wildcard character only affect the instances without an
instance-specific entry. These two types of settings can appear in any order in the
parameter file.
If another DBA runs the following statement, then Oracle Database updates the setting on
all instances except the instance with SID prod1:
*.OPEN_CURSORS=1500
prod1.OPEN_CURSORS=1000
$ORACLE_HOME/dbs/spfiledb_unique_name.ora.
ASM_PREFERRED_READ_ FAILURE_GROUPS
CLUSTER_DATABASE
CLUSTER_DATABASE_ INSTANCES
CLUSTER_INTERCONNECTS
DB_NAME
DISPATCHERS
GCS_SERVER_PROCESSES
INSTANCE_NAME
RESULT_CACHE_MAX_SIZE
SERVICE_NAMES
SPFILE
THREAD
The following parameters must have identical settings on all the instances:
COMPATIBLE
CLUSTER_DATABASE
CONTROL_FILES
DB_BLOCK_SIZE
DB_DOMAIN
DB_FILES
DB_NAME
DB_RECOVERY_FILE_DEST
DB_RECOVERY_FILE_DEST_SIZE
DB_UNIQUE_NAME
INSTANCE_TYPE (RDBMS or ASM)
PARALLEL_EXECUTION_MESSAGE_SIZE
REMOTE_LOGIN_PASSWORDFILE
UNDO_MANAGEMENT
The following parameters must be identical on every instance only if the parameter value is
set to zero:
DML_LOCKS
RESULT_CACHE_MAX_SIZE
Oracle recommends that you set the values for the following parameters to the same value
on all instances. Although you can have different settings for these parameters on different
instances, setting each parameter to the same value on all instances simplifies
administration.
ARCHIVE_LAG_TARGET
CLUSTER_DATABASE_ INSTANCES
LICENSE_MAX_USERS
LOG_ARCHIVE_FORMAT
SPFILE
TRACE_ENABLED
UNDO_RETENTION
Only the instance assigned to the undo tablespace can modify the contents of that tablespace.
However, each instance can read the undo data blocks created by any instance. Also, when
performing transaction recovery, any instance can update any undo tablespace, if that undo
tablespace is not currently being used by another instance for undo generation or transaction
recovery. Also, all instances of an Oracle RAC database must operate in the same undo mode
(AUTO or MANUAL).
When you configure Oracle ASM after installing Oracle Grid Infrastructure for a cluster, one
Oracle ASM instance is created on each node in your Oracle RAC environment if one does not
exist. Each Oracle ASM instance has either an SPFILE or PFILE type parameter file.
In a cluster, each Oracle ASM instance manages the metadata updates to the disk groups for
the node on which it is running. In addition, each Oracle ASM instance coordinates disk group
metadata with other nodes in the cluster.
You can use Enterprise Manager, Oracle ASM configuration assistant (ASMCA), Oracle ASM
command-line utility (ASMCMD), SQL*Plus, and SRVCTL to administer disk groups for Oracle
ASM in an Oracle RAC environment.
Redo logs contain a record of changes that have been made to data files. In a single-instance
Oracle database, redo logs are stored in two or more redo log file groups. Each of these groups
contains a redo log file and possibly one or more mirrored copies of that file. In an Oracle RAC
database, each instance requires its own set of redo log groups, which is known as a redo
thread. Mirrored copies of the redo
log files provide your system with extra protection against data loss that is due to hardware
failures or data corruption. If a redo log file is unreadable, then Oracle Database attempts to
access its mirrored copy. The redo log file mirrors should be located on different disk devices
from the primary redo log files.
Each instance's redo thread must contain at least two redo log groups. Each redo log group
should contain at least two members: a redo log and its mirrored copy. If you create your Oracle
RAC database using DBCA, then your Oracle RAC database automatically implements a
configuration that meets the Oracle recommendations.
You should create redo log groups only if you are using administrator-managed databases. For
policy-managed databases, if an instance starts due to a change in server pool cardinality, then
Oracle Database automatically creates redo log files, enables a redo thread for the instance if
there is not a redo thread allocated to that instance, and creates the undo tablespace if there is
not an undo tablespace allocated to that instance. The database must be using Oracle
Managed Files and Oracle ASM in this situation.
In an Oracle RAC database, all the redo log files reside on shared storage. In addition, each
instance must have access to the redo log files of all the other instances in the cluster. If your
Oracle RAC database uses Oracle ASM, then Oracle ASM manages the shared storage for the
redo log files and the access to those files.
In an Oracle RAC database, each instance writes and archives the redo log groups in its redo
thread in the same manner that single-instance Oracle databases do. However, in recovery
mode, the instance performing the recovery can read and process all the redo threads for the
database, regardless of which instance generated the redo thread. Being able to read all the
redo threads enables a running instance to recover the work completed by one or more failed
instances.
In case of instance failure, a surviving instance can read the redo logs of the failed instance.
Users can continue to access and update the database without waiting for the failed instance to
be restarted. For example, assume that you have an Oracle RAC database with two instances,
instance A and instance B. If instance A is down, then instance B can read the redo log files for
both instance A and B to ensure a successful recovery.
The archived redo log naming scheme that you use is important because when a node writes to
a log with a specific file name on its file system, the file must be readable by any node that must
access this archived redo log. For example, if node1 archives a log to
/oracle/arc_dest/log_1_100_23452345.arc, then node2 can back up this archived redo log only
if it can read /oracle/arc_dest/log_1_100_23452345.arc on its own file system.
The backup and recovery strategy that you choose depends on how you configure the archiving
destinations for each node. Whether only one node or all nodes perform archived redo log
backups, you must ensure that all archived redo logs are backed up. If you use RMAN
parallelism during recovery, then the node that performs recovery must have read access to all
archived redo logs in your cluster. Multiple nodes can restore archived logs in parallel. However,
during recovery, only one node applies the archived logs. Therefore, the node that is performing
the recovery must be able to access all of the archived logs that are needed for the recovery
operation. By default, the database determines the optimum number of parallel threads to use
during the recovery operation. You can use the PARALLEL clause in the RECOVER command
to change the number of parallel threads.
The primary consideration for Archived Redo Logs in RAC is to ensure that all archived redo
logs can be read from every node during recovery, and if possible during backups.
Also, for any archived redo log configuration, uniquely identify the archived redo logs with the
LOG_ARCHIVE_FORMAT parameter. If you do not specify a log format, then the default is
operating system-specific and includes %t, %s, and %r where t is thread number, s is log
sequence number and r is resetlogs identifier.
You can use either ASM or CFS (Cluster File System) for recovery area though Oracle
recommends to use ASM. When you use ASM storage, Oracle ASM uses an OMF based
naming format for archived redo log files. When you use CFS, you need to set
LOG_ARCHIVE_FORMAT for naming archived redo log files.
In case if you are not using either ASM or CFS, you can use local file system archiving scheme
in which each node archives to a uniquely named local directory. If recovery is required, then
you can configure the recovery node so that it can access directories on the other nodes
remotely. For example, use NFS on Linux and UNIX computers, or mapped drives on Windows
systems. Therefore, each node writes only to a local destination, but each node can also read
archived redo log files in remote directories on the other nodes.
Backups in RAC
Although RAC provides you with methods to avoid or to reduce down time due to a failure of
one or more (but not all) of your instances, you must still protect the database itself, which is
shared by all the instances. This means that you need to consider backup and recovery
strategies for your cluster database just as you would for a nonclustered database.
RMAN enables you to back up, restore, and recover data files, control files, server parameter
files (SPFILEs) and archived redo log files. RMAN is included with Oracle Database and does
not require separate installation.
Before you start using RMAN to backup RAC database you must configure various settings as
discussed in the below sections.
Configuring Channels
While performing backups in parallel, RMAN channels can connect to a different instance in the
cluster. Channel connections to the instances are determined using the connect string defined
by channel configurations. For example, in the following configuration, three channels are
allocated using dbauser/pwd@service_name. If you configure the SQL Net service name with
load balancing turned on, then the channels are allocated at a node as decided by the load
balancing algorithm.
However, if the service name used in the connect string is not for load balancing, then you can
control at which instance the channels are allocated using separate connect strings for each
channel configuration, as follows:
In the previous example, it is assumed that node1, node2 and node3 are SQL Net service
names that connect to pre-defined nodes in your Oracle RAC environment.
Alternatively, you can also use manually allocated channels to backup your database files. For
example, the following command backs up the SPFILE, controlfile, data files and archived redo
logs:
RUN
{
ALLOCATE CHANNEL CH1 CONNECT 'dbauser/pwd@mydb_1';
ALLOCATE CHANNEL CH2 CONNECT 'dbauser/pwd@mydb_2';
ALLOCATE CHANNEL CH3 CONNECT 'dbauser/pwd@mydb_3';
BACKUP DATABASE PLUS ARCHIVED LOG;
}
During backup operation, if at least one channel allocated has access to the archived log, then
RMAN automatically schedules the backup of the specific log on that channel. Because the
control file, SPFILE, and data files are accessible by any channel, the backup operation of these
files is distributed across the allocated channels. You can configure channels to use automatic
load balancing or you can specify specific channels for specific instances.
During backup operation, the instances to which the channels connect must be either all
mounted or all open. For example, if the instance on node1 has the database mounted while the
instances on node2 and node3 have the database open, then the backup fails.
Oracle RAC environments do not differ substantially from the media recovery procedures for
single-instance environments.
The node that performs the recovery must be able to restore all of the required data files. That
node must also be able to either read all the required archived redo logs on disk or be able to
restore them from backups. Each instance generates its own archive logs that are copies of its
dedicated redo log group threads. It is recommended that Automatic Storage Management
(ASM) or a cluster file system be used to consolidate these files
• Restoring Data Files: When restoring data files, the number of channels you allocate in the
RMAN recover script effectively sets the parallelism that RMAN uses. For example, if you
allocate five channels, you can have up to five parallel streams restoring data files.
• Applying Incremental Backups: Similarly, when you are applying incremental backups,
the number of channels you allocate determines the potential parallelism.
• Applying Archived Redo Logs with RMAN: Oracle Database automatically selects the
optimum degree of parallelism based on available CPU resources.
To disable parallel instance and crash recovery on a system with multiple CPUs, set the
RECOVERY_PARALLELISM parameter to 0.
Use the NOPARALLEL clause of the RMAN RECOVER command or ALTER DATABASE
RECOVER statement to force the RAC database to use nonparallel media recovery.
l.
ai
Workload Management with Services
gm
7@
This chapter describes how to manage workloads in Oracle Real Application Clusters (Oracle
71
RAC) to provide high availability and scalability for your applications.
n2
Services are a feature for workload management that organizes the universe of work execution
wa
within the database to make that work more manageable, measurable, tunable, and
ha
recoverable.
id
The concept of a service was first introduced in Oracle8i as a means for the listener to perform
it
connection load balancing between nodes and instances of a cluster. However, the concept,
Pr
A service is a grouping of related tasks within the database with common functionality, quality
expectations, and priority relative to other services. A service provides a single-system image
for managing competing applications running within a single instance and across multiple
instances and databases.Services divide workloads into mutually disjoint groupings. Each
service represents a workload with common attributes, service-level thresholds, and priorities.
To manage workloads, you can define services that you assign to a particular application or to a
subset of an application's operations. You can also use services to manage the workload for
different types of work. For example, online users can use one service while batch processing
can use a different service and reporting can use yet another service type.
Using standard interfaces, such as Enterprise Manager, and SRVCTL, services can be
configured, administered, enabled, disabled, and measured as a single entity.
Services are dynamic in that the number of instances a service runs on can be augmented
when load increases, and reduced when load declines. This dynamic resource allocation
enables a cost-effective solution for meeting demands as they occur.
To implement the workload balancing and continuous availability features of services, Oracle
Clusterware stores the HA configuration for each service in the Oracle Cluster Registry (OCR).
The HA configuration defines a set of preferred and available instances that support the service.
A preferred instance set defines the number of instances (cardinality) that support the
corresponding service. It also identifies every instance in the cluster that the service will run on
when the system first starts up.
An available instance does not initially support a service. However, it begins accepting
connections for the service when a preferred instance cannot support the service. If a preferred
instance fails, then the service is transparently restored to an available instance defined for the
service.
Oracle Database supports two broad types of services: Application services and Internal
services. Application services are mainly functional maps to workloads. Sessions doing work
for a common business function are grouped together. For Oracle E-Business suite, AP, AR,
GL, MFG, WIP, BOM, and so on create a functional division of work within the database and
can thus be categorized as services.
There is a limitation of 115 application services per database that you can create.
In addition to application services, the RDBMS also supports two internal services.
SYS$BACKGROUND is used by the background processes only. SYS$USERS is the default
service for user sessions that are not associated with any application service. Both internal
services support all the workload management features and neither one can be stopped or
disabled.
A special Oracle database service is created by default for the Oracle RAC database. This
default service is always available on all instances in an Oracle RAC environment.
The features discussed in this chapter do not work with the default database service. You must
create cluster managed services to take advantage of these features. You can only manage the
services that you create. Any service created automatically by the database server is managed
by the database server.
Delete a service. You can delete services that you created. However, you cannot delete or
modify the properties of the default database service that Oracle Database created.
Check the status of a service. A service can be assigned different roles among the
available instances. In a complex database with many services, you may not remember the
details of every service. Therefore, you may have to check the status on an instance or
service basis. For example, you may have to know the status of a service for a particular
instance before you make modifications to that instance or to the Oracle home from which it
runs.
Start or stop a service for a database or an instance. A service must be started before it
can be used for client connections to that instance. If you shut down your database, for
example, by running the SRVCTL command srvctl stop database -d db_unique_name
where db_unique_name is the name of the database you want to stop, then Oracle
Database stops all services to that database. Depending on the service management policy,
you may have to manually restart the services when you start the database.
Relocate a service to a different instance. You can move a service from one instance to
another instance to re-balance workloads, for example, after adding or deleting cluster
nodes.
Notes: When you use services, do not set a value for the SERVICE_NAMES parameter; Oracle
Database controls the setting for this parameter for the services that you create and for the
default database service.
Creating Services
To create a service with SRVCTL, enter a command from the command line using the following
syntax:
srvctl add service -database db_unique_name -service service_name [-eval]
-preferred preferred_list -available available_list] [-netnum network_number]
[-tafpolicy {BASIC | NONE | PRECONNECT}] [-edition edition_name]
[-role "[PRIMARY][,PHYSICAL_STANDBY][,LOGICAL_STANDBY][,SNAPSHOT_STANDBY]"
[-policy {AUTOMATIC | MANUAL}] [-notification {TRUE | FALSE}]
[-clbgoal {SHORT | LONG}] [-failovertype {NONE|SESSION|SELECT|TRANSACTION}]
[-rlbgoal {NONE | SERVICE_TIME | THROUGHPUT}] [-dtp {TRUE | FALSE}]
[-failovermethod {NONE | BASIC}] [-failoverretry failover_retries]
[-failoverdelay failover_delay] [-pdb pluggable_database]
[-sql_translation_profile sql_translation_profile]
[-global {TRUE | FALSE}] [-maxlag max_lag_time] [-commit_outcome {TRUE|FALSE}]
[-retention retention_time] [-replay_init_time replay_initiation_time]
[-session_state {STATIC|DYNAMIC}] [-pqservice pq_service]
[-pqpool pq_pool_list] [-force] [-verbose]
For example:
To create a service called GL with preferred instance RAC02 and an available instance
RAC01:
$ srvctl add service –database PROD1 –service GL -preferred RAC02 -available RAC01
Enter the following SRVCTL syntax from the command line to stop or start a service:
For example:
Stop a service:
$ srvctl stop service –database orcl –service AP
Use the following SRVCTL syntax from the command line to enable and disable services:
For example:
Relocating Services
Run the srvctl relocate service command from the command line to relocate a service. You
might use this command when a service has failed over to an available instance, but you want
to move it back to the preferred instance after that instance is restarted.
The following command relocates the crm service from instance apps1 to instance apps3:
For example:
Transparent Application Failover (TAF) or Run time connection failover (RTCF) transparently re-
routs application (query) clients to an available database node in the cluster when the
connected node fails. Application clients do not see error messages describing loss of service.
Furthermore, you can configure database clients to pre-connect, or to have redundant idle
connections. These redundant connections with another database node avoid delays if
thousands of users must migrate servers during a node failure. These rich capabilities provide
great benefits to e-business applications that cannot afford downtime.
When Oracle Net Services establishes a connection to an instance, the connection remains
open until the client closes the connection, the instance is shutdown, or a failure occurs. If you
configure Transparent Application Failover (TAF) for the connection, then Oracle Database
moves the session to a surviving instance when an outage occurs.
TAF can restart a query after failover has completed but for other types of transactions, such as
INSERT, UPDATE, or DELETE, the application must rollback the failed transaction and
resubmit the transaction. You must also re-execute any session customizations, in other words,
ALTER SESSION statements, after failover has occurred. However, with TAF, a connection is
not moved during normal processing, even if the workload changes over time.
tnsnames parameter FAILOVER_MODE enables TAF. When you enable TAF you will define
the following attributes of TAF in tnsnames.ora file.
For example, the following tnsnames.ora entry on the client side shows the TAF settings:
ERP=
(DESCRIPTION =(FAILOVER=ON)(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=cluster01-scan)(PORT=1521))
(CONNECT_DATA =
(SERVICE_NAME = ERP)
(FAILOVER_MODE =
(TYPE=SELECT)
(METHOD=BASIC)
(RETRIES=180)
(DELAY=5))))
The TAF setting on a service can be NONE, BASIC, or PRECONNECT and overrides any TAF
setting in the client connection definition. You can also specify a single TAF policy for all users
of a service by defining the FAILOVER_METHOD (-m option), FAILOVER_TYPE (-e option).
The TAF policy can also define the number of times that a failed session attempts to reconnect
to the service (-z option) and how long it should wait between reconnection attempts (-w option).
To define a TAF policy for a service, use SRVCTL as in the following example, where the
service name is tafconn.example.com and the database name is CRM:
Patchset
Bundle Patch (BP)
One-Off Patch or Interim Patch
Patchset Update (PSU)
Critical Patch Update (CPU)
Patchset
Patchset is a set of patches provided on top of a base release (for example, 10.2.0.1.0 or
11.2.0.1.0). A patchset increments the fourth digit of the release number. A patchset will include
updates for Oracle Grid Infrastructure and updates for the Oracle RDBMS software. You must
update Oracle Grid Infrastructure before updating the Oracle RDBMS product. Though the
reverse is not true, Oracle Grid Infrastructure may be patched without patching the RDBMS.
Patchsets are released less frequently than other types of patches. They are cumulative and
may contain hundreds of fixes.
A patchset must be installed using Oracle Universal Installer (OUI) and is generally considered
as an "upgrade".
Prior to 11gR2, a base release had to be installed before a patchset could be applied. For
example, to install 10.2.0.5 on Linux, the 10.2.0.1 base release has to be installed first and then
upgraded to 10.2.0.5. Starting with 11gR2, patchset releases are now full releases and no
longer require a "base" release, e.g. 11.2.0.2 can be installed directly without having to install
11.2.0.1 first.
Prior to 11gR2, the same patchset download is used to patch the Clusterware, ASM and
Database homes. For example, Patch 8202632 is the 10.2.0.5 patchset, this same patch (Patch
8202632) will be used to patch the 10.2.0.x Clusterware, 10.2.0.x ASM and 10.2.0.x Database
to 10.2.0.5.
Prior to 11gR2 - even though the CRS and RDBMS base releases were provided on separate
media (downloadable zip file or separate DVD/CD) - the patchsets for both products were
delivered as one i.e. the same patchset could be applied to the CRS as well as the RDBMS
home.
Starting with 11gR2, the patchsets for Grid Infrastructure and RDBMS are delivered separately
(as they are full releases).
Grid Infrastructure patchsets can be applied in a rolling fashion, while database patchsets
cannot. For example, you can rolling upgrade Grid Infrastructure to 11.2.0.2, but you have to
shutdown database on all nodes to upgrade to database 11.2.0.2.
Bundle Patch
The second common type of patch available for Oracle Grid Infrastructure is known as Bundle
Patch (BP). A Bundle Patch is a small grouping of individual fixes, usually around 20 to 30,that
are fully regression tested and released more frequently than patchsets. Each bundle patch on
the same patchset level is cumulative for that patchset level only. The naming convention is
11.1.0.6.0 BP#1, 11.1.0.6.0 BP#2, and so on. When the patchset level is incremented, the
bundle patch numbering will restart. Bundle patches are always applied with the opatch utility
instead of the OUI utility. BPs are binaries and do not require relinking.
One-Off Patch
Another type of patch that may be applied to Oracle Grid Infrastructure is known as “One Off”
or “Interim” patch. These are installed with the opatch utility and are usually priority 1 fixes that
are not contained in a bundle yet. An interim patch contains fixes for one or in some cases
several bugs (merge patch).
Grid Infrastructure interim patches are rare, they usually are built on top of the latest PSU (at the
time) and include the entire PSU they were built on. The same does not hold true for database
interim patches, they usually do not include a PSU.
Clusterware PSUs (pre-11.2) are referred to as CRS PSUs. Grid Infrastructure PSUs
(11.2 onwards) are referred to as GI PSUs.
Grid Infrastructure (GI) PSUs contains both GI PSU and Database PSU for a particular
quarter.
For example, 11.2.0.2.2 GI PSU contains both the 11.2.0.2.2 GI PSU and the 11.2.0.2.2
Database PSU. You are able to take note of this when you extract a GI PSU, you will
see 2 directories (named with the Patch number) one for the GI PSU and one for the
RDBMS PSU. Once the GI PSU is applied, "opatch lsinventory" will show that both GI
PSU and DB PSU are applied.
GI PSUs are always cumulative meaning that you can apply higher version GI PSU directly
without having to apply a lower version one first. For example, the 11.2.0.2.2 GI PSU can be
applied to a 11.2.0.2 home without having to apply GI PSU 11.2.0.2.1 first.
Database PSUs can be subject to overlay PSU packaging. In these cases, the PSUs are still
cumulative, but a higher PSU may require a lower PSU to be applied first; for example, to apply
database PSU 10.2.0.4.7, you must apply database PSU 10.2.0.4.4 first. If a previous PSU is a
prerequisite to a later PSU the requirement will be clearly documented in the PSU readme.
CPU Patches
Critical Patch Updates are collection of high-priority fixes for security related issues and are only
applicable to the Database Home. CPUs are released quarterly on the same cycle as PSUs and
are cumulative with respect to prior security fixes but may contain other fixes in order to address
patch conflicts with non-security patches. PSUs always contain the CPU for that respective
quarter.
Rolling Patches
A rolling patch allows one node to be patched to the latest version, while other nodes continue
to use the older version and provide business service.
Rolling patches are enabled by using a locally accessible, non-shared file system to store the
software files, known as Grid Home. Each node maintains its own copy. With this technique on
a 5-node cluster, there would be 5 copies of the Oracle Grid Infrastructure software—that is, 5
copies requiring disk space and 5 copies to update. This technique enables rolling patches and
rolling upgrades by allowing two versions in the cluster at the same time.
Rolling patches cannot be done when the Oracle Grid Infrastructure software files are stored in
a shared cluster file system in which a single copy of the software is shared among all nodes. A
single copy requires much less disk space and a single copy to patch or upgrade. However, to
patch or upgrade, the software must be stopped. Stopping the Oracle Grid Infrastructure
software also requires all databases, applications, and services that depend on Oracle Grid
Infrastructure to be stopped. This technique requires a complete outage with down time to patch
or upgrade.
Note: A patchset that can be rolled for the Grid Infrastructure may not be able to be rolled for
the RDBMS.
By default, Oracle supports rolling upgrade for patches for Oracle Grid Infrastructure. This is
available for patchsets, bundle patches, and most individual patches.
Rolling upgrades work for Oracle Clusterware and Oracle Automatic Storage Management
(Oracle ASM) (collectively known as Oracle Grid Infrastructure), but not for the Oracle Database
software.
You cannot upgrade to a new major release, maintenance release, or patch release of Oracle
Database using the rolling upgrade process in an Oracle Real Application Clusters (Oracle
RAC) environment.
You can, however, apply most database patches (sometimes referred to as one-off patches)
and all database patch set updates and critical patch updates (PSUs and CPUs) using a rolling
patch install in an Oracle RAC environment.
> pwd
/ora/install/4933522
l.
ai
gm
Then use the following OPatch command:
> opatch query is_rolling_patch
7@
...
71
Query ...
n2
Please enter the patch location:
/ora/install/4933522
wa
....
Rolling Patch: True.
it
With the out-of-place upgrade, both versions of the software are present on the nodes at the
same time, in different Grid homes; but only one is active.
Installing Oracle Grid Infrastructure in a separate home before the upgrade reduces the planned
outage time required for cluster upgrades, which assists in meeting availability service-level
agreements. After the software is installed, you can upgrade the cluster by stopping the
previous version of the software and starting the new version node by node (known as a rolling
upgrade).
Each node maintains a “software version” that indicates the version of the Oracle Grid
Infrastructure that is active on the machine. The software version is stored in the OCR. There is
also a clusterwide “active version” for the entire cluster that reflects the version of the
communication protocols and shared disk data structures that are currently in use on the
cluster.
The active version is the lowest version anywhere in the cluster. It applies to the cluster and not
an individual node. The active version is not updated until the last node has been updated to the
newest version. You can check the active version with the following command:
Permanently operating Oracle Grid Infrastructure at different versions is not supported. This is
allowed only for a short duration—that is, the time it takes to apply the patchset or patch to the
cluster.
Note: The version of Oracle Grid Infrastructure must be greater than the version running other
Oracle products such as the Real Application Clusters (RAC) database.
About OPatch
OPatch is a Java-based utility that allows the application and rolling back of interim patches to
an Oracle product. It is not used to apply patchsets. OPatch is included with the Oracle Grid
Infrastructure 11g installation. When patching an Oracle Grid Infrastructure installation using
the rolling-patch method, the user is prompted for which nodes to patch during the patch
installation process. A rolling patch is identified inside the patch and cannot be enabled when
invoking the patch tool.
OPatch also supports a minimum down-time mode. An inventory of all patches that have been
installed is maintained by OPatch.
Patching Oracle Grid Infrastructure has special considerations: Part of the software is owned by
root and part is owned by the account used to install the product (typically grid). To patch the
root-owned components, the Grid Infrastructure software will need to be unlocked by running
special rootcrs.pl . After patching, the clusterware software will need to be returned to its
previous secure settings by running rootcrs.pl script.
After you set the location of the new recovery area, existing backups remain in the old recovery
area and count against the total disk quota of the recovery area. The backups are deleted from
the old recovery area when space is needed. These backups are usable by RMAN. It is not
necessary to move legacy backups to the new Oracle ASM recovery area unless you need disk
space. To free space consumed by files in the old recovery area, you can back them up to tape
or migrate them to the Oracle ASM recovery area.
1. Back up the database and server parameter file, and disable Oracle Flashback Database.
1. Restore files to Oracle ASM, recover the database, and optionally migrate the fast recovery
area to Oracle ASM.
1. With the target database open, edit the initialization parameter control_files and
db_create_file_dest to point to the ASM disk group +DATADG. Also configure
db_recovery_file_dest to point to the ASM disk group +FRADG:
3. From an RMAN session, copy one of your control files from the local file system to its new
location in ASM. The new control file will be copied to the value specified in the initialization
parameter control_files:
4. From an RMAN session, copy the database files from the local file system to ASM, switch
the database to copy, perform recovery and open the database.
RMAN> run
{
Set newname for tempfile 1 to ‘+DATADG’;
Switch tempfile all;
}
5. Drop and recreate on ASM any online redo logfiles that are still currently on the local file
system
6. Relocate the SPFILE from the local file system to an ASM disk group
Delete spfilesid.ora and initsid.ora from the ORACLE_HOME/dbs directory and create a new
initsid.ora with the following line of content:
SPFILE='+DATADG/spfilesid.ora‘
SQL> STARTUP;
At this point, the target database is open with all of its datafiles, controlfiles, online redo logfiles,
tempfiles, and SPFILE stored in ASM. You may optionally remove the old files related to
database that were stored on local Filesystem.
In this chapter you will use rconfig utility to convert a single instance database to Oracle RAC.
rconfig is a command line tool introduced in Oracle Database 10g R2 to convert Single-Instance
10g R2 Database to RAC(Real Application Cluster). rconfig is located at $ORACLE_HOME/bin/.
rconfig takes in a xml input file and converts the Single Instance database whose information is
provided in the xml. A sample xml can be found
$ORACLE_HOME/assistants/rconfig/sampleXMLs/ ConvertToRAC_AdminManaged.xml
$ ./rconfig ConvertToRAC_AdminManaged.xml
The Convert verify option in the ConverttoRAC_AdminManaged.xml file has three options:
Convert verify="YES": rconfig performs checks to ensure that the prerequisites for
single-instance to RAC conversion have been met before it starts conversion
Convert verify="NO": rconfig does not perform prerequisite checks, and starts
conversion
Convert verify="ONLY" : rconfig only performs prerequisite checks; it does not start
conversion after completing prerequisite checks
Restarting Rconfig
After running RCONFIG in Convert verify="YES" Mode, if you get any fatal error(e.g. Disk space
not available, Some issue with 10g Parameters, Shared Storage Issues) which exits the rconfig
tool & stops the conversion.
You can restart the rconfig tool again by running the command
$ ./rconfig ConvertToRAC_AdminManaged.xml
This will perform the clean-up operation on converted Instance or will delete the files created by
earlier run. It again starts the process of conversion from start i.e. taking rman backup of Single
Instance. After the conversion you can check the filesystem to verify if there is anything left from
the previous failed run.
< n:TargetDatabaseArea>+DATADG</n:TargetDatabaseArea>
< !--Specify Fast Recovery Area to be configured for rac database. -->
< n:TargetFlashRecoveryArea>+FRADG</n:TargetFlashRecoveryArea>
< /n:SharedStorage>
< /n:Convert>
< /n:ConvertToRAC>
< /n:RConfig>
$ rconfig ConvertToRAC_AdminManaged.xml
Converting Database "test" to Cluster Database. Target Oracle Home:
/u02/app/oracle/database/12.1. Database Role: PRIMARY.
Setting Data Files and Control Files
Adding Database Instances
Adding Redo Logs
Enabling threads for all Database Instances
Setting TEMP tablespace
Adding UNDO tablespaces
Adding Trace files
Setting Fast Recovery Area
Updating Oratab
Creating Password file(s)
Configuring Listeners
Configuring related CRS resources
Starting Cluster Database
..