2008 341 Norris PPR
2008 341 Norris PPR
2008 341 Norris PPR
In the sections that follow, you will learn: An architectural overview of RAC System requirements for a VMWare Cluster VMWare product overview Configuration and installation processes Paper 341
DBA, Clustering Challenges and common mistakes This paper is intended to be used by Oracle DBAs that have at least a little bit of Linux knowledge and have installed some version of Linux at least once. RAC experience is also helpful, but most DBAs will probably be able to complete the installation without having done it before. This document is just a recipe on how to install the products, not how to use them. If youre new to RAC and need a more basic introduction to RAC, check out the RAC For Beginners: The Basics whitepaper and presentation found online at http://www.dannorris.com/professional-activities/ under the Presentations section.
SYSTEM ARCHITECTURE
The diagram above shows a simple RAC cluster architecture. A minimal cluster consists of multiple servers, a private network, a public network, and shared storage. Most DBAs are familiar with the single-instance configuration in which storage is not shared between nodes and all access to the database comes from the single instance. Sharing the same storage poses several challenges. First, the operating system must be able to handle accesses from other nodes at the same time as it accesses the storage. In the early days of Oracle Parallel Server, this was addressed by using raw devices. The operating systems were accustomed to file systems that supported access from exactly one host. When only one host was involved, the host could manage all locking and synchronization from a single operating system kernel. With multiple hosts, the filesystem access must be coordinated between all hosts sharing the filesystem. This introduces some latency and, until fairly recently, cluster-wide filesystems were very uncommon. Today, several vendors offer cluster filesystems that are supported by Oracle RAC. Oracle also offers its OCFS filesystem for Linux and Windows with reported plans to make it available for more platforms in the future. Sharing a database also creates challenges. Being able to handle the cross-host locking required to ensure database integrity is the first and most important challenge. Previously, that challenge was addressed by exchanging blocks that required serialization via disk. This was known as a block ping and was a major limiting factor for OPS. With Cache Fusion technology introduced in Oracle9i, the block ping was put on the endangered species list and was only required for a small, infrequently seen set of circumstances. The additional hardware required for a RAC cluster usually consists of a few additional networks to support the private interconnect. The private interconnect can be made up of multiple physical network interfaces. It is common to utilize two or more gigabit Ethernet interfaces for the interconnect. In past versions, expensive and proprietary interconnect technologies were used. These proprietary interconnects were required because gigabit Ethernet created too much latency and did not have the bandwidth necessary to support some environments. With advancements in Ethernet technology and the ability to utilize multiple interconnects, gigabit Ethernet is the most common interconnect in use today. Infiniband is a relatively new technology that was recently (10.2.0.3) certified for use as an Oracle private interconnect. Infiniband promises to be a significant player in the RAC environment because of its impressive technical specifications. Finally, before we move on, lets review some points about databases and instances. Oracle RAC is, by definition, a multiinstance database. There is one database. Thats o-n-e, as in 1, uno. That (one) database resides on shared storage and is accessed concurrently (thats at the same time) by multiple instances that run on separate hosts. Read that sentence againit is 2 Paper 341
DBA, Clustering critical that you understand what youre building and that last sentences sums it all up. Database and instance are defined in the table above. Its a good idea to review their meanings since oftentimes in normal Oracle environments, the terms database and instance tend to be used interchangeably. Thats fine, but it will benefit you to try to use those words properly in the RAC environment context while youre learning how the environment works. As we move on to the cluster implementation processes, keep in mind the terminology and big picture of what were trying to build. For me, it always helps to refer back to the overall architecture design were trying to create frequently during complex installations. The implementation topics covered in the remainder of this document are: System requirements for a VMWare Cluster VMWare product overview Configuration and installation processes Challenges and common mistakes
RECOMMENDED CONFIG
More is almost always better when it comes to CPU and memory on the system. Of course, disk space is usually easy to come byI dont know that you can easily buy a 200 GB (or smaller) drive these days, so youll likely have more. My test server has about 1 Tb of disk space and 4 GB of RAM. For my testing, I used a small Dell server, similar in configuration to a high-end desktop system. Its specifications are in the table below: My server configuration CPU 1 x 3 Ghz CPU with HyperThreading (appears as two CPUs in OEL 4) Memory 4 Gb RAM Disk 931 Gb formatted More resources are always welcome, but I found this little server serves my needs pretty well.
Paper 341
DBA, Clustering
VMWARE WORKSTATION
VMWare Workstation provides the capability to create and run virtual machines on a workstation. It is a licensed product and also has support available for purchase. Virtual machines created with VMWare Workstation can be played with VMWare Player. It is available for evaluation and purchase via the VMWare website. VMWare Workstation has a limitation related to disk sharing, so thats one reason were using VMWare Server (which has no such restriction).
VMWARE SERVER
VMWare Server is a free product capable of creating and running multiple virtual machines concurrently (up to the limits of the hardware). VMWare Server has some additional server-side features and management tools that make it an attractive solution for server-based VM management. With some web-based tools and easy-to-use GUI management tools as well, multiple people can utilize a server for VMs for testing. Advanced features like multiple snapshots, teaming, automated cloning, and clustering are not available with VMWare Server. Support can be purchased for VMWare Server. This is the product well use to build our RAC cluster.
PREPARE HOST OS
Theres not much to do in this step other than install the host OS. I didnt see any need to change much from a default installation. The minimum requirements are buried in the VMWare Server documentation (it isnt easily possible to link directly to them due to the way that VMWares online documentation is organized). See the earlier section for hardware configuration recommendations. No special kernel parameters, groups, or users are required to support the VMWare software. If you have support, its always a good idea to install the latest updates available.
Once thats installed (should take just a few seconds), you next need to configure the virtual networks that VMWare Server will provide to the guest OSes. Note that were using version 1.0.4. The menus and wizards changed slightly from version 1.0.3 to 1.0.4, so while either version should work okay, you might notice differences if you use a version other than 1.0.4 on Linux. The virtual network setup happens as part of the server settings configuration in the next step. 4 Paper 341
DBA, Clustering
At this point, you should be able to start the graphical VMWare Server console. There are several ways to start the console: On the server console, you can run vmware at a command prompt. On the server console, you can run the run the shortcut found on the Applications -> System Tools -> VMWare Server Console. Or, you can download the VMware Server Windows client package from the same download site where you obtained the VMWare Server software and install the Windows VMWare Console to manage your VMWare Server virtual machines over the network from your Windows desktop (this is what I typically do).
CREATE VM
With the server console fired up, next step is to create the first virtual machine that will serve as the first RAC node. Ive found it much easier to install the software if you are able to download the ISO images for Linux to the VMWare Server system directly. You can obtain the Oracle Enterprise Linux (OEL) download at http://edelivery.oracle.com/linux. Were using OEL 5 for this installation. Create a new virtual machine by following these steps: 5 Click on the New Virtual Machine button on the console or going to File -> New -> Virtual Machine. This starts the New Virtual Machine Wizard. We choose the Custom configuration and then choose Linux -> Red Hat Enterprise Linux 4 (current VMWare Server doesnt list version 5). OEL is based on Red Hat and thats the reason that this choice is the best one. For the name, enter RAC1 (the location automatically defaults appropriately) Uncheck the Make this virtual machine private checkbox. Choose defaults for the Startup/Shutdown options (these settings can be changed later). Choose One CPU Allocate no more than 40% of the host RAM to this VM. In my case, I have approximately 4 Gb available on my host, so Im going to allocate 1024 Mb to this VM. Choose Use bridged networking for the network type for nowwell add another NIC later, after the VM is created. Accept the default I/O adapter settings Choose Create a new virtual disk Choose SCSI for the disk type Choose a Disk size of 10.0 GB and uncheck the Allocate all disk space now to conserve a little more disk space. Leave the split option checked for ease of maintenance. Choose RAC1.vmdk as the disk file name Paper 341
DBA, Clustering Finish the wizard Back on the main screen for this VM, choose Edit virtual machine settings Remove the Floppy device Add a new device, type Ethernet Adapter. Choose Bridged for network type. Choose OK to exit the VM settings dialog. Now were ready to install the guest OS into this VM.
INSTALL GUEST OS IN VM
I prefer to use ISOs loaded on the VMWare Server. To do that, you just need to edit the CD-ROM virtual device to reference the location of the first ISO file. Edit the virtual machine settings, choose the CD-ROM and then select the Use ISO image radio button and browse for the .iso image representing the first CD. Check the connect at power on box. With the CD in place, boot the new virtual machine by clicking Start this virtual machine. Once booted, follow these steps to install the guest OS: Install as you would normally install any server. For detailed instructions on server installation, Tim Hall has another fine article posted at http://www.oracle-base.com/articles/linux/OracleEnterpriseLinux5Installation.php that walks through the installation step-by-step. When prompted, choose the following package groups: GNOME Desktop Environment Editors Graphical Internet Text-based Internet Development Libraries Development Tools Server Configuration Tools Administration Tools Base System Tools X Window System hostname: rac1.dannorris.net IP Address eth0: 192.168.169.51 (public address) Default Gateway eth0: 192.168.169.1 (public address) IP Address eth1: 192.168.150.51 (private address) Default Gateway eth1: none
For our testing, its easiest to disable the built-in firewall and also disable the SELinux subsystem. Disabling these components will make it easier to configure and use the system. Once the initial installation is complete, install the following packages from the installation media (you need to mount/umount the proper CD images as indicated:
# From Enterprise Linux 5 Disk 1 cd /media/cdrom/Server rpm -Uvh binutils-2.*
Paper 341
DBA, Clustering
rpm -Uvh rpm -Uvh rpm -Uvh rpm -Uvh rpm -Uvh rpm -Uvh rpm -Uvh cd / eject elfutils-libelf-0.* glibc-2.* glibc-common-2.* libaio-0.* libgcc-4.* libstdc++-4.* make-3.*
# From Enterprise Linux 5 Disk 2 cd /media/cdrom/Server rpm -Uvh compat-libstdc++-33* rpm -Uvh elfutils-libelf-devel-* rpm -Uvh glibc-headers* rpm -Uvh glibc-devel-2.* rpm -Uvh libgomp* rpm -Uvh gcc-4.* rpm -Uvh gcc-c++-4.* rpm -Uvh libaio-devel-0.* rpm -Uvh libstdc++-devel-4.* rpm -Uvh unixODBC-2.* rpm -Uvh unixODBC-devel-2.* cd / eject # From Enterprise Linux 5 Disk 3 cd /media/cdrom/Server rpm -Uvh sysstat-7.* cd / eject
With the installation complete and packages installed, we can move on to configuring the Oracle prerequisites.
Add the following lines to the /etc/sysctl.conf file (Note that on OEL 5, many of the parameters that you had to add in previous versions are now in the file by default):
kernel.shmmni = 4096 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default=4194304 net.core.rmem_max=4194304 net.core.wmem_default=262144
Paper 341
DBA, Clustering
net.core.wmem_max=262144
Once the /etc/sysctl.conf file is modified, force the new parameters into effect immediately with sysctl p Add the following lines to /etc/security/limits.conf:
@dba @dba @dba @dba soft hard soft hard nproc nproc nofile nofile 2047 16384 1024 65536 /lib/security/pam_limits.so pam_limits.so
Add the necessary groups and user to the system with the following commands:
groupadd -g 500 oinstall groupadd -g 501 dba groupadd -g 502 asmadmin useradd -u 500 -g oinstall -G dba,asmadmin -s /bin/bash oracle passwd oracle mkdir -p /u01/app/oracle /u01/crs/oracle chown -R oracle:dba /u01
Set default shell variables by adding the following to the /home/oracle/.bash_profile file:
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE PATH=/usr/sbin:$PATH; export PATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/bash" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi
Once these changes are in place, login to the oracle account and run ssh-keygen t dsa to create an SSH key. Do not specify a pass phrase for the key. After the key is created, run cp .ssh/id_dsa.pub .ssh/authorized_keys to prepare us for the host equivalency required later for the installations. Once the prerequisites are configured, you may wish to install the VMWare Tools. It is an optional step that wont affect the success of the Oracle installations, but may make things easier to manage and may make console usage easier since it will install video and mouse drivers that are generally better than the Linux defaults.
On the host (VMWare Server host), create a directory to hold the virtual shared disk files: mkdir p
/u01/vmware/vms/RAC-shared-disks
Using the VMWare Server Console, edit the virtual machine settings. Add a device of type Hard Disk -> Create a new virtual disk -> SCSI -> Disk size 10 Gb (uncheck Allocate all disk space now) -> Name: /u01/vmware/vms/RACshared-disks/ocr1.vmdk -> Click Advanced button On the Advanced dialog, Choose SCSI 1:0 as the node for this disk and select Independent and Persistent for the mode. Repeat this process for additional disks, until youve configured all disks as follows (settings not listed in the table should match the example above: Name
/u01/vmware/vms/RAC-shared-disks/ocr1.vmdk
Size 10 Gb
Size 10 Gb 10 Gb 10 Gb 10 Gb
Next step is to hand edit the virtual machines configuration file to add some parameters. The resulting file should contain these settings (as well as some others). Some of the following lines may already exist. If they already exist, their values should be set as shown below and if they dont exist, add them. Leave any other lines not shown below as they are found.
disk.locking = "FALSE" diskLib.dataCacheMaxSize = "0" diskLib.dataCacheMaxReadAheadSize = "0" diskLib.dataCacheMinReadAheadSize = "0" diskLib.dataCachePageSize = "4096" diskLib.maxUnsyncedWrites = "0" scsi1.sharedBus = "VIRTUAL" scsi1.present = "TRUE" scsi1:0.present = "TRUE" scsi1:0.fileName = "/u01/vmware/vms/RAC-shared-disks/ocr1.vmdk" scsi1:0.mode = "independent-persistent" scsi1:0.deviceType = "plainDisk" scsi1:0.redo = "" scsi1:1.present = "TRUE" scsi1:1.fileName = "/u01/vmware/vms/RAC-shared-disks/voting1.vmdk" scsi1:1.mode = "independent-persistent" scsi1:1.deviceType = "plainDisk" scsi1:1.redo = "" scsi1:2.present = "TRUE" scsi1:2.fileName = "/u01/vmware/vms/RAC-shared-disks/asm1.vmdk" scsi1:2.mode = "independent-persistent" scsi1:2.deviceType = "plainDisk" scsi1:2.redo = "" scsi1:3.present = "TRUE" scsi1:3.fileName = "/u01/vmware/vms/RAC-shared-disks/asm2.vmdk" scsi1:3.mode = "independent-persistent" scsi1:3.deviceType = "plainDisk" scsi1:3.redo = "" scsi1:4.present = "TRUE" scsi1:4.fileName = "/u01/vmware/vms/RAC-shared-disks/asm3.vmdk" scsi1:4.mode = "independent-persistent" scsi1:4.deviceType = "plainDisk" scsi1:4.redo = "" scsi1.virtualDev = "lsilogic"
Next, you need to boot the virtual machine again so that you can partition the new disks. Once booted, you can partition the disks with these commands for each disk (/dev/sdb through /dev/sdf): fdisk /dev/sdb, then enter n, p, 1, <return>, <return>, and w. That will create a single partition that is the size of the whole disk on each disk. In order for the oracle user to be able to access these partitions, you need to add these lines to the /etc/rc.local:
chown chown chown chown chown chmod chmod chmod chmod oracle:oinstall oracle:oinstall oracle:oinstall oracle:oinstall oracle:oinstall 600 /dev/sdb1 600 /dev/sdc1 600 /dev/sdd1 600 /dev/sde1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
Paper 341
DBA, Clustering
chmod 600 /dev/sdf1
See the notes later in this document about using udev to configure device access. udev is a better and more supportable method and the notes mentioned in that section will provide instructions on how to configure and use udev on your system. For this installation, it isnt needed to get our environment working and stable. There are no raw devices to create on OEL 5, so were done configuring our shared storage at this point. Next, we will create our second cluster node by cloning the first node. So, shut down rac1 now (init 0) and follow the next section to clone the rac1 VM to create the rac2 VM.
CLONE THE VM
In our process, the next step is to create the second virtual machine. We will do this by copying the first one to clone it. To perform the clone, follow these steps:
Shut down the rac1 virtual machine cleanly (init 0) if not already shut down Copy all virtual machine file to create a new directory on the VMWare Server host: cp R /u01/vmware/vms/RAC1
/u01/vmware/vms/RAC2
Edit the /u01/vmware/vms/RAC2/RAC1.vmx file to change displayName = RAC1 to be RAC2 Using the VMWare Console, open the new VMs configuration file. Then start the new rac2 VM. Leave the rac1 VM down. During RAC2 power on, answer Create when prompted about handling the identifier. Once RAC2 is booted, login to the console and update the following files to set the correct network information for the rac2 node:
/etc/sysconfig/network (change hostname) /etc/sysconfig/network-scripts/ifcfg-eth0 (replace with following contentssubstitute your HWADDR
as appropriate):
DEVICE=eth0 ONBOOT=yes HWADDR=00:0c:29:e8:4f:5e NETMASK=255.255.255.0 IPADDR=192.168.169.52 TYPE=Ethernet /etc/sysconfig/network-scripts/ifcfg-eth1 (replace with following contentssubstitute your HWADDR
as appropriate):
DEVICE=eth1 ONBOOT=yes HWADDR=00:0c:29:e8:4f:68 NETMASK=255.255.255.0 IPADDR=192.168.150.52 TYPE=Ethernet
Once the files are changed, you need to restart the networking by running service network restart. Now rac2 should have the correct hostname (check with the hostname command). It is now safe to restart the RAC1 virtual machine. Once both machines are up again, login to each of them and make sure that you can ping each of the following hostnames from each node: rac1 rac1-priv rac2 rac2-priv
DBA, Clustering Note that once the installations are completed, there is no requirement for host equivalency (other than convenience for maintenance operations), so you may disable the host equivalency established in this section once all installations are complete. To establish host equivalency, follow these steps: Login to rac1 as the oracle user. Run ssh-keygen t dsa. Take all default settings, do not enter any passphrase. Run cp .ssh/id_dsa.pub .ssh/authorized_keys For each host in the cluster, ssh to that host to connect by running the following commands in succession from the oracle account. Note that you will be prompted to accept each hosts key; respond yes when prompted.
ssh ssh ssh ssh rac1 date rac1-priv date rac2 date rac2-priv date
With all settings in place, now run these commands as the oracle user from rac1: cd ; scp rp .ssh rac2:. Now, when you run the ssh statements above, you should see nothing but the output from the date commandno prompts of any kind. At this point, weve done all the OS preconfiguration and have all the necessary items in place to begin installing software. If youre cautious and want to save some time in the event of a catastrophe, this is a good point to shut down the virtual machines and take a copy of them for backup purposes.
INSTALL CLUSTERWARE
Finally, we can start installing some software. In a production installation, wed usually take some extra time to run the cluster verification utility (CVU) and double-check that all prerequisites are in place. If youve followed the recipe in this document up until this point, we can save a little time and not run that utility. Well move right in to running the clusterware installer. Follow these steps to install Oracle Clusterware on both nodes. Make a Oracle Clusterware 11g installation CD (or the download of the CD contents) available to the rac1 virtual machine. If youve followed along carefully to this point, you wont have a graphical console. If thats true, then youll probably want to start VNC in order to perform the graphical installations. To start VNC, while logged in to rac1 as the oracle user, create the following file named $HOME/.vnc/xstartup (you might have to create the $HOME/.vnc directory first):
xsetroot -solid grey gnome-session &
Then, you can start the VNC server by running vncserver from the command prompt. The first time you run it, youll be prompted to set a password for your desktop. I usually use the same password as the login password for the oracle user. Once VNC is running, connect to the server with the a VNC Viewer. Then, ensure your environment has the
ORACLE_BASE environment variable set to /u01/app/oracle.
After confirming the proper environment settings, run these commands: cd <download_location>/clusterware;
./runInstaller
On the Specify Inventory directory screen, take the defaults (/u01/app/oraInventory and oinstall). On the Specify Home Details screen, take the defaults of OraCrs11g_home and /u01/app/11.1.0/crs All prerequisite checks should succeed. On the Specify Cluster Configuration screen, the local node (rac1) settings should already be displayed. Add the information for rac2 to the screen using the appropriate names for Public, Private and Virtual. On the Specify Network Interface Usage screen, set eth0 to be Public and leave eth1 as Private. On the Specify OCR Location screen, choose External Redundancy and enter /dev/sdb1 for the OCR location.
11
Paper 341
DBA, Clustering On the Specify Voting Disk Location screen, choose External Redundancy and enter /dev/sdc1 for the voting disk location. On the Summary screen, ensure that the Cluster Nodes list includes both nodes and then click Install. Once the installations finish, run the root scripts as prompted. Remember to run them in sequencenot concurrently. After the root scripts are run, click OK and then the final assistants run. All assistants should run without issue or error.
On the Specify Hardware Cluster Installation Mode screen, choose Cluster Installation and click Select All to choose all nodes. All prerequisite checks should be successful. On the Select Configuration Option screen, choose Configure Automatic Storage Management (ASM) and enter an ASM SYS password. On the Configure Automatic Storage Management screen, you may first have to Change Disk Discovery Path to /dev/sd* in order to see your candidate drives. Then, create one External Redundancy disk group named DATA and it uses all three drives (/dev/sdd1, /dev/sde1, and /dev/sdf1), so check the boxes next to all of these drives. On the Privileged Operating System Groups screen, take the defaults of OSDBA: dba, OSOPER: oinstall, OSASM: asmadmin. On the Oracle Configuration Manager Registration screen, click next to bypass registration. Click Install on the summary screen to start the installation. Go get some caffeine and wait.
Once the installation finishes, youll be prompted to run the root.sh script on each node. Run them sequentially (either node first). Take the default for the local bin directory as prompted by the root.sh script. At this stage, the ASM instances should be running on each node (you should see lots of asm_XXXX_+ASM? processes running on each node. The crs_stat t command should also show ASM and listener resources configured for each node.
12
Paper 341
DBA, Clustering Then run the dbca tool by entering: dbca & On the opening screen, choose database type Oracle Real Application Clusters database Choose the operation Create a Database On the node selection screen, choose Select All On the templates screen (step 3 of 19), choose General Purpose or Transaction Processing For the Global Database Name enter RAC.world and let the SID Prefix default accordingly (default should be RAC) On the configuration assistant screen (step 5 of 18), choose the default of Configuration Enterprise Manager and you do not need to enable alert notifications nor daily disk backups For simplicity (instead of security), choose Use the Same Administrative Password for All Accounts on step 6 of 18 and enter a password. On the storage selection screen, choose Automatic Storage Management (ASM) Select the DATA disk group and click next on the disk group selection screen. Use Oracle-Managed Files on the Database File Locations screen (step 9 of 17) No need to configure a flash recovery area nor archiving at this time You may install the sample schemas if you would likeI didnt. I chose to use the default and suggested Typical memory configuration. You can use whatever youd like, but ensure a minimum SGA size of 400 Mb. On the Character Set tab, I prefer to Use Unicode. Keep the enhanced 11g security settings on step 13 of 16 It is recommended to leave the automatic maintenance tasks enabled There are no required changes on the Database Storage screen On the final screen, click Finish, then click OK on the final summary. Wait. Once the DBCA is complete, it will display the summary confirmation screen. Make a note of the URL for the DB Control website. When you click Exit on the summary screen, the instances are started and it takes a while before any messages are displayed, so be patient. All done! RAC database is created, services created and everything should be running.
DBA, Clustering device identitifers unique to each device in order to be certain that the right devices are identified from each node. This configuration document didnt use udev as vmware keeps the devices fairly stable. If you do not configure udev or device mapper, the cluster verification utility may report success, but after rebooting the nodes (especially when adding or reconfiguring storage), you may find that the cluster no longer comes up because the device names all changed during the reboot cycle. To learn more about how to configure udev, see the following notes on Metalink: 443996.1 How to map raw device on RHEL5 and OEL5 414897.1 How to setup /etc/udev/rules.d for OCR and Voting for RAC on SLES10/RHEL5 and EL5 371814.1 Using udev with Oracle Architecture (RAC & ASM) Red Hat 4.0 456239.1 Understanding Device-mapper in Linux 2.6 Kernel 357472.1 Configuring device-mapper for CRS/ASM 465001.1 Configuring raw devices for Oracle 10gR2 (10.2.0) Clusterware on Red Hat/Enterprise Linux 5 (RH/EL5)
DBA, Clustering The only other setup that can be somewhat challenging is the shared disk setup. Without going in to much detail, youll see in this document that you must manually edit the VMs configuration file to allow the disks to be accessible to two VMs at the same time. The parameters added and changed may not be documented well by VMWare, but they have been tested and work well for RAC environments on VMWare. Forgetting to make these manual edits generally results in a completely nonfunctional RAC environment since the clusterware requires access to shared disks from all nodes.
REFERENCES
15
Paper 341