device file is /dev/dsk/c1t0d0, hardware path 16/0.0; it is not being used.

The disk
types we will be using are the 2-GB hot swappable disks.

The volume group contains five logical volumes that are each 200 MB in size
These logical volumes contain filesystems that house an Oracle database (see
Listing 1). We have decided that we want this data protected from hardware
MirrorDisk/UX failure, and want to use MirrorDisk/UX to accomplish this. The first task is to find
an unused disk that is at least the same size as the disk currently in the volume
group. This disk would preferably be on a different SCSI channel, as this will
by Galen Scalone provide for redundancy not only in case of disk failure, but for channel, and/or,
cable failure. We will add this disk to the /dev/test volume group, then create
MirrorDisk/UX is a product developed by HP that enables us to store multiple mirror copies of the existing logical volumes on the new disk. Once we have
copies of data on separate disks so that, in the event of hardware failure, the data is chosen a disk we do the following:
still accessible, and the system will continue to run. MirrorDisk/UX (HP part
number B2491BA) does not come with the standard HP-UX operating system. pvcreate /dev/rdsk/c1t0d0 This makes the disk an LVM disk.
Please note that you will need double your amount of disk space if you decide to
mirror your data. Besides being a mirroring tool, it also offers us a tool to move vgextend /dev/test /dev/dsk/c1t0d0 This puts the new disk into the /dev/test
data, load balance disks or SCSI channels, and do backups with little or no volume group
downtime. This article is written for administrators who have an understanding of
Logical Volume Manager, as MirrorDisk/UX will add functionality to two LVM lvextend m 1 /dev/test/data1 /dev/dsk/c1t00 This makes one mirror copy
commands, lvextend, and lvreduce, and add the commands lvsplit, lvmerge, lvsync,
of /dev/test/data1 on the disk /dev/dsk/c1t0d0, and should be repeated for all
and vgsync. We wil cover the following topics:
logical volumes in the volume group.
how to mirror a non-root volume group and a root volume group
Now that the mirrors are in place, use lvdisplay v /dev/test/data1 to look at
how to recover a volume group from hardware failure
the logical and physical extents of the logical volume /dev/test/data1 (see
using mirroring to move data within a volume group
load balancing your disks and/or SCSI channels using mirrors Listing 2). Notice one logical extent being mapped to two physical extents, and that
splitting mirrors to back up your data all extents are current, or fully synced. At this point all logical volumes in /dev/test
are mirrored with disks on different channels and protected from disk failure, SCSI
card failure, and cable failure.
This will all be done from the command line.
Backups Using Mirrors
Background
MirrorDisk/UX offers system administrators the ability to split a logical volume
Let's recall the logical extent to physical extent relationship. When LVM is
into two logical volumes and then mount the mirror copy elsewhere on the system
working with physical disk drives, it divides those drives into sections called
for backup. The original volume stays online, while the split volume will go
physical extents. By default, a physical extent is 4 megabytes, but the size is
offline, its extents not being updated. Once you have split the mirror, you should
configurable at volume group creation. LVM allocates an address for each extent
check the mirror's filesystem for errors, make a mount point for it, and mount the
starting at the beginning of the disk, excluding a reserved header area. The LVM
filesystem to the new mount point.
software builds internal tables that hold these maps of physical extents.
We will use the /dev/test volume group from the previous section, since it is now a
When we create a logical volume, LVM creates logical extents that map to
fully mirrored volume group.
physical extents on disk. In a non-mirrored LVM environment, every logical extent
will be mapped to one physical extent. The LVM software keeps an internal table
of logical to physical extent mappings. For example, the Oracle database needs to be backed up every night, but cannot be
down for backup for an extended period of time. So we need to shut the database
down and have it started again in the least amount of time possible. The way to
When we use MirrorDisk/UX, we map one logical extent to two or three physical
accomplish this is to shut the database down, split the mirror copies of the logical
extents, depending on the number of mirrors we would like to have.
volumes, then restart the database. Once the database is restarted, we will mount,
and back up the mirror. This requires only a few minutes of downtime.
Mirroring a Non-Root Volume Group
After the database is down, and there is no activity on the disks in the volume
In the following examples we will assume that a volume group named /dev/test group, we do the following:
contains one disk, device file /dev/dsk/c0t0d0, with a hardware path of 8/0.0. There
is one other disk on the system, on another SCSI controller or channel, and its
1/8 2/8
lvsplit /dev/test/data1 This command splits the /dev/test/data1 logical for a in 1 2 3 4 5 6 7 8
volume, and creates a logical volume called /dev/test/data1b using the extents from do
the mirror copy of the volume (see Listing 3).We will run this command for all
logical volumes in the volume group. After all volumes are split, we restart the lvextend m 1 /dev/vg00/lvol${a} /dev/dsk/c1t0d0
Oracle database.
done
fsck F vxfs /dev/test/rdata1b This command checks the filesystem on the
split volume for errors, and will also be run for all newly created logical volumes. At this point, all of the logical volumes in vg00 are mirrored to separate disks. This
means that each logical extent maps to two physical extents, one set of extents on
mkdir -p /oracle.backup/data1 This creates a directory where we will mount one disk, and the other set of extents on the other disk. LVM will write to both sets
the mirror volumes. We will make a directory for each logical volume in the of extents at the same time (default). Should you lose one disk, the system still has
volume group. a full copy of the logical volumes on the other disk to read from, and to write to;
therefore the machine continues to run. The system is now protected against a disk
mount /dev/test/data1b /oracle.backup/data1 This mounts the newly failure, a SCSI card failure, or SCSI cable failure.
created /dev/test/data1b filesystem to a mount point called /oracle.backup/data1.
We will mount all mirror volumes to their respective mount points (see Listing 4). Recovery of a Failed Disk in vg00
Then we start a backup.
As system admins, we should expect disks to fail every now and then. The idea
When the backup is complete, we unmount and merge the volumes: behind using MirrorDisk/UX is that if you do have a disk failure or SCSI card
failure, the system still has one copy of data that it needs to operate and will
umount /oracle.backup/data1 This unmounts the split volume. continue to run. It still has one set of extents it can read from and write to, and it
does not matter which set it has--the original or the mirror copy.
lvmerge /dev/test/data1b /dev/test/data1 This command will merge the
mirror copy of the logical volume with the original, removing the separate mirror There are a few ways to find out whether a disk has failed, although vgdisplay v
volume and doubling the number of extents in the original volume. is probably the quickest. When vgdisplay v /dev/vg00 is run, it will show the
logical volumes in the vg00 volume group and the disks in the vg00 volume group.
Mirroring the Root Volume Group When one of the disks in the volume group has failed, vgdisplay will show the disk
as unavailable, and will also show some or all of the logical volumes as being in a
When mirroring the root logical volumes, what you are trying to achieve is a type stale state (see Listing 5).
of high availability. You want the system to continue to run in the event that a root
disk, SCSI controller, or SCSI cable fails. In order to accomplish this, the system Once you determine which disk has failed, you will need to replace it. Since these
will need two copies of the logical volumes in /dev/vg00, on two separate disks. examples are using hot swappable HP disks, replacement should be pretty simple--
These disks will be on separate SCSI channels. So we'll add a bootable disk to the just remove the bad disk and insert a new disk. We will use the same disks as the
volume group, put the boot utilities on that disk, then mirror the logical volumes to previous example, c0t0d0 and c1t0d0, but we'll have them in the /dev/vg00 volume
that disk: group, and say that c0t0d0 has failed. Remove the disk from the cabinet, and
replace with the same size disk. Next is the software portion. Once the new disk is
in place:
pvcreate B /dev/rdsk/c1t0d0 Make the disk a bootable, LVM disk.
vgcfgrestore n /dev/vg00 /dev/rdsk/c0t0d0 This command restores LVM
vgextend /dev/vg00 /dev/dsk/c1t0d0 Add the disk to the volume group.
configuration to the reserved header area on the new disk from the
file /etc/lvmconf/vg00.conf, the volume group configuration backup file.
mkboot /dev/rdsk/c1t0d0 Place LIF area on the disk.
vgchange a y /dev/vg00 This command tries to activate any physical volumes
mkboot a "hpux (16/0.0;0)/stand/vmunix" /dev/rdsk/c1t0d0 Place Auto
in the volume group that had previously been unavailable, or missing.
file in LIF.
vgsync /dev/vg00 This command will re-sync the physical extents of all logical
lvextend m 1 /dev/vg00/lvol1 /dev/dsk/c1t0d0 Create a mirror copy
volumes in the volume group.
of /dev/vg00/lvol1, and put it on disk /dev/dsk/c1t0d0. You will need to repeat this
command for each logical volume in vg00.
Alternatively:
The following is a loop to mirror all of the logical volumes in vg00 consecutively,
lvsync /dev/vg00/lvol1 This command will re-sync one logical volume at a
with minimal keystrokes:
time; repeat for all volumes.

3/8 4/8
At this point your mirror copies are valid again. Notice that in the previous alternate link cables from the new array to the HP.
example, the original disk is the disk that failed, not the mirror. If for some reason
you cannot immediately replace the disk, you will need to make sure the machine, Again run insf e, and pvcreate the disks. Then vgextend the volume group to
at boot time, knows which disk to boot from, in this case the mirror. We would contain the new alternate link disks.
setboot p 16/0.0 to set the primary boot path to point to the mirror disk.
When you add alternate links to a volume group, the order in which you add them
Load Balancing is the way they will be used. Example: You have a disk, c0t0d0, and its alternate
link is c2t0d0. When you add c0t0d0 to a volume group, then add c2t0d0 to the
In some cases you may not want, or need, your data mirrored. If you ever have a volume group, c0t0d0 will be the primary path. If you have 10 disks, and 10
disk or I/O bottleneck, you can use MirrorDisk to move data from one disk to alternate links, you want to alternate primary paths to use both SCSI controllers.
another. There is no need for downtime, since you will create a mirror copy of a This will also help with I/O bottlenecks, as you are using both channels to move
logical volume, then remove the original copy. For instance, we had an Oracle data, instead of only one.
database that was in a volume group that contained disks on a disk array. We had
only two SCSI channels from the HP to the array. With archive logging turned on, vgreduce /dev/test /dev/dsk/c1t0d0 /dev/dsk/c1t1d0 /dev/dsk/c1t2d0 /dev
Oracle was writing its archive logs to a filesystem on the array. When the database This command reduces all of the paths to the disks on one channel, c1, from the
started to get busy, we were getting I/O bottlenecks on the disk that had the archive test volume group, regardless of whether it is a primary or alternate path, leaving
log filesystem on it, which in turn slowed the whole channel down. We also had a one path down one channel to each disk.
Jamaica cabinet on the system that was on a separate SCSI channel and contained
four 4-GB drives. We added one of these disks to the volume group, made a mirror vgdisplay v |grep /dev/dsk/c1 Shows only disks that are on the c1 channel
copy of the archive log logical volume on the new disk, then removed the original in any volume group. After ensuring that there are indeed no disks from this
copy from the array disks, actually moving the filesystem from the array to the disk channel configured into any volume group, we disconnect the c1 cable from the
in the Jamaica cabinet. Now when the system gets busy writing archive logs, it is HP.
writing to a separate disk, on a separate channel, not the array--therefore no more
I/O bottleneck. rmsf H 16.0 -H 16.1 -H 16.2 -H 16.3 -H 16.4 This command removes all
special files for the disks located at the hardware paths specified, or all disks on the
Moving Data from Disk to Disk Within a Volume Group c1 channel (see ioscan).

At one site, we had an HP K-Class server with a disk array attached. We had two At this point, we connect the cable from the new array to the SCSI card we
SCSI channels from the HP to the disk array, one of those channels being alternate disconnected the old cable from, on the HP.
links (see note 1). We had one Oracle instance on the disk array. We needed to
move the data from the one disk array to another newer updated array that the insf ev This command installs special files for new or existing devices.
company had just purchased. The new array was to be configured the same way as
the old array, two SCSI channels, one being an alternate. The following is probably pvcreate /dev/rdsk/c1t0d0 /dev/dsk/c1t1d0... This command makes all
not supported by HP, and if you decide to try this, be sure to have a backup done of the new disks on the new array LVM disks.
before starting.
vgextend /dev/test /dev/dsk/c1t0d0 /dev/dsk/c1t1d0... This command
What we did was vgreduce the alternate links from the volume group, leaving only takes all of the new disks and adds them to the /dev/test volume group.
one path to the disks in the array. After making sure that all disks on the one
channel were not being used, we disconnected that SCSI cable from the HP. The lvextend m 1 /dev/test/data1 /dev/dsk/c1t0d0 This command creates a
system was up and running when we did this. After disconnecting the cable, we ran mirror copy of the /dev/test/data1 on the newly added disk, /dev/dsk/c1t0d0 (on the
rmsf H <hardware path to disk> (see note 2) to remove all of the device files
new array). This is repeated for all logical volumes in the volume group, and new
that existed on that channel. We connected one of the cables from the new array to disks in the volume group.
the card we just disconnected from, then ran insf e (see note 3) to install the new
device files for the new disks. With the new device files installed, we pvcreated the Now we have a copy of the data on both disk arrays.
new disks, and extended the volume group to contain all of the new disks.
lvreduce m 0 /dev/test/data1 /dev/dsk/c5t0d0 This command will
The next step was to mirror every logical volume in the volume group to the new remove the data from the old array, leaving only one copy on the new array. Again
set of disks, and then remove the original copy from the disks on the old array. repeat for all logical volumes, and all disks on c5.
Again this can be done with the system running. After making sure again that none
of the disks on the old array were being used, we vgreduced the old disks out of the
At this point all of the data is on the new array only.
volume group, and disconnected the cables for those disks from the HP. The next
step was to use rmsf again to remove the old device files, and then connect the
vgreduce /dev/test /dev/dsk/c5t0d0 /dev/dsk/c5t1d0... This removes

5/8 6/8
the disks on the c5 channel from the test volume group. When this is complete, we vgextend /dev/informix /dev/dsk/c3t0d0 Adds the disk to the /dev/informix
disconnect the cable for the c5 channel from the HP. volume group.

rmsf H 8.0 -H 8.1 -H 8.2 -H 8.3 -H 8.4 This command removes all special lvcreate n rawvol /dev/informix Creates an empty volume called rawvol in
files for the disks located at the hardware paths specified, or all disks on the c5 the /dev/informix volume group.
channel. At this point we connect the other cable from the new array to the SCSI
card we disconnected the c5 channel from. lvextend L 200 /dev/informix/rawvol /dev/dsk/c3t0d0 Extends the
volume to the /dev/dsk/c3t0d0 disk. The data is already in the volume--simply
insf ev This again is used to create the new device files for the newly added point the Informix link to the new volume, in the new volume group.
disks.
In Closing
vgextend /dev/test /dev/dsk/c5t0d0 /dev/dsk/c5t1d0... this will add the
alternate paths to all of the disks in the new array to the /dev/test volume group. MirrorDisk/UX is a powerful tool that is used to protect our systems from disk
failure, channel failure, and cable failure. With redundant hardware, and
At this point we have successfully moved the test volume group from one disk MirrorDisk/UX, we can make our systems highly available, we can make our
array to another disk array, using mirrors. systems use multiple channels for better I/O performance, and we can back up our
systems and applications with minimal downtime. Some of the more uncommon
We then use pvchange -s /dev/dsk/c5t1d0 /dev/dsk/c5t3d0 to alternate the ways of using mirrors, such as moving volumes around the system to the same or
primary and alternate links for these disks to load balance the SCSI channels. even different volume groups, can easily be accomplished, once you have the
knowledge and the experience in MirrorDisk/UX use.
You will want to run sar reports for disk I/O, and make any primary or alternate
link changes, depending on the results of the disk reports. Note 1: Alternate links, or pvlinks, are multiple paths to the same physical disk.
Used for redundancy, in case of channel failure, alternate links will automatically
Moving a Logical Volume from One Volume Group to Another change paths to a disk in the event of a card failure, increasing availability.

In some instances we may need to move a logical volume from one volume group Note 2: rmsf is the command to remove the special files or device files of a given
to another volume group. This is a little tricky, because you create a mirror copy of device. In this case, we need to remove the device files because the new disk array
a logical volume on a clean disk, remove the mirror--which in fact only removes attached to that same SCSI card will have the same device file names, although the
the headers on the disk and leaves the data intact--then move the disk to another disks are on a different array and may be of a different size.
volume group. For example we had a system that had an Informix raw volume in
the /dev/vg00 volume group, and needed to move the volume to the Informix Note 3: insf e rebuilds all of the device files for the new disks on the system. This
volume group. We added a new disk to the /dev/vg00 volume group, created one needs to be done if you have used rmsf to remove device files, or if you are adding
mirror copy of the logical volume that was 200 MB, on the new disk, which started a set of disks to a system while the system is online. A form of this command is
from the first physical extent on the disk, and immediately removed the mirror run every time the system is restarted, in order for the system to see any newly
from the new disk. The removal of the mirror copy only removes the headers on added devices.
the disk, and not the data contained on the disk. We then removed the disk from
the volume group, and added it to the other volume group. We then created the
logical volume, exactly the way the mirror was created, first on the disk, and the Galen Scalone works for Lucent in St. Petersburg, Florida as a Senior System
data was still accessible--after changes to the Informix config. to use the new Administrator. Prior to that he was with Symbol Technologies for about four years.
logical volume, in the new volume group. It looked like this: He has been working with UNIX since 1993. He can be reached at
[email protected] or [email protected].
pvcreate /dev/rdsk/c3t0d0 Makes the disk an LVM disk.

vgextend /dev/vg00 /dev/dsk/c3t0d0 Adds the disk to the volume group.

lvextend m 1 /dev/vg00/rawvol /dev/dsk/c3t0d0 Makes one mirror copy

lvreduce m 0 /dev/vg00/rawvol /dev/dsk/c3t0d0 Removes the headers, as

7/8 8/8