CH A P T E R

11

Configuring PowerPath Multipathing for Linux

This chapter contains the following sections:

Configuring Boot over Multipathing, page 11-1

Preparing the System and Performing PowerPath Installation, page 11-1

Setting Up Files and Directories, page 11-2

Copying Required Objects and Configuration Files, page 11-2

Modifying Files, page 11-4

Archiving the bootdisk Directory, page 11-6

Configuring Boot over Multipathing

The following steps configure Boot over IB using PowerPath, using Linux RHEL3 Update7 on a X86_64
(AMD) system as an example.

Preparing the System and Performing PowerPath Installation

To prepare the system and install PowerPath, follow these steps:
Step 1

Set up the system for boot over SRP. When the system can boot over SRP, proceed with the steps below
to get the system boot over PowerPath.

Step 2

Install PowerPath on the system. Follow the usual EMC PowerPath installation procedure to install
PowerPath. Refer to the PowerPath installation document for instructions.
pm -ivh EMCpower.LINUX-4.5.1-022.rhel.x86_64.rpm
/etc/init.d/PowerPath start
emcpreg -install (register powerpath)

Step 3

After PowerPath is installed properly and started, enter the fdisk -l command. You should see output
similar to the example below. In this case, only one LUN is assigned to this host, and the LUN is
partitioned accordingly.
[root@sw-n4300-1 ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System

Cisco VFrame Third Party Integration Guide

OL-12603-01

11-1

Chapter 11

Configuring PowerPath Multipathing for Linux

Setting Up Files and Directories

/dev/sda1 * 1 38 305203+ 83 Linux

/dev/sda2 39 293 2048287+ 82 Linux swap
/dev/sda3 294 2610 18611302+ 83 Linux
Disk /dev/emcpowera: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/emcpowera1 * 1 38 305203+ 83 Linux
/dev/emcpowera2 39 293 2048287+ 82 Linux swap
/dev/emcpowera3 294 2610 18611302+ 83 Linux

PowerPath is now up and running. The emcpowera* shows the PowerPath pseudo devices.

Setting Up Files and Directories

To set up files and directories, follow these steps:
Step 1

Create a new directory called bootdisk.

Step 2

Copy the initrd file from the /boot to the new bootdisk directory.
-cp -p /boot/initrd-2.6.9-34.ELsmp.img bootdisk/

Step 3

Unzip the initrd-2.6.9-34.ELsmp.img file in the same bootdisk directory.

zcat initrd-2.6.9-34.ELsmp.img > initrd-2.6.9-34.ELsmp

Step 4

Copy the initrd directory structure and all the associated files into the same bootdisk directory, as follows
-cpio -i -I initrd-2.6.9-34.ELsmp

Step 5

Verify that the bootdisk directory now contains the following files and directories, in addition to
initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files.
bin dev etc init lib loopfs proc sbin sys sysroot

Step 6

Delete the initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files from the bootdisk directory.

Step 7

Create two more directories in the bootdisk directory and call them lib64 and usr.

Step 8

Verify that the following files and directories are in the bootdisk directory:
bin dev etc init lib lib64 loopfs proc sbin sys sysroot usr

Copying Required Objects and Configuration Files

To copy the requireed PowerPath objects and configuration files, follow these steps:
Step 1

Copy the PowerPath objects and configuration files into the new bootdisk directory. Some Linux objects
also need to be copied.

Cisco VFrame Third Party Integration Guide

11-2

OL-12603-01

Chapter 11

Configuring PowerPath Multipathing for Linux

Copying Required Objects and Configuration Files

The entire list is displayed in the table below.

Note that it is not required to copy the IB module since it should already be in the /lib directory in
the original initrd file.

The /lib64 directory is required since it is a x86_64 system. You will copy all the required library
files to either the /lib or /lib64 directory.

Use the ldd command to obtain the required library, as shown in this example:
[root@host-1 bin]# ldd /bin/mkdir
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003977b00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003976800000)
/lib64/ld-linux-x86-64.so.2 (0x0000003976600000)
[root@host-1 bin]# ldd /sbin/powermt
linux-gate.so.1 => (0xffffe000)
libc.so.6 => /lib/tls/libc.so.6 (0x00adf000)
/lib/ld-linux.so.2 (0x00ac6000)

Table 11-1

Copy the PowerPath Objects and Configuration Files to the Directory

Notes

Copy the whole directory

This directory contains

Copy

To

Bootdisk Directory

/bin/ash

---->

/sbin/ash

/bin/gawk

---->

/sbin/gawk

/bin/mkdir

---->

/sbin/mkdir

/sbin/powermt

---->

/sbin/powermt

/sbin/rmmod

---->

/sbin/rmmod

/bin/mount

---->

/sbin/mount

/bin/umount

---->

/sbin/umount

/bin/cat

---->

/sbin/cat

/bin/mknod

---->

/sbin/mknod

/sbin/pivot_root

---->

/sbin/pivot_root

/bin/sleep

---->

/sbin/sleep

/etc/emc

---->

/etc/emc

/etc/emcp_devicesDB.dat

---->

/etc/emcp_devicesDB.dat

/etc/emcp_devicesDB.idx

---->

/etc/emcp_devicesDB.idx

/etc/emcp_registration

---->

/etc/emcp_registration

/etc/opt/emcpower

---->

/etc/opt/emcpower

/etc/fstab

---->

/etc/fstab

/lib/modules/2.6.9-34.ELsmp/extr
a/emc*.ko

---->

/etc/lib

/usr/lib/libemcp_core.so

---->

/lib/libemcp_core.so

/usr/lib/libemcp_lam.so

---->

/lib/libemcp_lam.so

/usr/lib/libemcp_lic_rtl.so

---->

/lib/libemcp_lic_rtl.so

/usr/lib/libemcp_mp_rtl.so

---->

/lib/libemcp_mp_rtl.so

/etc/opt/emcpower/powercf and
/etc/opt/emcpower/emcpmgr object.
These objects are used in the init file.

Copy the whole PowerPath module.

Cisco VFrame Third Party Integration Guide

OL-12603-01

11-3

Chapter 11

Configuring PowerPath Multipathing for Linux

Modifying Files

Table 11-1

Copy the PowerPath Objects and Configuration Files to the Directory

Notes

Copy

To

Bootdisk Directory

/usr/lib/libemcpmp.so

---->

/lib/libemcpmp.so

/usr/lib/libemcp.so

---->

/lib/libemcp.so

/lib/ld-linux.so.2

---->

/lib/ld-linux.so.2

/lib/tls/libm.so.6

---->

/lib/libm.so.6

/lib/tls/libc.so.6

---->

/lib/libc.so.6

/lib64/libdl.so.2

---->

/lib64/libdl.so.2

/lib64/tls/libm.so.6

---->

/lib64/tls/libm.so.6

/lib64/tls/libc.so.6

---->

/lib64/tls/libc.so.6

/lib64/ld-linux-x86-64.so.2

---->

/lib64/ld-linux-x86-64.so.2

/lib64/libselinux.so.1

---->

/lib64/libselinux.so.1

/lib64/tls/librt.so.1

---->

/lib64/tls/librt.so.1

/lib64/tls/libpthread.so.0

---->

/lib64/tls/libpthread.so.0

/lib/libpn.so

---->

/lib/libpn.so

/usr/sbin/chroot

---->

/usr/sbin/chroot

Modifying Files
To modify the /etc/fstab file, which you previously copied, follow these steps:
Step 1

Modify the /etc/fstab file to reflect the PowerPath devices:

In this example, /dev/emcpowera3 is the root partition and /dev/emcpowera1 is the boot partition.
# This file is edited by fstab-sync - see man fstab-sync for details
/dev/emcpowera3 / ext3 defaults 1 1
/dev/emcpowera1 /boot ext3 defaults 1 2
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
/dev/emcpowera2 swap swap defaults 0 0
/dev/hdc /media/cdrom auto
pamconsole,exec,noauto,managed 0 0
/dev/fd0 /media/floppy auto
pamconsole,exec,noauto,managed 0 0

Step 2

Modify the default init file to the file below:

#!/bin/ash
mount -t proc /proc /proc
echo Mounted /proc filesystem
echo Mounting sysfs
mount -t sysfs none /sys

Cisco VFrame Third Party Integration Guide

11-4

OL-12603-01

Chapter 11

Configuring PowerPath Multipathing for Linux

Modifying Files

echo Creating /dev

mount -o mode=0755 -t tmpfs none /dev
mknod /dev/console c 5 1
mknod /dev/null c 1 3
mknod /dev/zero c 1 5
mkdir /dev/pts
mkdir /dev/shm
echo Starting udev
udevstart
echo -n /sbin/hotplug > /proc/sys/kernel/hotplug
echo Loading mlxsys.ko module
insmod /lib/mlxsys.ko
echo Loading mod_vip.ko module
insmod /lib/mod_vip.ko
echo Loading mod_thh.ko module
insmod /lib/mod_thh.ko
echo Loading mod_rhh.ko module
insmod /lib/mod_rhh.ko
echo Loading ts_kernel_services.ko module
insmod /lib/ts_kernel_services.ko
echo Loading ts_ib_packet_lib.ko module
insmod /lib/ts_ib_packet_lib.ko
echo Loading ts_ib_core.ko module
insmod /lib/ts_ib_core.ko
echo Loading ts_ib_mad.ko module
insmod /lib/ts_ib_mad.ko
echo Loading ts_ib_tavor.ko module
insmod /lib/ts_ib_tavor.ko
echo Loading scsi_mod.ko module
insmod /lib/scsi_mod.ko
echo Loading sd_mod.ko module
insmod /lib/sd_mod.ko
echo Loading ts_kernel_poll.ko module
insmod /lib/ts_kernel_poll.ko
echo Loading ts_ib_client_query.ko module
insmod /lib/ts_ib_client_query.ko
echo Loading ts_ib_sa_client.ko module
insmod /lib/ts_ib_sa_client.ko
echo Loading ts_ib_dm_client.ko module
insmod /lib/ts_ib_dm_client.ko
echo Loading ts_ib_cm.ko module
insmod /lib/ts_ib_cm.ko
echo Loading ts_srp_host.ko module
insmod /lib/ts_srp_host.ko
echo Loading emcplib.ko module
insmod /lib/emcplib.ko
echo Loading emcp.ko module
insmod /lib/emcp.ko
echo Loading emcpmp.ko module
insmod /lib/emcpmp.ko
echo Loading emcpmpc.ko module
insmod /lib/emcpmpc.ko
echo Loading emcpmpaa.ko module
insmod /lib/emcpmpaa.ko
echo Loading emcpmpap.ko module
insmod /lib/emcpmpap.ko
echo Loading emcphr.ko module
insmod /lib/emcphr.ko
echo Loading emcpioc.ko module
insmod /lib/emcpioc.ko
echo Removing module emcpioc.ko
rmmod emcpioc
echo Loading mptbase.ko module
insmod /lib/mptbase.ko

Cisco VFrame Third Party Integration Guide

OL-12603-01

11-5

Chapter 11

Configuring PowerPath Multipathing for Linux

Archiving the bootdisk Directory

echo Loading mptscsih.ko module

insmod /lib/mptscsih.ko
echo Loading jbd.ko module
insmod /lib/jbd.ko
echo Loading ext3.ko module
insmod /lib/ext3.ko
udevstart
rm -f /dev/emcpower
mmaj=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $1}'`
mmin=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $2}'`
mknod /dev/emcpower c $mmaj $mmin
sleep 2
echo Start the configuration of powerpath
powermt config
powermt load
/etc/opt/emcpower/emcpmgr map
sleep 7
/etc/opt/emcpower/powercf -C powermt save
powermt register
sleep 2
echo Creating root device
echo mkrootdev /dev/root | /sbin/nash
umount /sys
echo Mounting root filesystem
mount -o defaults --ro -t ext3 /dev/root /sysroot
mount -t tmpfs --bind /dev /sysroot/dev
umount /proc
cd /sysroot
echo Switching to the new root
pivot_root . initrd
exec chroot . /sbin/init <dev/console >dev/console 2>&1
echo Unmount the old dev now
umount /initrd/dev

Archiving the bootdisk Directory

To archive the bootdisk directory, follow these steps:
Step 1

Archive the bootdisk directory now that all the required files are copied into it.
In this example the archive file is put at /root/
a.

cd to the bootdisk directory.

b.

Locate and copy the file.

find . | cpio -v -c -o > /root/initrd-2.6.9-34.ELsmp

c.

Compress the file.

Cisco VFrame Third Party Integration Guide

11-6

OL-12603-01

Chapter 11

Configuring PowerPath Multipathing for Linux

Archiving the bootdisk Directory

gzip -9 initrd-2.6.9-34.ELsmp

d.

Move the directory.

mv initrd-2.6.9-34.ELsmp.gz initrd-2.6.9-34.ELsmp.img

Completing the PowerPath Configuration

To complete the PowerPath configuration, follow these steps:
Step 1

Move the initrd-2.6.9-34.ELsmp.img file to the /boot directory.

Step 2

Change the bootloader append for the root parameter of the kernel to the appropriate PowerPath pseudo
device.
In this example the PowerPath pseaudo device was root=/dev/emcpowera3

Step 3

Reboot the server. It will boot the root filesystem on a PowerPath device instead of a sd device.

Cisco VFrame Third Party Integration Guide

OL-12603-01

11-7

Chapter 11

Configuring PowerPath Multipathing for Linux

Archiving the bootdisk Directory

Cisco VFrame Third Party Integration Guide

11-8

OL-12603-01