ActiveAnswers

hp messaging

april 2003

technical overview

Exchange Server 2003 Clusters on Windows Server 2003

abstract: Microsoft Cluster Service on Windows Server 2003 provides
many benefits for Exchange Server 2003 Clusters. This document outlines some of those benefits and provides some practical examples of how they improve deployment, administration and reliability of Exchange Server 2003.

contents
introduction..................... 3 eight node support .......... 3 ease of installation .......... 3 account setup ............. 3 creating the cluster ...... 4 supplemental changes ................... 13 preparing for Exchange ...................... 15 required services ...... 15 a needed resource .... 17 installing Exchange........ 18 summary of storage ... 21 file system mount points............................ 22 faster failover................ 24 security ......................... 24 conclusion ..................... 24

Exchange Server 2003 Clusters on Windows Server 2003

notice
The information in this document is subject to change without notice. Microsoft, Windows and Exchange are U.S. registered trademarks of Microsoft Corporation. april 2003

Exchange Server 2003 Clusters on Windows Server 2003

introduction
With the release of Windows Server 2003 Enterprise and Datacenter editions, Microsoft improved many features of Microsoft Cluster Server (MSCS) which benefit Exchange Server 2003 clusters. Exchange Server 2003 also includes many improvements related to clusters. Some of those joint improvements are: 8 node cluster support, ease of installation, quality control during installation, faster failover, support of file system mount points, and security improvements. Please note that this document is based on pre-release versions of both Windows Server 2003 Enterprise Edition and Exchange Server 2003 the final products may differ from the information provided here. This guide is not intended to be a complete guide to cluster or application configuration, but rather an overview and discussion of new features.

eight node support

Windows 2000 Advanced Server and Datacenter Server support two and four nodes respectively on a MSCS cluster. The Enterprise and Datacenter editions of Windows Server 2003 support eight nodes each in MSCS clusters. These additional nodes reduce the overall cost of deploying large clusters. In a Windows 2000 Advanced Server active/passive cluster 50% of the nodes are passive: one active/one passive. A Windows 2000 Datacenter cluster has better efficiency with three active nodes and one passive node, or 25% of the nodes passive. With eight node support, the economy of cluster deployment improves such that a cluster can be deployed with seven active nodes and one passive node, or 12.5% of the nodes passive. However in this 7+1 deployment, the cluster could only survive a single failure, which may not satisfy particular availability requirements. Enterprises may want to consider 6+2, 5+3, or other cluster models for Exchange Server 2003 depending on their specific requirements. Exchange Server 2003 does support active/active clusters, but only with 2 nodes and with similar connection limitations as Exchange Server 2000 active/active clusters.

ease of installation
account setup
This example uses four domain level accounts. The domain administrator account is used for creating all other accounts and for running /forestprep for Exchange setup, during which examdin1 will be designated as the Exchange Full Administrator account. The exadmin1, clusadmin1, and redclussvc accounts are domain level user accounts with no special permissions at that level and are added to the Administrators group of each cluster node. redclussvc is the cluster service account, and clusadmin1 is the cluster administrator account that will be used to create the cluster. exadmin1 is used to install Exchange on the cluster nodes and to create and administer the Exchange virtual server.

Exchange Server 2003 Clusters on Windows Server 2003

creating the cluster

These steps are performed while logged in as clusadmin1. Installation of Windows 2000 clusters involved adding additional services from media. In Windows Server 2003, the cluster administration tool is installed by default and a new cluster can be created and configured without installation media or a reboot. The cluster service is not installed by default, but will be installed during cluster setup. Cluster Administrator is part of Administrative Tools.

One of the options when Cluster Administrator opens is to create a new cluster.

Selecting Create a new cluster starts the New Cluster Server Wizard which is a new feature of Windows Server 2003 and will guide us through the cluster creation process

Exchange Server 2003 Clusters on Windows Server 2003

The wizard interface is a very good guide through this process. The online help system provides detailed information about the prerequisites and process for cluster installation. What follows are screenshots and discussion of a sample installation. This sample installation was performed on the HP Proliant DL 380 G3 Packaged Cluster with the following specifications: Node 1 & 2 2x2.8 Ghz Intel Xeon Processors 4GB RAM 2x36 GB 15K RPM hard disk drives Smart Array Cluster Storage 14x72GB 15K RPM hard disk drives 256 MB controller cache For this sample installation the first information we need to provide is the domain that the cluster will be created in and the network name of the cluster.

Exchange Server 2003 Clusters on Windows Server 2003

The next step is to select the first node to be added to the new cluster.

Clicking on the Advanced button launched the following dialog. A typical configuration will configure all of the shared storage drives as cluster resources. An advanced configuration will require a quorum resource but will not configure any other shared storage as cluster resources. In this example we leave it at the default, or Typical configuration.

Exchange Server 2003 Clusters on Windows Server 2003

Next the wizard will perform a thorough analysis of the cluster including network resources, storage resources, and other configuration details. This analysis helps to ensure that the cluster is being set up correctly and conforms to requirements.

As the wizard progresses through the analysis, each line item is checked off and a + appears to the left which is used to expose more details. The View Log and Details buttons show much more detailed information for each task. The log file is written to c:\windows\system32\logfiles\cluster\clcfgsrv.log and is appended to rather than overwritten for each analysis. If the task has not yet been completed, the details will not show up in the log or in the Details dialog. Dont be surprised if you see an empty details dialog if you invoke it before that particular step is completed.

Exchange Server 2003 Clusters on Windows Server 2003

Just as an example, in this analysis one network interface is set to utilize DHCP. The Wizard warns that this is not recommended or supported.

And the task details dialog provides a bit more information.

The next couple steps are wizard dialogs for the IP address for the cluster, and the service account for the cluster, redclussvc (screen shots are not included here).

Exchange Server 2003 Clusters on Windows Server 2003

After all requisite information is provided, the wizard displays a summary of the proposed cluster configuration, which is show in the following screen shots.

Notice how all disk resources are marked as Managed. This is a result of the Typical configuration option as noted earlier.

Exchange Server 2003 Clusters on Windows Server 2003

10

Clicking on the Quorum button provides an option to select the type of quorum resource the cluster should use. The options are Local Quorum, Majority Node Set, and whichever shared drive the configuration wizard determined was the best quorum candidate, in this case Q:. The wizard will automatically select the smallest shared drive of at least 500MB as the quorum. If there are none, it will select the smallest drive of at least 50MB and create a warning. In this example Q: of 1000MB is selected as the quorum.

Local Quorum and Majority Node Set are new options in Windows Server 2003. Local quorum is just that, a quorum resource that is not on a shared disk, but directly on the cluster node. The limitation of a local quorum is that it must remain a single node cluster until the quorum is moved to a shared cluster resource. This is useful for application testing, development, or other limited scenarios. The Majority Node Set quorum option is one where each node of the cluster maintains its own copy of the cluster configuration information and those independent copies are kept in synchronization through file shares. This type of quorum is useful when a shared quorum is not possible or practical, typically where the nodes are geographically distributed. One of its significant limitations is that a cluster utilizing this type of quorum can only remain operational if half or more of its nodes are functional; otherwise quorum is lost, as there is not a majority at that point, and the cluster fails. Traditional shared disk quorums should meet most clustering needs and will be utilized in this example. After clicking on Next, the wizard reanalyzes the cluster to ensure the environment meets requirements and proceeds through complete configuration of the cluster.

Exchange Server 2003 Clusters on Windows Server 2003

11

During this process the previously mentioned log file (clcfgsrv.log ) is appended to with more information and another log file is created (c:/windows/cluster/cluster.log) which contains details of the actual installation and starting of the cluster services and associated resources. Those two log files contain a plethora of information about the analysis and installation of the cluster and can be used for troubleshooting if needed. After exiting the wizard, Cluster Administrator launches the administrative interface for the newly created cluster.

In this typical or default configuration the wizard created the Cluster Group with the required resources and created additional groups for the remaining storage resources that we will use to install Exchange Server 2003 later in this document. At this point we have a single node Windows Server 2003 Enterprise Edition cluster.

Exchange Server 2003 Clusters on Windows Server 2003

12

The Add Nodes Wizard is started in two ways, either by selecting File/New/Node, or by File/Open Connection and selecting Add New Nodes to Cluster.

The process for adding a note to an existing cluster is very similar to creating the first node except for the dialog of selecting which computers to add to the cluster.

In this dialog, the Advanced button is the same as the one we saw in the Select Computer dialog when adding the first node and determines the Typical or Advanced storage options for this additional node. For this example well leave it at the default, typical. For the second node, the wizard performs the same analysis, asks for the cluster service account, analyzes the node again and then installs and configures the cluster service on that node. These dialogs are the same as those shown earlier for node 1 and will not be repeated here.

Exchange Server 2003 Clusters on Windows Server 2003

13

Upon completion we are presented with the two node cluster in the Cluster Administrator interface.

supplemental changes
To prepare for installation of Exchange Server 2003, we need to make some changes from this default configuration. The first thing to change is the default network configuration which has both of the network interfaces configured for both cluster and client communications. Before creation of the first node of the cluster, I changed the names of the network interfaces to Internal and Client-External to reflect the purpose. Since the Internal interface is for cluster communications, I used a crossover cable that is included with the Proliant DL 380 G3 Packaged Cluster to connect the first Ethernet interface of each node. A private IP address was used for each of these interfaces as can be seen in the cluster configuration summary on previous pages.

Exchange Server 2003 Clusters on Windows Server 2003

14

The next step is to change the role of each set of interfaces by clicking on Networks and then right clicking on each network name and selecting properties.

Next we change the Client-External network to Client access only (public network) and the Internal network to Internal cluster communications only (private network).

Exchange Server 2003 Clusters on Windows Server 2003

15

One best practice of cluster configuration is to unbind Client for Microsoft Networks and File and Printer Sharing for Microsoft Networks from the internal cluster interface. This is done on each cluster node. The IP configuration for those interfaces should not include DNS servers or a default gateway, should not attempt to register that interface with DNS, and NETBIOS should be disabled.

preparing for Exchange

required services
From a default server configuration, a couple required services must be installed before proceeding with Exchange Server 2003 installation. Exchange setup warns or any missing components.

Exchange Server 2003 Clusters on Windows Server 2003

16

IIS and associated components are added with Windows Components Wizard launched from the Add/Remove Programs tool in the control panel. The needed IIS components are listed under Application Server which is a new component category in Windows Server 2003 and is not selected by default. Click on Details to get to the individual application server components.

At the Application Server level, select ASP.NET and Internet Information Services (IIS). We also need to bring up the details of IIS.

Exchange Server 2003 Clusters on Windows Server 2003

17

Here we need to select NNTP Service and SMTP Service because they are not part of the default IIS configuration.

Before each node was added to the cluster, the Manage Your Server center could have been used to add the Application Server role to each node. However, this role does not include NNTP or SMTP, so these would have to be added separately using this same process. Once a server is made a cluster node, the Manage Your Server center cannot be used.

a needed resource
Now that those required services are added to each node, we need to add the Distributed Transaction Coordinator (DTC) to the cluster by right clicking in the cluster group and creating a new resource as shown below.

Exchange Server 2003 Clusters on Windows Server 2003

18

Here we choose a reasonable name and the correct resource type.

In the remaining dialogs we confirm both cluster nodes as possible owners of the DTC resource and add cluster network name resource and quorum disk resource as dependencies. The rest of the New Resource dialogs are very straight forward.

installing Exchange
Before installing Exchange Server 2003 software on the cluster nodes, the Active Directory needs to be prepared. This is done by running the Exchange Server setup program with the /forestprep switch when logged in as a domain administrator. The forestprep and domainprep processes are no different than in Exchange 2000. During this process I selected the exadmin1 as the Exchange Full Administrator account. The following steps are performed with the exadmin1 user account. Installation of the Exchange Server 2003 software on the cluster nodes does not differ significantly from installation of Exchange 2000 so a detailed walkthrough is not included here. Ensure that the Exchange software is installed on each cluster node or it will not be able to participate in the Exchange cluster.

Exchange Server 2003 Clusters on Windows Server 2003

19

Once Exchange Server 2003 is installed on each node a new group for the Exchange virtual server and associated resources is created by either selecting File/New/Group or by right clicking in the right pane and selecting New/Group.

The New Group dialog sequence is very straight forward. REDEX01 is the group name and Exchange 2003 Virtual Server is the description and both nodes of the cluster are listed as possible owners. The next step in this process is to move the remaining disk resources from other groups to the REDEX01 group just created. Click on the Resources container on the right side to display all resources no matter which group they belong to. Right click on each disk resource in Group X (1, 2, or 3 in this example) to move that resource to REDEX01. A confirmation prompt is required for each resource move.

Once those disks are moved and the source Groups are empty, they are removed by clicking on the Groups container, right clicking on each and selecting delete.

Exchange Server 2003 Clusters on Windows Server 2003

20

Next we need to add IP address and Network Name resources to this group. These are each added by selecting File/New/Resource or by right clicking in the right pane and selecting New/Resource. In adding the IP address and Network Name resources, we must ensure we add them to the correct group, REDEX01 in this case, and that the correct resource types are chosen. The IP Address resource is not dependent on any other resources, but the network name resource is dependent on the IP address, so the IP address must be added first. Each resource can be started after it is created to ensure it is created correctly. One of the parameters available when creating the Network Name resource is Enable Kerberos Authentication. Before Windows 2000 Service Pack 3, Kerberos authentication was not supported for cluster virtual servers. For now well leave this option unselected because it and the DNS Registration Must Succeed option will be configured when the Exchange System Attendant resource is created.

The Exchange virtual server is created by adding the Microsoft Exchange System Attendant (SA) resource to the group. The SA requires dependencies on the network name resource and at least one physical disk resource. Configuration of the SA prompts for the location of the working directory, which has to be on a shared resource. In this example, E: is that disk resource, so E:\exchsrvr is the working directory. Installation of the System Attendant resource results in the installation of all the other Exchange resources.

Exchange Server 2003 Clusters on Windows Server 2003

21

Right clicking on REDEX01 and selecting Bring Online results in an operational Exchange cluster.

From here, configuration of storage groups and databases is the same as Exchange 2000 and will not be detailed here.

summary of storage
This example used the following storage configuration on the Smart Array Cluster Storage enclosure.

Logical Logical Logical Logical

Drive Drive Drive Drive

1 2 3 4

is is is is

the the the the

quorum drive as drive Q: working drive E: log drive D: database drive F:

This configuration is not intended to support a production environment at any particular level of service. It is intended for example only.

Exchange Server 2003 Clusters on Windows Server 2003

22

file system mount points

File system mount points are a key feature in large clusters. With a four node Windows 2000 Datacenter Exchange cluster, twenty six drive letters was a real limitation. With three active and one passive nodes and each active node requiring, for a reasonable configuration, four log and database drives, a quorum drive for the cluster and a C: drive, M: drive, floppy drive and CDROM for each node, the twenty six drive letters were consumed quickly. Through support of file system mount points, this problem is alleviated. These mount points permit a physical disk to be mounted in an empty directory on another disk, thus making that disk resource available without consuming a drive letter. For example, in our previous Exchange cluster configuration we had three Exchange drives each with their own drive letter (D:, E:, and F: ). The Exchange System Attendant cluster resource depends on at least one physical disk resource, E: in our example. We can create a folder structure on E: and use that folder structure to mount physical disks for logs and databases for our Exchange cluster. A sample folder structure might look like this.

To mount physical disks on to each directory, right click on the physical disk from Disk Management and select Change Drive Letter and Paths. Prior to this step, the disk was formatted and labeled, but was not assigned a drive letter or mounted elsewhere.

Exchange Server 2003 Clusters on Windows Server 2003

23

Clicking Add in the Change Drive Letter.. dialog opens the Add Drive Letter or Path dialog. In this example we choose the appropriate folder on E:.

This changes the display of our folder structure to show that SG1/Logs is now a mounted disk rather than a folder. Completing these tasks for all the folders created earlier results in this changed display.

A similarly configured server utilizing traditional drive letters might look like this. Note the cleaner view of My Computer in the example above.

Exchange Server 2003 Clusters on Windows Server 2003

24

faster failover
Cluster resources typically have dependencies on other resources for their operation. For example we saw that the Exchange System Attendant resource is dependent on the network name and a physical disk resource. The other Exchange resources that are created when the SA resource is created have similar dependencies. In Exchange Server 2003 this dependency tree is flattened so that individual Exchange resources are dependent on fewer resources and can come online quicker. In Exchange 2000, many resources, such as POP3, SMTP, IMAP4, HTTP, and MS Search are dependent on the Information Store (IS), which is then dependent on the System Attendant (SA). Exchange Server 2003 flattens the dependency tree such that those same resources are dependent directly on the SA rather than the IS. This permits those resources to come online faster. The first screen shot if from an Exchange 2003 cluster and the second is from an Exchange 2000 cluster to show the difference in dependencies.

security
Exchange Server 2003 provides security enhancements in a cluster environment in several ways: Support for Kerberos authentication at the virtual server, no requirement for the cluster service account to be an Exchange Full Administrator, and IPSEC support between front end and back end servers.

conclusion
The release of Windows Server 2003 Enterprise and Datacenter editions paired with the future release of Exchange Server 2003 greatly enhance the function of Exchange in a cluster environment. From installation, to security, support and failover these product enhancements provide significant advantage over previous versions and will cause Enterprises to reevaluate the position of Exchange clusters in their environment.