Deploying boot from SAN with Microsoft Exchange Server 2003 Clusters white paper

Executive summary............................................................................................................................... 3 Results summary .................................................................................................................................. 3 Boot from SAN solution overview .......................................................................................................... 4 Test environment .............................................................................................................................. 5 Boot from SAN.................................................................................................................................... 5 Factors in selecting a BFS Exchange cluster solution ............................................................................. 5 BFS deployment best practices........................................................................................................... 7 RDP best practices ........................................................................................................................ 7 Fibre Channel HBA best practices .................................................................................................. 7 BFS LUNs best practices ................................................................................................................ 8 SAN fabric zoning best practices ................................................................................................... 8 HP ProLiant BladeSystems best practices.......................................................................................... 8 Microsoft Clusters best practices..................................................................................................... 8 Verify Exchange performance best practices.................................................................................... 8 Deploying a new BFS Exchange configuration ........................................................................................ 9 BFS solution testing ............................................................................................................................ 10 Host bus adapters .......................................................................................................................... 10 Recovery from the loss of a local boot disk ........................................................................................ 11 Migration from local boot to BFS...................................................................................................... 11 RDP deployment console ............................................................................................................. 13 ROM BIOS settings for BFS.......................................................................................................... 15 HBA settings for BFS ................................................................................................................... 17 Cluster failover before migration .................................................................................................. 19 Additional passive cluster nodes................................................................................................... 19 Path failure recovery....................................................................................................................... 20 Recovery from the loss of a BFS disk................................................................................................. 20 Repairing the failed LUN ............................................................................................................. 20 New boot from SAN LUN ........................................................................................................... 21 Recovery from the loss of a BL45p blade server ................................................................................. 21 New blade server re-using HBA from failed blade .......................................................................... 21 New blade with new HBA........................................................................................................... 22

Summary .......................................................................................................................................... 22 Appendix A: Bill of materials............................................................................................................... 23 Appendix B: Reference material .......................................................................................................... 24 For more information.......................................................................................................................... 25

Executive summary
Booting from SAN-based system disks, rather than from local system disks, affords organizations many advantages, which include minimizing hardware costs, maximizing management benefits, and consolidation. This white paper discusses deploying boot from SAN (BFS) Microsoft Exchange clusters to determine the benefits of an Exchange cluster consisting of HP ProLiant BladeSystems, boot from SAN-based system disks, deployed using HP ProLiant Essentials Rapid Deployment Pack (RDP) software, and further uses of RDP as a backup and recovery mechanism for BFS disks. It answers questions that often arise, particularly in a Microsoft Exchange environment, on potential deployment and configuration complexities, as well as recovery issues, in a BFS environment. The objectives for this solution were to: Document how to deploy a new BFS Microsoft Exchange cluster using RDP Demonstrate how to migrate a local boot Microsoft Exchange cluster to BFS using RDP Demonstrate procedures for backup and recovery of server images using RDP for a local boot server configuration Demonstrate recovery of BFS server after BFS disk or server failures using RDP Demonstrate continued system operation from path failures through automated path failover using MPIO multi-path software

Results summary
Testing of this solution demonstrated that: RDP simplifies system disk failure and recovery procedures by utilizing RDP server replication and deployment capabilities. Migration from local boot to BFS is easy using RDP to deploy a saved local boot system image to a new BFS disk. Adding BFS to an existing Microsoft Exchange Server 2003 cluster environment had little effect on Exchange and required no changes to the Exchange configuration. Recovery from system disk failures can use RDP to restore saved system images to replacement system disks whether they are local boot or BFS. Deployment or restoration of an Exchange cluster members RDP system image during BFS migration or system recovery took less than 60 minutes including reboots. Migration of the test active/active/passive Microsoft Exchange cluster to BFS was completed with minimal disruption of Exchange service in less than 3 hours. Creation of an RDP system image of an Exchange cluster member took less than 30 minutes including reboots and caused minimal disruption of Microsoft Exchange service. Recovering a failed BladeSystem took less than 30 minutes if the host bus adapter (HBA) from the failed system was re-used. RDP system images must be re-captured anytime a system is updated, upgraded, or changed in anyway. MPIO multi-path software maintains BFS integrity despite path failures.

Boot from SAN solution overview

This solution describes an active/active/passive Microsoft Exchange Server 2003 cluster configuration utilizing Boot from SAN (BFS) technology. The cluster members were HP ProLiant BL45p blade servers. A goal of this solution was to demonstrate the advantages of BladeSystems and BFS in this Exchange environment, as well as the advantages of server deployment using HP ProLiant Essentials RDP. A further goal was to demonstrate the migration from a local boot to BFS for a Microsoft Exchange Server 2003 cluster. BladeSystems were designed with configuration growth in mind. This solution shows that smaller configurations where systems booted from local disks could be easily and quickly migrated to BFS as server growth occurred. BFS allows the use of SAN-based storage for consolidation, ease of management, and higher availability of blade system disks.

Figure 1. BFS SAN

Test environment
Testing was performed on an Exchange Server 2003 cluster environment accessing Exchange storage groups stored on an HP StorageWorks Enterprise Virtual Array (EVA) as shown in Figure 1. The configuration consisted of the following: Exchange cluster serverThree-node active/active/passive Exchange Server 2003 cluster. ServersCluster nodes consisted of HP ProLiant BL45p BladeSystems running Microsoft Windows Server 2003 Enterprise Edition and Exchange Server 2003 SP2. StorageHP StorageWorks 8000 Enterprise Virtual Array (EVA8000) storage array. Domain controllersDomain controllers consisting of HP ProLiant DL380 G3 servers running Microsoft Windows Server 2003 Enterprise Edition and configured as active directories and global catalogs. Storage management serversHP OpenView General Purpose Servers running on DL380 servers configured with Microsoft Windows Server 2003 Enterprise Edition. HP StorageWorks Command View EVA 5.0 for storage management. SANTwo redundant HP StorageWorks SAN Switch 4/32 Fibre Channel switches.

Boot from SAN

Boot from SAN (BFS) refers to a server booting from a disk that resides somewhere in the SAN instead of a disk that is local to the server or presented by direct-attached storage. In todays computing environment, BFS configurations using blade servers offer many advantages such as lower total cost of ownership (TCO), centralized and simplified system administration, improved disaster tolerance, and increased business continuity. In an Exchange environment, the system disk is moved to SAN-based storage.

Factors in selecting a BFS Exchange cluster solution

Booting from a SAN in an Exchange cluster environment has many advantages, particularly when combined with ProLiant Essentials RDP. Advantages include: Improved disaster tolerance Higher application availability Increased business continuity Easier administration Shorter recovery time objectives (RTO) Lower TCO Disaster tolerance is improved in several ways. First, just using a clustered Exchange configuration increases application availability. Beyond that, BFS technology allows system disks to reside on higher performance HP StorageWorks EVA storage arrays. These storage arrays contain redundant components, thus providing a more robust storage environment, thereby reducing the potential for loss of BFS system disks. Higher levels of disaster tolerance can be achieved by replicating BFS disks to remote sites, in addition to Exchange data, using HP StorageWorks Continuous Access. In the event of a disaster, the BFS system disks are failed over to the remote site where standby recover BladeSystems can be up in minutes restoring Exchange service.

Note Replication of BFS disks to a disaster site was not tested as part of this solution.

Higher Exchange application availability is achieved through the use of BFS disks. The potential for loss of a BFS disk residing on HP StorageWorks EVA storage arrays is much lower than residing on a local BladeSystem disk. This higher availability is in addition to the application availability provided by an Exchange cluster. Increased business continuity is accomplished because SAN-based storage provides better data protection mechanisms for BFS disks such as system backup/restore and HP StorageWorks Continuous Access data replication. Easier administration through centralized management is accomplished by placing BladeSystem boot disks on SAN-based storage arrays, using RDP to manage creation and deployment of all servers from a single location and using HP Integrated Lights Out (iLO) consoles for consolidated blade server console access and management. Shorter RTO from BladeSystem loss are achievable because recovery is simpler. Recovery involves replacing the failed BladeSystem, pointing the new BladeSystem to the proper BFS disk, and booting. The recovery process can be manual or automated. The system can be available in minutes. Recovery time is shortened because a new system does not have to be reconfigured or re-installed. Centralized, rapid deployment of Exchange configurations can be achieved through customizable, automated RDP capabilities. In addition, concurrent server deployments are possible, which increases productivity. The TCO is lower because BladeSystems have lower initial cost of purchase especially if they are diskless servers. Administration and maintenance costs are lower and more centralized through BladeSystem iLO and RDP management. Operating costs are reduced because BladeSystems require less floor space, require less power, have fewer parts to fail, and are less expensive than stand-alone servers. The use of BFS technology further lowers cost by amortizing storage array costs for system disks over many servers and lowering storage management costs by consolidating servers system disks on SAN-based storage arrays. Potential disadvantages of a BFS Exchange cluster include: More complex system deployments New system recovery procedures RDP processes cause short Exchange service outages Additional system tuning may be required Clustered BFS configurations require additional planning and setup Hardware deployment is more complex as knowledge of and changes to the boot process are required. System administrators are required to modify the boot process to boot from the BFS disk. This entails changes to the ROM BIOS and HBA settings. RDP provides scripted procedures to modify ROM BIOS/HBA settings for BFS configurations. BFS deployments can be further complicated by using the SCSIport driver instead of the Storport driver. Use of the SCSI port requires more HBAs and additional SAN fabric zoning to isolate system disk traffic from data traffic. BFS has different recovery procedures requiring additional work to determine what impacts BFS failures present. A change from local system disk backup and recovery procedures to SAN-based procedures is required. The use of RDP in these processes requires additional knowledge of RDP and how it can best be used in failure scenarios. Further, using RDP for system image recovery requires updating saved system images anytime a system is upgraded or updated. The RDP procedures for creating these images require system reboots. During these procedures any Exchange service must be failed over to another node in the cluster causing short Exchange service outages.

BFS configurations require additional system performance monitoring to determine if system performance is affected having the paging file residing on a BFS system disk. If needed, the paging file can be easily moved to a local BladeSystem disk if needed. Clustered BFS configurations require modifications to the registry to support system, page file, and data disks residing on the SAN. In addition, multiple clusters accessing the same SAN require further SAN zoning configurations steps to isolate each cluster and its members from each other.

BFS deployment best practices

This paper provides the following best practices when configuring a BFS solution. These BFS fundamentals ensure a properly configured BFS configuration. In addition, check the referenced materials in Appendix B: Reference material that apply to your planned configuration. RDP best practices First, determine how your configuration is best served by using RDP for server deployment, server replication, server recovery, and migration to BFS. Next develop procedures for each scenario and test them fully. Finally, always have current RDP system images of each server in the BFS configuration. These images are required for recovery procedures due to loss of system disks, local boot or BFS, and migration from local boot to BFS configurations. Update server RDP images after any server maintenance, software upgrades, or security patches. To ensure easy server deployment, always have a current generic Microsoft Windows Exchange RDP server image. This image would contain: Microsoft Windows Server 2003 Enterprise Edition Installation files for: Microsoft Exchange 2003 Microsoft Exchange 2003 SP1 Microsoft Exchange 2003 SP2 Any other required software

Utilize RDP as much as possible to ease system deployments. Determine any specific custom RDP scripts than can be used in deployments. RDP has many features such as creation of automated Microsoft Exchange deployment scripts or customization of RDP scripts to prepare HBAs and servers for BFS deployments. Fibre Channel HBA best practices The most important choice when configuring HBAs is whether to use the Storport or the SCSIport driver. For this solution test, the Storport driver was used as it is SAN-aware, making SAN configuration easier, and requires fewer HBAs. Use the following HBA best practices:
1. Ensure the latest versions of the HBA drivers are installed. 2. Ensure HBA allows booting from a SAN LUN. 3. Use the Spin Up Delay parameter to allow the storage arrays to come online and present the

BFS LUNs before the HBA looks for the boot LUNs. This may be necessary as power failure causes the server and the storage array to restart simultaneously. The system may not boot if the BFS disks are not present when needed. Emulex and QLogic HBAs have similar functionality. 4. Follow documented HBA procedures for enabling HBA BIOS and specifying the BFS boot LUN. 5. In case of BladeSystem failures, re-use HBAs from failed BladeSystem in the new BladeSystem. Reusing HBAs reduces recovery time because SAN fabric zoning does not have to be modified during recovery operations.

BFS LUNs best practices When creating the BFS LUNs on HP StorageWorks EVA storage arrays, follow these best practices:
1. Separate BFS LUNS into their own disk group on HP StorageWorks EVA storage arrays to isolate

system I/O traffic from Exchange traffic.

2. Create a separate BFS LUN for each server. (OS LUNs cannot be shared between servers.) 3. Isolate access to each BFS LUN by presenting each LUN to only one server. This is accomplished

through the HP StorageWorks Command View EVA storage management interface. SAN fabric zoning best practices Perform the following fabric best practices for both:
1. Use redundant Fibre Channel switches to provide redundant fabrics to increase disaster tolerance. 2. Use redundant HBAs to provide redundant paths to the SAN fabric. 3. Create multiple paths to each BFS LUN. In the event of path loss, the MPIO multi-pathing software

will fail over to the other redundant path.

4. Create one zone per server HBA port to include the host HBA port and both of the redundant EVA

storage array controllers. For the tested 3-node cluster, there were six zones. HP ProLiant BladeSystems best practices
1. Ensure the latest ROM BIOS is installed and follow BladeSystem server procedures for changing

the ROM BIOS to specify booting from an HBA.

2. Have spare BladeSystems available for faster recovery from failed systems. 3. Re-use HBA from failed blade server for faster server recovery.

Microsoft Clusters best practices Clusters require additional configuration to better operate in a BFS environment. For each cluster, modify the registry on each cluster member to allow the startup disks, page file disks, and cluster disks to reside on the same SAN fabric. The ManageDisksOnSystemBuses DWORD parameter with a decimal value of 1 must be added to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ClusDisk\Parameters registry key. See the Microsoft KB article at http://support.microsoft.com/kb/886569. If multiple clusters are accessing the same SAN fabric, then restricting access to each clusters disk resources is required. First, segregate cluster I/O activity for each cluster by way of SAN fabric zoning as documented in Microsoft KB304415 article (http://support.microsoft.com/kb/304415). Second, use LUN masking to present cluster/BFS disk resources to owning cluster members only. Verify Exchange performance best practices In an Exchange BFS configuration, performance testing is required to ensure SAN-based system disks do not affect Exchange performance. The main issue is the paging file. A SAN-based paging file may cause performance issues in some instances. A system may stop responding or have slowed response times due to higher access latencies to the pagefile. The Support for booting from a Storage Area Network (SAN) Microsoft KB article provides information on determining system issues with a SANbased page file.

Customers must determine specific performance criteria to decide if and when the paging file cannot reside on SAN-based storage due to performance issues. If system pagefile performance issues arise during peak Exchange usage, then moving the paging file might be considered. If the paging file must be moved to a server-based disk, follow Microsoft published procedures.

Note Performance of Exchange with a SAN-based paging file was not tested as part of this solution.

Deploying a new BFS Exchange configuration

Testing of this solution did not include creating a BFS Exchange configuration. Rather the focus was on migration of an existing local boot configuration to BFS. The Exchange cluster was created using standard Exchange cluster installation procedures then migrated to BFS as shown later in this document. Use the Microsoft Exchange Server 2003 automated deployment with HP BladeSystem white paper, referenced in Appendix B, for automated, unattended deployment of new, single Microsoft Exchange servers using RDP. Microsoft Exchange clusters cannot be deployed unattended. Deploying a new BFS Exchange cluster requires modification to the documented Microsoft Exchange cluster installation procedure to allow for BFS of each cluster member. After each cluster member has the base OS installed using BFS, then installation of Microsoft Exchange can be accomplished. The Exchange installation procedure does not change. RDP provides mechanisms for deploying new BFS Microsoft Windows servers. There are scripts to support modification of ROM BIOS and HBA settings to point to BFS LUNS before executing OS installation scripts. For deploying a new Windows Server BFS configuration, use the following documents, which are referenced in Appendix B: Reference material: Booting HP ProLiant server from a storage area network HOWTO HP StorageWorks Booting Windows Server 2003 and Windows Server x64 Edition systems from a storage area network application notes HP ProLiant Essentials Rapid Deployment PackWindows Edition Users Guide Microsoft Exchange clusters cannot be deployed in an unattended manner. Therefore, after the base OS has been installed, Exchange is configured as before. If the base OS install script also contained any additional software kits, then proceed using documented Microsoft Exchange cluster installation procedures.

BFS solution testing

The main goal of testing was to determine how to migrate from local boot to BFS in an Exchange cluster environment. Additional goals were to determine the failure and recovery procedures for local boot and BFS configurations using RDP as a recovery mechanism and document any issues using RDP in these situations. These recovery procedures do not replace standard backup and restore procedures of system disks. Solution testing consisted of the following: Recovery from the loss of a local boot disk Migration from local boot to BFS Path failure recovery Recovery from the loss of a BFS disk Recovery from the loss of a BL45p server These recovery procedures assume knowledge of the following products: Microsoft Server 2003 boot process ProLiant Essentials Rapid Deployment Pack (RDP) ProLiant Essentials Integrated Lights Out (iLO) interface HP ProLiant ROM BIOS update procedures HP StorageWorks Command View EVA SAN configuration concepts MPIO multi-pathing software Microsoft Storport driver Microsoft Server 2003 installation procedures Microsoft Exchange Server 2003 cluster installation procedures

Note These procedures are provided as examples and reference material only. They do not cover all configurations or all possible situations. Users should consult relevant documentation to determine the correct procedure for their specific application of this solution.

Host bus adapters

These procedures document configuring QLogic QLA2312 HBAs used with the HP ProLiant BL45p blade servers. Procedures for the use of Emulex HBAs are different. Use these procedures as a reference only if Emulex HBAs are present. The HP StorageWorks Booting Windows Server 2003 and Windows Server x64 Edition systems from a storage area network application notes document provides Emulex-specific information.

10

Recovery from the loss of a local boot disk

Initially, a local boot Exchange cluster was created. A test was performed to verify recovery of a local boot system disk image by way of RDP. Testing was accomplished by shutting down an existing Exchange cluster member, substituting new local disks, and then using RDP to restore a previously saved system image to the new disks. When restored, the system reboots and becomes an active member of the Exchange cluster again. In the test solution, the Exchange Virtual Server automatically failed back to the recovery node and Exchange servers were available from the recovered node. Some installations may not use auto failback cluster features requiring manual failback to the recovered node per site recovery procedures. Restoration to the new system disks took less than 60 minutes and required several reboots. The recovery steps are as follows: Bring up the iLO console window for the failed server to monitor the re-deployment. Ensure the failed blade server is powered off from the iLO console. Remove the failed system disks from the HP ProLiant BL45p server. Insert the replacement system disks into the HP ProLiant BL45p server. Initiate a system deployment job by way of RDP. Using the RDP Job wizard, create a job to deploy the saved system image to the failed blade (or drag and drop a pre-designated job onto the affected node within the RDP Console). 6. Boot the BL45p server and press F12 at the appropriate time to perform the PXE boot process. RDP will deploy the system image to the new system disks on the HP ProLiant BL45p server. 7. Monitor the progress of the re-imaging process by way of iLO console. Several reboots will occur as the system is reconfigured. 8. When the re-imaging is completed, the server is ready for service.
1. 2. 3. 4. 5.

See the RDP Users Guide for specific details. Loss of a local boot system disk can be mitigated by using a mirrored system disk configuration. Using mirrored system disks will further reduce the need for this recovery procedure. However, this does not remove the need for a test recovery procedure.

Migration from local boot to BFS

Some customers may decide to initially boot BladeSystems from local system disks with plans to migrate to BFS as the number of BladeSystems increases. Testing of this migration meant ensuring the Exchange cluster with local boot performed properly. LoadSim was run to ensure proper configuration of the Exchange cluster environment had occurred. Only then did the migration begin. The migration procedure was performed on a single cluster member at a time. In this manner, the Exchange environment was still available with minimal Exchange service disruption. Before each node is migrated to BFS, any Exchange services must be failed over to the passive node. This failover causes a short Exchange service outage. The length of this outage will be different for each Exchange cluster. If the migration is performed during off-peak hours, then disruption to Exchange users should be kept to a minimum. This migration procedure showed that there were no Exchange-specific changes required. Each cluster member booted back into the Exchange cluster and assumed its Exchange role after being migrated to a BFS system disk. If proper preparation is performed, then each cluster member should take approximately 60 minutes to migrate to BFS. This preparation is documented in step 1.

11

The procedure is as follows:

1. Ensure the following preparation is completed before migration of each cluster member to BFS: a. Ensure the latest version of the MPIO multi-path driver is installed. b. Ensure the latest version of the Storport HBA driver is installed. c. Ensure that latest ROM BIOS firmware is installed for each server. d. Using Command View EVA:

i. Add a separate disk group for BFS LUNs. ii. Create a new BFS LUN. If multiple servers are being migrated, then a SANboot system disk must be created for each server. iii. Present the LUN to the server. Each LUN can only be presented to a single server. e. Ensure SAN zoning is completed. On each switch, a zone is created for each HBA port that contains the HBA and the redundant storage arrays. Zoning may already be completed since this is a migration of an existing local boot system to a SAN boot system. f. Use RDP to save a server system image, if not already completed. g. Modify the registry of each cluster member to allow a Windows Server 2003based system to start from a SAN where the startup disk, the page file disks, and the cluster disks are all on the same SAN fabric as described in Microsoft Knowledge Brief 886569, http://support.microsoft.com/kb/886569. 2. For each cluster member: a. Failover any cluster applications residing on this node to other cluster nodes. b. Create an iLO session to the server and bring up a console window. c. From the iLO console, reboot the server boot. d. During the reboot process, modify the ROM BIOS setting to change the Boot Controller Order to boot from the Fibre Channel HBA. e. Before exiting the ROM BIOS, initiate a system deployment job by way of RDP. Using the RDP Job wizard, create a job to deploy the saved system image for the server being migrated. The job will be scheduled. f. Exit the ROM BIOS utility. g. During server boot, modify the HBA settings to specify the SAN-boot LUN. The server will reboot when completed. h. Press F12 to ensure a PXE boot to allow the RDP server to download the image to the BFS disk. The deployment of the system image to the SAN-based system disk will complete. Several reboots of the server will take place to configure the server. When completed, the system is available for use. Cluster applications can be moved back to this cluster member.

12

RDP deployment console All RDP jobs are initiated by the deployment console (Figure 2). The interface is divided into multiple panes: Computers, Jobs, and Job Progress. Jobs are initiated by dragging a job from the Jobs window pane and dropping them on top of a computer in the Computers window pane. Figure 2 shows the TOPBL2 Deploy job in progress. A system image is being deployed to the system TOPBL2. The status shows Downloading disk image (as master).

Figure 2. RDP Deployment Console

13

Figure 3 shows the status of the deployment on the TOPBL2 iLO console.

Figure 3. TOPBL2 iLO Console

14

ROM BIOS settings for BFS To boot from SAN, the ROM BIOS boot controller order must be changed. By default, the local HP Smart Array is first in the boot order. For BFS, the Fibre Channel HBA must be specified to be first in the boot order. To modify the Boot Controller Order, follow the documented procedures for your HP ProLiant system. Figure 4 shows the ROM BIOS Setup Utility interface. The Boot Controller Order option is selected.

Figure 4. ProLiant BL45p ROM BIOS setup utility

15

Figure 5 shows the Boot Controller Order function. The Fibre Channel HBA is selected, and then made first in the controller boot order.

Figure 5. Change controller boot order

16

HBA settings for BFS HBA settings must be changed for BFS configuration. After the ROM BIOS has been modified to boot first from the HBA, the HBA must be configured to enable the HBA BIOS and then point to the BFS LUN using the QLogic Fast!UTIL utility shown in Figure 6. The settings for each redundant HBA changed were: Host Adapter SettingsEnable the BIOS shown in Figure 7 Selectable Boot SettingsEnable selectable boot, then for each storage array pair, select the BFS LUN shown in Figure 8

Figure 6. QLogic Fast!UTIL HBA utility

17

The BIOS for the HBA is enabled here; it is changed from Disabled to Enabled.

Figure 7. QLogic Fast!UTIL Enable HBA BIOS

18

To point to the BFS LUN, the HBA selectable boot option is enabled, and then the storage array/LUN combination is specified.

Figure 8. QLogic Fast!UTIL selectable boot

Cluster failover before migration This process causes the server to be restarted as the first step causing applications to fail over to other nodes in the cluster. However, it is recommended to manually fail over any application to other nodes before starting this procedure as noted in the procedure. Again, this failover causes a short Exchange service outage. Additional passive cluster nodes If possible, the addition of another passive cluster node can prevent extended loss of Exchange service during the migration process when the cluster has one passive node. Since the migration process utilizes the passive node for migration, a failure in one of the remaining cluster members could result in extended loss of service. By adding another passive node, Exchange services are protected from an extended outage. If a node failure occurs on an active cluster member during the migration process of another node, then the additional passive node would be used.

19

Path failure recovery

Path failure testing was performed to ensure proper path failover for BFS disks. This testing was performed by initiating a load on the Exchange cluster with the Microsoft Load Simulator (LoadSim), then causing path failures to ensure that proper path failover for the BFS system disks occurred. The test SAN configuration had the following configuration: Two redundant HP StorageWorks SAN Switch 4/32 Fibre Channel switches. Two paths from each BL45p server to the SAN; each path went to a different switch. Two paths from each EVA controller to each switch. Zoning was configured so each HBA port could see each EVA storage controller. Testing was performed by disabling each path using the Brocade Fabric Manager interface. All possible failures were performed. These path failures caused MPIO to fail all system disks to a different path. Proper path failover to a redundant path occurred without errors at the system or Exchange level. Exchange service continued without interruption.

Recovery from the loss of a BFS disk

A recovery procedure is also required for the loss of a BFS disk. It is a given that HP StorageWorks EVA storage arrays provide high levels of availability and redundancy to prevent loss of a virtual disk LUN. However, it could happen even though the chances are remote. The following procedures show how to recover from the loss of a BFS LUN. If a BFS system disk fails, then two recovery options are available. Either the failed LUN is repaired then represented to the BL45p server or a new LUN is created and presented. Restoration of the RDP image took less than 60 minutes. Repairing the failed LUN Repairing a failed system disk can be accomplished by either restoring the system image from a backup set or re-imaging the failed LUN. If you recover from a system backup, then follow your local site recovery procedures. If you re-deploy the system image using RDP, then use this procedure:
1. Repair the LUN. 2. Initiate a system deployment job by way of RDP. Using the RDP Job wizard, create a job to deploy

the saved system image for the failed system. 3. Boot the BL45p server and press F12 at the appropriate time to perform the PXE boot process. RDP will deploy the system image to the repaired BFS disk. 4. Monitor the progress of the re-imaging process by way of iLO console. When the re-imaging is completed, the server is ready for service. See the RDP Users Guide for specific details. After the test was completed, the restored server was rebooted into the cluster. The Exchange cluster services were moved back to the node without incident.

20

New boot from SAN LUN If a new LUN is created, then the following procedure is used:
1. Using HP StorageWorks Command View EVA: a. Un-present the failed BFS LUN. b. Create a new system disk vdisk. c. Present this disk to the server, re-using the LUN from the failed LUN for the new Vdisk. 2. Bring up iLO console window for this server to monitor the progress of the RDP job. 3. Initiate a system deployment job by way of RDP. Using the RDP Job wizard, create a job to deploy

the saved system image for the failed BFS disk.

4. Boot the BL45p server and press F12 at the appropriate time to perform the PXE boot process. RDP

will deploy the system image to the repaired BFS disk.

5. Monitor the progress of the re-imaging process by way of iLO console. When the re-imaging is

completed, the server is ready for service. See the RDP Users Guide for specific details. After the test was completed, the restored server was rebooted into the cluster. The Exchange cluster services were moved back to the node without incident.

Recovery from the loss of a BL45p blade server

A recovery procedure is also needed in case a BladeSystem fails. Loss of an individual blade server may require a new replacement server. The recovery procedure involves substituting a new server, pointing the new server to the BFS LUN, and booting the new server. The server boots directly into the cluster and becomes an active member without any further recovery steps. There are two possible recovery procedures: Install a new blade server and re-use the HBA from the failed blade. Install a new blade server using a new HBA. New blade server re-using HBA from failed blade If possible, re-use the HBA from the failed blade. By re-using the HBA from the failed blade, the recovery procedure is simpler and shorter. Using the old HBA removes the need to modify the SAN zoning to see the new HBA. To swap HBAs between the new and the failed blade server, follow this procedure. Follow documented procedures for your ProLiant BladeSystem.
1. 2. 3. 4. 5. 6. 7.

Remove failed blade server from enclosure. Remove the HBA from failed blade server and insert into the new blade server. Place the replacement blade server into the enclosure. Bring up the iLO console for the new blade. Boot the new blade. Change the controller boot order in the ROM BIOS of the new to boot from the HBA. Ensure the HBA still has proper BFS LUN settings, although BFS LUN setting on the re-used HBA should be there already. Blade reboots and is ready for service.

This recovery procedure takes less than 30 minutes if a spare BladeSystem is always available. This procedure is also much faster than recovery from the loss of a local boot system.

21

New blade with new HBA If the old HBA from the failed blade cannot be re-used, then the new HBA has to be configured. Using a new HBA requires modifying the SAN zoning to remove the old HBA then adding the new HBA. It also requires changing the presentation ports for the host object on the EVA.
1. When installing the HBA into the new blade, note the WWIDs for the HBA. The WWIDs are noted

on the outside of the HBA card. 2. Swap the failed blade in the blade enclosure with the new blade server. 3. Bring up the iLO console for the new blade. The iLO interface may require updating to see the new blade. Follow iLO procedures for adding a new blade to the enclosure. 4. Boot the new blade. 5. Change the controller boot order in the ROM BIOS to boot from the HBA from the iLO console. 6. Change the HBA settings to boot from the existing BFS LUN that belonged to the failed server. 7. Before completing the HBA settings process, modify the SAN zoning and EVA host port settings for the new host. Blade reboots and is ready for service. This procedure takes somewhat longer than the previous procedure as additional work to modify SAN zoning to see the new HBA is required. This procedure takes less than 1 hour.

Summary
This solution demonstrates that deploying a Microsoft Exchange 2003 Server cluster consisting of HP ProLiant BL45p BladeSystems and BFS technology has many advantages. The solution further demonstrates that adding BFS has little effect on Exchange as there were no Exchange configuration changes required. Even during migration from local boot to BFS, Exchange services can be protected from an extended outage by adding an additional passive cluster member. This solution shows that using HP ProLiant BladeSystems and BFS technology combined with ProLiant Essentials RDP adds value to Exchange environments by increasing availability, improving disaster tolerance, lowering TCO, centralizing administration, simplifying system recovery, and reducing RTO.

22

Appendix A: Bill of materials

Quantity Microsoft Cluster Servers HP ProLiant BL45p server 3.5 Universal Plug SCSI Disk (300 GB) AMD 2.6-Ghz Opteron Processor HP 1-GB PC3200 DDR Memory (2 x 512-MB Sticks, 8 sticks per server) Dual port Fibre Channel Adapter (2 Gb) for ProLiant BL45p server HP BladeSystem p-Class Server Blade Enclosure with enhanced backplane components and the HP BladeSystem Management Suite HP ProLiant BL p-Class C-GbE Interconnect Kit (with pair of GbE Interconnect Switches and two QuadT Interconnect Modules)1,4,5 BL p-Class F-GbE2 Interconnect with or without optional ProLiant BL p-Class Storage Connectivity Kit (with 16 Fibre Channel connectors, required for HP ProLiant BLXXp server blade Fibre Channel connectivity when using the GbE2 Interconnect Switch) 4 2 per server 3 per server 16 4 1 374966-B21 350964-B22 390603-B21 376638-B21 381881-B21 380625-B22 Part number

249655-B21

321745 B21

1 (kit)

321745-B21

ProLiant BL p-Class Server Blade Enclosure with enhanced backplane components (includes 8 ProLiant Essentials Rapid Deployment Licenses) 1U power enclosure with six power supplies Ethernet cables (3 meter) Ethernet cables (2 meter) MPIO DSM Software HP ProLiant Essentials HP StorageWorks Replications Solutions Manager 1.2 HP Systems Insight Manager 5.0 HP Storage Essentials Enterprise Exchange plug-inStorage Essentials HP OpenView Operations for Windows LE HP Rapid Deployment Pack HP Vulnerability and Patch management pack HP ProLiant Essentials Performance Management Pack Microsoft Windows 2003 Enterprise Edition Microsoft Exchange 2003 Enterprise Server w/SP2 Microsoft Operations Manager 2005 SP1 Microsoft Windows XP Professional SP2 Microsoft Office 2003 Professional

1 2 16 16 4

281404-B22

378284-B21 Third party Third party

Included w/servers 1 1 1 4 1 1 1 1 4 4 1 16 16 T3680B Web download T4283A T4288AA BA217AA 269817-B21 371646-B21 306697-B21

23

Appendix B: Reference material

The following information was used in the integration of this solution. Reference these documents as background for configuring a BFS Exchange cluster configuration: HP ProLiant Essentials Rapid Deployment PackWindows Edition Users Guide; October 2005 (Sixth Edition), Part # 315378-006, Product Version 2.20 Storport in Windows Server 2003: Improving Manageability and Performance in Hardware RAID and Storage Area Networks:
http://www.microsoft.com/windowsserversystem/wss2003/techinfo/plandeploy/storportwp.mspx

Booting HP ProLiant server from a storage area network HOWTO:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00580093/c00580093.pdf

Boot from SAN in Windows Server 2003 and Windows 2000 server:
http://www.microsoft.com/windowsserversystem/wss2003/techinfo/plandeploy/BootfromSANinWindows.m spx

HP StorageWorks Booting Windows Server 2003 and Windows Server x64 Edition systems from a storage area network application notes:
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00099862/c00099862.pdf

Support for booting from a Storage Area Network (SAN):

http://support.microsoft.com/default.aspx?scid=kb;en-us;305547

Support for multiple clusters attached to the same SAN device:

http://support.microsoft.com/kb/q304415/

How to add a registry value to a Windows Server 2003-based computer that you start from a SAN so that the startup disk, the pagefile disks, and the cluster disks are all on the same SAN fabric:
http://support.microsoft.com/kb/886569

Highly Available Storage: Multipathing and the Microsoft MPIO Driver Architecture White Paper :
http://www.microsoft.com/WindowsServer2003/technologies/storage/mpio/default.mspx

Boot From SAN, Part I: Reducing Cost and Complexity of Server Management
http://www.QLogic.com/documents/datasheets/knowledge_data/whitepapers/boot_from_SAN_pt1.pdf

Boot From SAN, Part II: High Availability Environments:

http://www.QLogic.com/documents/datasheets/knowledge_data/whitepapers/boot_from_SAN_pt2.pdf

How to Change the Quorum Disk Designation: http://support.microsoft.com/kb/280353 Microsoft Exchange Server 2003 automated deployment with HP BladeSystem:
http://h71019.www7.hp.com/ActiveAnswers/cache/304010-0-0-0-121.html

Automation-assisted server recovery for HP BladeSystem using HP ProLiant Essentials Rapid Deployment Pack 2.2:
http://presales.hp.com/iss/blades/bladedocs/sae/ServerRecovery_Guide_R2.2_BladeSystem_RDP_200512 02.doc

Microsoft SAN Integration Technologies webpage:

http://www.microsoft.com/windowsserversystem/storage/sansupport.mspx

24

For more information

For more information about HP Solutions for Exchange 2003, visit:
http://www.hp.com/go/hpcft

2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. 4AA0-5603ENW, June 2006