Solution

Question: How do I properly configure HP-UX for Symmetrix storage?

Question: What are the proper settings for pvtimeout and lvtimeout?
Question: What are the proper settings for bad block replacement?
Question: How to properly configure for Aasynchronous I/O.
Question: How to properly configure for Sybase.
Environment: OS: HP-UX
Environment: Application SW: VERITAS Volume Manager (VxVM)
Environment: Protection: Asynchronous
Fix: This document describes the current EMC recommendations for setting up d
evice parameters and patches required for Symmetrix volumes on HP-UX.
Patches:
Currently, the only critical patch known to EMC is for HP-UX 11.00 and it is PHK
L_18543. However, for all versions of HP-UX, the customer should consider consul
ting with HP to obtain the latest recommended patches and dependencies for the f
ollowing cumulative patches:
lvm cumulative patch
lvm commands cumulative patch
scsi IO cumulative patch
fiberchannel mass storage driver cumulative patch
fiberchannel cumulative patch
tachyon TL fibre channel driver patch
cumulative SAM/ObAM patch
Not all of the above patches are applicable to all versions of HP-UX.
Timeout Values:
pvtimeout controls how long an I/O will wait before timing out on a physical pat
h to a device before it fails the I/O and attempts an alternate path. This path
switching will go on for as long as lvtimeout is set for. The default behavior i
s to allow an I/O to wait for 30 seconds on a path then attempt the I/O on anoth
er path and to continue to do so until the I/O can complete without EVER returni
ng a failure to the overlying application. This causes the application to stall
for as long as necessary to allow at least one path to a device to be restored.
If all paths to a volume time out, the volume becomes "Powerfailed". The reactio
n of application to a "powerfailed" condition varies. Typically, databases do no
t handle this well. In order to avoid this condition, we set the pvtimeout to 90
seconds so as to avoid the "powerfailed" condition. The default value for pvtim
eout is 30 seconds and the default value for lvtimeout is 0 (wait forever). The
one exception to this rule is fibre channel link down conditions. A link down wi
ll cause all I/O on all volumes on the link to timeout in 10 seconds.
EMC recommends that lvtimeout be set to 0 (this is the default value) and pvtime
out to a value of 90 seconds** for all systems running PowerPath and for all sys
tems running PVLINKS, to prevent application outages in all environments other t
han Sybase with Sybase controlled data mirroring.
This change may be accomplished as follows:
lvchange t 0 /dev/[lvname] #for each logical volume
pvchange t 90 /dev/dsk/[cXtXdX] #for each physical volume
The settings are persistent across reboots for all 11.X systems. For 10.20 syste
ms (depending on patch levels), it may be necessary to run a script to set these
values at each system boot up.
In a Sybase mirror environment, the recommendation would be to set pvtimeout to
90 and lvtimeout to 180 because Sybase host based mirroring requires that it han
dle it s own retries in order to properly break the mirror set.
**EMC recommends a larger than default value of 90 for pvtimeout to prevent unne
cessary path switch and errors in the syslog during normal operation events (suc
h as Symmetrix service operations). EMC recently changed the recommended value f
or pvtimeout from 180 to 90. Those sites that have the value set to 180 do not n
eed to change this setting unless the customer requests faster path fail-over ti
me during a path outage. EMC also recommends using the default and recommended H
P setting of 0 for lvtimeout. As always, discuss these settings with your system
s vendors to obtain the most current and accurate information. At the customers
option, the pvtimeout value can be left at default (30) during normal operations
and raised to 90 only during Symmetrix service operations to achieve faster fai
l-over times during normal operation.
The only known situation where lvtimeout should be changed from default is for a
pplications that MUST perform their own retries. The only application in wide us
e that requires this behavior is Sybase.
Bad Block Replacement Settings:
EMC Symmetrix arrays detect media errors perform their own bad block replacement
s. However, the Symmetrix makes use of media errors to indicate conditions than
normal media flaws. Therefore, to prevent the operating system from taking ANY a
ction in the case of uncorrectable media errors, the logical volume parameter fo
r bad block reallocation MUST be set to NONE (IMPORTANT: OFF is NOT the same as
NONE). One exception exists for this; if the customer is using Symmetrix volumes
in a an HPmirrorux mirrored volume, then these volumes should have bad block re
placement set to ON. This is required as the mirror software intercepts these er
rors and uses them to handle disabling of volumes.
To set the value correctly for Symmetrix, use the command:
lvchange r N [lvname] #for each logical volume
The default value of ON is for devices that report media errors but require that
the O/S send a specific request to re-allocate the bad block and the optional s
etting of OFF is for devices that do not support bad block replacement in any fo
rm and require that the host keep track of bad media blocks.
In the event of a volume failure with bad block replacement set incorrectly:
The bad block table may need to be cleared to regain access to the data. Interve
ntion from EMC technical support personnel is required. Contact the EMC Customer
Support Center or your service representative for technical assistance and quot
e this solution ID.
Special considerations for raw async I/O:
For customers running Oracle or other databases (other than Sybase mirroring) co
nfigured for raw Asynchronous I/O, the /dev/async device must be configured with
the minor number of 0x0. This will allow I/O to be retried on a redundant path
device. Oracle benchmarking software may require that this be set to 0x1, this w
ill not affect the retry behavior.
EMC recommends you consult with Oracle (or the appropriate database vendor) to d
etermine their current recommendation for this. In either case, the value must b
e 0x0 or 0x1 for EMC's redundant architecture to operate properly. For Sybase mi
rroring applications, the minor number must be set to 0x4 to allow Sybase to ret
ry the I/O after breaking the mirror set. For normal (non mirroring) Sybase conf
igurations, use the same settings as is recommended for Oracle. Consult with the
database vendor to get their current recommendations for timeout settings on HP
. Any discrepancies should be reported to Open Systems Software Support.
The following information is from the Oracle support site and is provided for re
ference only. Consult Oracle for current recommendations:
The below information is from Oracle Metalink website (current as of September 2
002):
2.0 Configuring asynchronous I/O
================================
Asynchronous I/O requires configuration of both Oracle and HP-UX
2.1 HP-UX Configuration
-----------------------
a. create the /dev/async character device
% /sbin/mknod /dev/async c 101 0x0
% chown oracle:dba /dev/async
% chmod 660 /dev/async
b. configure the async driver in the kernel using SAM
-> Kernel Configuration
-> Drivers
-> the driver is called 'asyncdsk'
Generate new kernel
Reboot
c. set max_async_ports using SAM
-> Kernel Configuration
-> Configurable Parameters
-> max_async_ports
max_async_ports limits the maximum number of processes that can concurrently use
/dev/async. Set this parameter to the sum of 'processes' from init.ora + number
of background processes.
The background processes started at instance startup will open /dev/async twice.
If max_async_ports is reached, subsequent processes will use synchronous I/O.
d. set aio_max_ops using SAM
-> Kernel Configuration
-> Configurable Parameters
-> aio_max_ops
aio_max_ops limits the maximum number of asynchronous I/O operations that can be
queued at any time. Set this parameter to the default value (2048), and monitor
over time (use glance).
2.2 Oracle Configuration
------------------------
The following init.ora parameters are required.
a. pre 7.3.0
no init.ora parameters required
b. 7.3.X
use_async_io = TRUE
c. 8.X
disk_asynch_io = TRUE
Veritas Volume Manager for HPUX
Power fail timeout (pfto) Can be changed on VERITAS volumes using:
vxdisk set rootdisk01 pfto=XX
where XX is a value in seconds. Currently EMC recommend setting this value to 9
0 seconds.
to check the current value:
vxdisk list [nnnnnn] | grep timeout
where: nnnnnn is the name of the volume.
A word about gatekeepers:
In a clustered environment, the temptation is to share gatekeepers between nodes
in the cluster. This is not supported. Each node must have it's own non shared
gatekeepers. In other words, there is no situation where a gatekeeper should be
mapped to more than one host.
Notes: This information should be confirmed for new installations, prior to ser
vice operations, and when creating new logical volumes.
Notes:
See white paper entitled "HP-UX Volume Manager I/O Time-out Parameters Configura
tion for Symmetrix" For additional details on volume timeout settings.
Notes:
In the event of a volume failure with bad block replacement set incorrectly, the
bad block table may need to be cleared to regain access to the data. Refer to E
MC Knowledgebase solution 6.0.16829617.2083487 for information on this procedure
.