Lpfc Emulex LightPulse Solaris HBA Driver

Version 5.00f Audience

Chapter 1 - Introduction
This document provides the information needed to use the Emulex LightPulse Fibre Channel host bus adapter (HBA) device driver, known as lpfc. For system administrators, this document includes information about the configuration files, differences particular to running the driver on SPARC and x86 platforms, how to install the driver and how to boot the system with the driver present. At the very least, system administrators should be familiar with Solaris and have access to standard system documentation. Anyone working with this product should have some familiarity with the nature and use of Fibre Channel.

Typographic Conventions
Bold face is used to highlight cautions and other important points. Italics are used to mark the first occurrences of words that will subsequently be explained or defined. Cross-references are underlined. Command syntax, file names, command output, and similar items are placed in courier font.

Fibre Channel, In Brief

Fibre Channel is a general-purpose, high-throughput, low-latency interconnect. It employs serial bit transmission over copper media, short-wave and long-wave optical media. Cable distances may range up to 30 meters for copper media and up to 10 kilometers for optical media. Transmission speeds currently range up to 1.0625 gigabaud, or roughly 100 megabytes per second. Fibre Channel transmission protocols provide high reliability, with bit error rates less than 1 in 10-12. Interconnects may be configured as point-to-point, loop, or fabric (network switch). The Fibre Channel specifications provide for the emulation of two traditional protocols: SCSI and IP. For storage, Fibre Channel provides an emulation of SCSI; this emulation is dubbed FCP, short for Fibre Channel Protocol a confusing acronym, in that it refers specifically to SCSI-on-Fibre Channel rather than to the lower-level protocols by which Fibre Channel itself operates. Throughout this document, we generally refer to FCP; when we mention SCSI, we are referring to the particular SCSI properties within FCP, or to the original SCSI protocol. For networking, Fibre Channel provides an encapsulation of IP (Internet Protocol), referred to in this document simply as IP.

General Description of the Device Driver

A device driver acts as a translator between an operating system and the hardware so that the operating systems kernel need not know the specifics of the device it uses. A device driver contains all of the code specific to operating a device and provides an input/output (I/O) interface to the rest of the system. The Emulex LightPulse HBA device driver for Solaris, lpfc, simultaneously supports two functions: IP and FCP over Fibre Channel. The FCP function provides access to Fibre Channel disk and tape drives. The IP function provides peer-to-peer networking, such as TCP or UDP, between Fibre Channel hosts. The section of the driver that handles FCP is

lpfc; the section that handles IP is also called lpfc. In this document, unless otherwise specified, lpfc also refers to the driver as a whole. Both the IP and FCP functions sit on top of common code in the driver that interfaces to the LightPulse adapter and handles Fibre Channel issues. The driver can be loaded dynamically, at any time, although typically the driver is loaded when the host boots. The IP and FCP functions can be loaded separately. The following diagram shows the software stack for network operations and SCSI operations from the standpoint of a Solaris host.
Application Kernel STREAMS stack (TCP/IP) DLPI LightPulse IP HBA driver, lpfc Common Driver Code Application Kernel Solaris Common SCSI Architecture (SCSA) framework SCSI target driver LightPulse FCP/SCSI HBA Driver, lpfc

lpfc

Emulex LightPulse Host Bus Adapter (HBA) Fibre Channel Interconnect

The FCP part of the driver is implemented within the Solaris Common SCSI Architecture (SCSA) framework, which defines the drivers expected behaviors and entry points from Solaris. (Recall that FCP emulates SCSI.) The SCSI target driver generates SCSI commands for each device in use, sending those commands to the lpfc HBA driver, which in turn has the job of transporting those commands to the device itself. The IP part of the driver, lpfc, uses the STREAMS/Data Link Provider Interface (DLPI). The DLPI uses the Solaris clone driver to allow multiple processes to open the device.

Chapter 2 - Specifics of the Device Driver

This chapter lays out the specifics of the device driver: what operating systems it supports, what hardware it supports, and what files and variables you may need to modify. It also discusses aspects of the drivers behavior that you must consider to successfully install and boot the system with the driver included.

Supported Operating Systems

The lpfc device driver supports Solaris 2.6, Solaris 7, Solaris 8 and Solaris 9 (both 32 and 64 bit) on SPARC platforms. On Intel x86 platforms, the operating system must have been patched Solaris 2.6 driver update 1 or later and will only work on single processor X86 systems. With Solaris 7 and Solaris 8 the driver should work on all x86 platforms listed in the Solaris Hardware compatibility guide.

Supported Hardware
On the hardware side, the lpfc driver supports Intel x86 platforms with Peripheral Component Interconnect (PCI) host bus subsystems and SPARC machines with Peripheral Component Interconnect (PCI) host bus subsystems and SBUS subsystems. The Power PC is not supported at this time. The lpfc driver supports the Emulex LightPulse Family of Fibre Channel host adapters, including the LightPulse 6000, LightPulse 7000, LightPulse 8000, LightPulse 8000S, LightPulse 9000, LightPulse 9002, LightPulse 9002S and LightPulse 9802. On all platforms, lpfc should be run with a minimum of 32 megabytes of host memory.

Important Files
You need to be aware of the following files because you may need to modify them after installing the driver. You might also need to modify one or more of these files if your hardware configuration changes, e.g., by adding additional disk drives to the Fibre Channel interface. lpfc.conf Configuration file specifying tunable parameters for all LightPulse adapters attached to the host machine; this file is located where you installed the driver, e.g., in /platform/<arch>/kernel/drv or in /kernel/drv. /kernel/drv/cmdk.conf Options for the SCSI disk driver. (Solaris 2.6 x86 only) /kernel/drv/sd.conf Options for the SCSIdisk driver (all other platforms). /kernel/drv/st.conf Options for the SCSI tape driver (all other platforms). /etc/hostname.lpfc<0-N> Configuration files specifying adapters for use with IP networking. For each LightPulse adapter involved in networking, there must be a hostname.lpfc<0-N> file in /etc containing the hosts name on that network. Note that these files need not be present if the driver is only being used for SCSI support. /etc/system Host-wide tunable parameters.

Configuring these files is discussed in greater detail in the following sections.

Configuration
The installation procedure for lpfc handles common cases but your hardware setup may require additional configuration changes beyond the common case. Ideally, you will make these changes when you install the driver and before you reboot the machine. For instance, you might modify lpfc.conf to allocate additional memory for use as buffers, configure multiple adapters for networking, or modify the SCSI target drivers configuration file to probe for additional targets on Fibre Channel. In rare circumstances, you may also have to modify /etc/system.

Device Numbering
When configuring the driver, there are various configuration parameters that rely upon device numbers. This section will attempt to explain the two different levels of device numbering and how they apply to specific configuration parameters. There are two different levels of device numbering. The first level of device numbering is the driver instance number. This is the number that gets displayed in log messages to syslog or the console, e.g. lpfcX. The lpfc.conf configuration parameters that use lpfcXtY, rely on X to be the driver instance number and Y to be the FCP target number. This format is used for configuring LUN throttles and persistent binding. In systems where adapters are moved around or changed, it is possible for the driver instance number to exceed the number of adapters. The second level of numbering is the SCSI controller number. This number is assigned by the system for each SCSI target driver it detects. It is the typically in the special files created to access each SCSI device, e.g. /dev/dsk/cXt0d0s2. Once these numbers get assigned, they will stay the same between reboots if there are no LightPulse hardware changes to the system.

Diagnostic Interface
A user library, which allows applications access to the Driver Diagnostic Interface, resides in /usr/lib/libdfc.a. The library features access to driver statistical information and a pass through CT interface for Nameserver queries. More detailed information can be found in the Functional Specification, Fibre Channel Diagnostic and Utility Library.

lpfc Configuration (lpfc.conf)

All the variables that control driver initialization may be found in a file called lpfc.conf. This file is located in the installation directory containing the binary for the driver itself, lpfc. Typically, this directory will be /platform/<arch>/kernel/drv or /kernel/drv. Other information is obtained dynamically by probing the attached Fibre Channel adapter. The variables described in this section are the ones a system administrator is most likely to be concerned with. The table that follows summarizes the variables in this section. It is not a substitute for reading the remainder of this section. Important: These values are set per adapter; e.g., setting num-iocbs to 1000 allocates 1000 I/O control blocks for every LightPulse present in the host machine. If you wish to have a configuration parameter apply to a specific LightPulse adapter, just prepend lpfcX- to the configuration parameter (where X is the driver instance number); e.g., setting lpfc0-num-iocbs to 2000 allocates 2000 I/O control blocks for the driver interface lpfc0.

Important: The fcp-bind-WWNN, fcp-bind-WWPN and fcp-bind-DID parameters are not apply to a specific LightPulse adapter. They are the global parameters. You can only use one type of bind per system. In the descriptions below, the annotation (F) means that the variable only has meaning when FCP functionality is enabled in lpfc; the annotation (I) means that the variable only has meaning when IP functionality is enabled in lpfc. Variables lacking either annotation have meaning for the entire driver, regardless of which specific functionality is enabled. A Boolean variable should be set to either 0 (FALSE) or 1 (TRUE), although any nonzero value will suffice for TRUE. The infinity symbol, , denotes the largest possible unsigned integer value (e.g., 232-1). Any delays or timers specified in seconds have a one second granularity. If you specify a delay of 2 seconds, this will give you a delay of somewhere between one and two seconds. Variable ack0 automap check-cond-err cr-delay Default 0 1 0 0 Min Max 0 1 0 3 0 0 1 63 Comments Use ACK0 for Class 2 Automatically assign SCSI IDs to FCP targets detected. (Boolean) Treat certain FCP Check conditions as FCP RSP errors. Specify a count of milliseconds after which an interrput response is generated if cr-count has not been satisfied. Specify a count of I/O completions after which an interrupt response is generated. (Boolean) Treat FCP RSP errs like no-device-delay. (Boolean) Decrement LUN throttle on queue fulls Amount to increment LUN queue depth each time Time interval, in secs, to increment LUN queue depth Bind specific SCSI IDs to targets based on Fibre Channel WWNN. Bind specific SCSI IDs to targets based on Fibre Channel WWPN. Bind specific SCSI IDs to targets based on Fibre Channel Port ID. Fibre Channel class for

cr-count

255

delay-rsp-err dqfull-throttle dqfull-throttle-up-inc dqfull-throttle-up-time fcp-bind-WWNN fcp-bind-WWPN fcp-bind-DID fcp-class

0 1 1 1 Inactive Inactive Inactive 3

0 0 0 0

1 1 128 30

N/A N/A N/A N/A N/A N/A 2 3

Variable fcpfabric-tmo fcp-on ip-class link-speed linkdown-tmo lpfcNtM-lun-throttle lpfcNtM-tgt-throttle lun-queue-depth network-on no-device-delay nodev-holdio nodev-tmo num-bufs num-iocbs scan-down tgt-queue-depth topology use-adisc use-lomempages log-only log-verbose

Default 0 1 3 0 30

Min Max 0 0 2 0 0 1 1 255 1 3 2 255 128 10240 128 1 30 1 255 4,096 10,240 2 10240

30 0 1 0 0 512 512 2 0 0x04 0 0 1 0

1 0 0 0 0 64 128 0 0

N/A N/A 0 0 0 0 1 1 1 0xff

post-ip-buf 128 64 1024 xmt-queue-size 256 128 10,240 ack0 If ack0 is 1, the adapter will try to use ACK0 when running Class 2 traffic to a device. If

Comments FCP data transmission. Extra FCP cmd timeout when connected to a fabric (in seconds). (Boolean) Turn on FCP Fibre Channel class for IP data transmission Set link speed: auto, 1G, 2G (seconds) How long driver waits before deciding FC link is down Override lun-queue-depth for lpfc adapter N target M Override tgt-queue-depth of lpfc adapter N target M Default max commands sent to a single logical unit(disk) (Boolean) Turn on IP networking. Sec delay to fail back an I/O Hold I/O errors forever if device disappears (seconds) to hold I/O if device disappears ELS and misc data buffers Adapter command/response structures Select method for ALPA to SCSI ID mapping Default max commands sent to a single target Fibre Channel link topology (defaults to loop only) (Boolean) Send ADISC instead of PLOGI for device discovery. (x86only) DMA memory allocation method Logging to log file only (bit mask) Extra activity logging Buffers to post to IP ring Xmit queue length for mbuf

the device doesnt support ACK0, then the adapter will use ACK1. If ack0 is 0, only ACK1 will be used when running Class2 traffic. automap (F) If automap is non-zero, SCSI IDs for all FCP nodes without persistent bindings will be automatically generated. If FCP devices are added to or removed from the Fibre Channel network when the system is down, there is no guarantee that these SCSI IDs will remain the same when the system is booted again. If one of the FCP binding methods below is specified, then automap devices will use the same mapping method to try to preserve SCSI IDs between link down and link up. If no persistent binding method is specified, the binding is by WWNN for automap 1 (default), by WWPN for automap 2, and by D_ID for automap 3. If automap is 0, only devices with persistent bindings will be recognized by the system. check-cond-err (F) The driver will treat check condition 0x44, Internal Target Failure, and check condition 0x20, Invalid Command Operation Code, as FCP RSP errors. cr-delay This value specifies a count of milliseconds after which an interrupt response generated if cr-count has not been satisfied. This value is set to 0 to disable the Coalesce Response feature as default. cr-count This value specifies a count of I/O completions after which an interrupt response generated. Default is set to 8. delay-rsp-err (F) The driver will delay FCP RSP errors being returned to the upper SCSI layer based on the no-device-delay configuration parameter. dqfull-throttle (F) dqfull-throttle-up-inc (F) dqfull-throttle-up-time (F) The dqfull-throttle causes lpfc to decrement a LUNs throttle (maximum queue depth) if a queue full condition is received from the target. The queue depth will be decremented down to a minimum of 1. The varibles dqfull-throttle-up-inc and dqfull-throttle-up-time are used to restore the queue depth back to the original. The dqfull-throttle-up-time parameter defines a time, in seconds, that is used to tell when to increase the current queue depth. If the current queue depth isn't equal to the lun-queue-depth, and the driver stop_send_io flag is equal to 0 for that device, increment the current queue depth by dqfull-throttle-up-inc (don't exceed the lun-queue-depth). So, if both parameters are set to 1, then driver increment the current queue depth once per second until it hits the lunqueue-depth. The only other way to restore the queue depth (besides rebooting), back to the original LUN throttle, is by running the command /usr/sbin/lpfc/resetqdepth X. This will restore the LUN throttle of all LUNs for adapter X back to the original value. fcp-bind-WWNN (F) Setup persistent FCP bindings based on a target devices WWNN. It is a global paramenter across all the LigthPulse adapters in the system. This binding guarantees that target assignments will be preserved between reboots. The format for a bind entry is NNNNNNNNNNNNNNNN:lpfcXtY where NNNNNNNNNNNNNNNN is a 16 digit representation of the targets WorldWide Node Name, X is the driver instance number and Y is the target assignment. Multiple entries must be separated by a comma (,) with

the last entry terminated with a semi-colon (;). Target assignments, with all supported LUNs must also be configured in sd.conf, st.conf, or cmdk.conf. A sample entry follows: fcp-bind-WWNN="20000020370c396f:lpfc1t0", "20000020370c27f7:lpfc0t2"; fcp-bind-WWPN (F) Setup persistent FCP bindings based on a target devices WWPN. It is a global paramenter across all the LigthPulse adapters in the system. This binding guarantees that target assignments will be preserved between reboots. The format for a bind entry is NNNNNNNNNNNNNNNN:lpfcXtY where NNNNNNNNNNNNNNNN is a 16 digit representation of the targets WorldWide Port Name, X is the driver instance number and Y is the target assignment. Multiple entries must be separated by a comma (,) with the last entry terminated with a semi-colon (;). Target assignments, with all supported LUNs must also be configured in sd.conf, st.conf, or cmdk.conf. A sample entry follows: fcp-bind-WWPN="21000020370cf8263:lpfc1t0"; fcp-bind-DID (F) Setup persistent FCP bindings based on a target devices Port ID. It is a global paramenter across all the LightPulse adapters in the system. This binding guarantees that target assignments will be preserved between reboots. The format for a bind entry is NNNNNN:lpfcXtY where NNNNNN is a 6 digit representation of the targets Port ID, X is the driver instance number and Y is the target assignment. Multiple entries must be separated by a comma (,) with the last entry terminated with a semi-colon (;). Target assignments, with all supported LUNs must also be configured in sd.conf, st.conf, or cmdk.conf. A sample entry follows: fcp-bind-DID="0000ef:lpfc0t0"; fcp-class (F) The lpfc driver is capable of transmitting FCP data in Class2 or Class 3. The lpfc driver defaults to using Class 3 transmission. fcpfabric-tmo (F) Extra timeout value, in seconds, to be applied to each FCP command sent. When connecting through a large fabric, certain devices may require a longer timeout value. fcp-on Enable the FCP functionality of lpfc. This variable is a Boolean; zero disables FCP while non-zero enables it. ip-class (I) Fibre Channel is capable of transmitting IP data in Class2 or Class 3. The lpfc driver defaults to using Class 3 transmission. link-speed This causes lpfc to set the adapter link speed to auto-detect (0), 1 gigabit/sec (1) or 2 gigabits/sec (2). This setting only applies to the LightPulse 9002, 9002S and 9802 adapter. linkdown-tmo How long the driver will wait (1 to 255 seconds) before declaring the Fibre Channel link to be down. Commands waiting to be serviced will be aborted. For instance, FCP commands will be returned to the target driver. The lower this value is, the quicker the driver will fail commands back to the upper levels. There is a tradeoff here: small values risk retrying commands when the link is bouncing, while large values risk delaying failover in a redundant loop environment. Setting linkdown-tmo to 0 will cause all

commands issued to the device driver to be held internally (never failed) until the link comes back up. lun-queue-depth (F) The driver uses this value as the default limit for the number of simultaneous commands to issue to a single logical unit on a single target on the loop. A single logical unit will never be sent more commands than allowed by lun-queue-depth; however, less may be sent when sd-max-throttle or tgt-queue-depth is reached for the entire target. lpfcNtM-lun-throttle (F) The maximum number of outstanding commands to permit for any logical unit on a specific target. This value overrides lun-queue-depth. LpfcNtM-tgt-throttle (F) The maximum number of outstanding commands to permit for any target, including all LUNs on that target. This value overrides tgt-queue-depth. network-on This variable controls whether lpfc provides IP networking functionality over Fibre Channel. This variable is a Boolean: when zero, IP networking is disabled; when nonzero, IP networking is enabled. no-device-delay This variable [0 to 30] - determines the length of the interval between deciding to fail an I/O because there is no way to communicate with its particular device (e.g., due to device failure or device removal) and actually failing the command. A value of zero implies no delay whatsoever. This delay is specified in seconds. A minimum value of 1 (1 second) is recommended when communicating with any Tachyon based device. nodev-holdio This variable controls if I/O errors are held by the driver if a device on the loop disappears. If set, I/O errors will be held until the device returns back to the loop (potentially indefinitely). The upper sd layer may retry the command once the error is returned. nodev-tmo This variable controls how long I/O errors are held by the driver if a device on the loop disappears. If set, I/O errors will be held for the specified number of seconds. The upper sd layer may retry the command once the error is returned. num-bufs This variable specifies the number of command buffers to allocate. These buffers are used for Fibre Channel Extended Link Services (ELS), and one for each FCP command issued in SLI-2 mode. If you want to queue lots of FCP commands to the adapter, then you should increase num-bufs for better performance. These buffers consume physical memory and are also used by the device driver to process loop initialization and rediscovery activities. Important: The driver must always be configured with at least several dozen ELS command buffers; we recommend at least 256. num-iocbs This variable indicates the number of Input/Output control block (IOCB) buffers to allocate. IOCBs are internal data structures used to send and receive I/O requests to and from the LightPulse hardware. Too few IOCBs can temporarily prevent the driver from communicating with the adapter, thus lowering performance. (This condition is not fatal.) If you run heavy IP traffic, you should increase num-iocbs for better performance.

scan-down (F) There are three scanning algorithms used to map an ALPA to a SCSI ID in a private loop. Refer to the informative annex of the FC-AL profile, which describes the algorithm for this translation. Scan-down values 0 and 1 do not apply if a loop map is obtained. See the FC-AL profile for the definition of a loop map. If the adapter is not attached to a fabric, these parameters do not apply (use persistent binding). If persistent binding is used, scan-down MUST not be 2. If scan-down is 1, SCSI IDs are scanned starting from ALPA 0xEF down through ALPA 1 (e.g. If only two targets exist on the loop, the one with the highest ALPA gets assigned SCSI ID 0 and the other one gets SCSI ID 1). If scan-down is 0, SCSI IDs are scanned starting from ALPA 1 on up (e.g. If only two targets exist on the loop, the one with the lowest ALPA gets assigned SCSI ID 0 and the other one gets SCSI ID 1). If scan-down is 2 (default value), SCSI IDs are hard coded based on the ALPA (0xEF=SCSI ID 0, 0xE8=SCSI ID 1,). This parameter is used to maintain backward compatibility with the SCSI ID assignments of the 2.05 Solaris driver. tgt-queue-depth (F) The driver uses this value as the default limit for the number of simultaneous commands to issue to a single target on the loop. A value of 0 causes no target throttling to occur. A single target will never be sent more commands than allowed by tgt-queue-depth; however, less may be sent when sd-max-throttle is reached for the entire target. topology This variable controls the Fibre Channel topology expected by lpfc at boot time. Fibre Channel offers point-to-point, fabric, and arbitrated loop topologies. This variable encodes these topologies as follows: 0 Attempt loop mode, if it fails attempt point-to-point mode 2 Attempt point-to-point mode only 4 6 Attempt loop mode only Attempt point-to-point mode, if it fails attempt loop mode (default)

To make the adapter operate as an N_Port, select point-to-point mode (used for N_Port to F_Port, and N_Port to N_Port connections). To make the adapter operate in a Fibre Channel loop as an NL_Port, select loop mode (used for private loop and public loop topologies). The driver will reject an attempt to set the topology to a value not in the above list. The auto-topology settings 0 and 6 will not work unless the adapter is using firmware version 3.20 or higher. use-adisc This variable controls the ELS command used for FCP address authentication during discovery. If set, ADISC is used, otherwise, PLOGI is used. use-lomempages This variable applies only to x86 platforms. The setting of this variable can affect whether lpfc runs correctly on an x86 host. The Solaris DDI/DKI requires that kernel memory be allocated using special primitives if that memory may be the target of a DMA operation. On the x86, this memory comes from a special low memory pool that is

guaranteed to be safe for DMA operations across all x86 platforms. (Some platforms do not correctly handle DMA operations on physical memory above the 16 megabyte boundary.) Unfortunately, the size of the low memory pool is very limited; in fact, the standard Solaris releases defaults to a size of 36 pages in this pool. While the size of the pool can be increased (see System Wide Configuration (/etc/system)), on most x86 systems a reasonable alternative is to not use this pool at all. On these systems, ordinary memory works correctly as a target for DMA operations. As lpfc is distributed, we set the value of this variable to 0, disabling the use of the low memory pool. If you suspect that your x86 platform cant handle arbitrary DMA correctly above the 16MB boundary, set this variable to 1; you must also increase the size of the low memory pool. log-only When set to 1, log messages are only logged to syslog. When set to 0, log messages are also printed on the console. log-verbose (bit mask) When set to non-zero this variable causes lpfc to generate additional messages concerning the state of the driver and the I/O operations it carries out. These messages may go to the system console and/or the system log file, /var/adm/messages. See Error Messages for detailed information on the bit mask. post-ip-buf (I) This variable specifies the number 4K STREAMS buffers to allocate and post to the fibre channel IP ring. Increase this setting for better IP performance under heavy loading. xmt-que-size (I) This variable specifies the number of network packets that can be queued or outstanding at any time in the driver. Increase this setting for better IP performance under heavy loading.

Probing for FCP Targets -- Target Driver Configuration (cmdk.conf, sd.conf)

The following discussion applies only when you have enabled FCP functionality in lpfc. Solaris target drivers must be instructed which disks to probe for at boot time. These instructions are contained in a configuration file specific to each target driver. On SPARC the disk target driver is called sd; on x86, the disk target driver is called sd for Solaris 7 and cmdk for Solaris 2.6. The configuration files are, respectively, sd.conf and cmdk.conf and are usually found in /kernel/drv. There will only be one of these two target drivers on your system. See the driver.conf(4) man page for additional details. The tape target driver is called st.conf, and entries must be included in this file in order to run tape devices. Each disk drive on the Fibre Channel must have an entry in sd.conf or cmdk.conf. The system administrator may have to define additional targets and LUNs in these files. For FCP there can be up to 512 targets, with a maximum of 256 Logical Unit Numbers (LUN) per target. Initially, the sd.conf or cmdk.conf file has entries of the format:
name="sd" class="scsi" target=0 lun=0;

We use the class keyword to ensure that Solaris specifically probes all adapters controlled by all drivers, that register themselves as class="scsi", for the specified targets and LUNs. The lpfc driver registers itself as class="scsi The installation procedure for lpfc automatically added to sd.conf or cmdk.conf to e look like this:
#name=sd #name=sd #name=sd #name=sd #name=sd parent=lpfc parent=lpfc parent=lpfc parent=lpfc parent=lpfc target=16 target=17 target=17 target=17 target=17 lun=0; lun=0; lun=1; lun=2; lun=3;

(On x86, Solaris 2.6, you will see name=cmdk.) No entries are automatically added to the st.conf file. If you want to use disks (e.g., RAID arrays) with non-zero LUNs, or more than 18 targets, simply add lines to the target driver configuration file specifying the desired target ID and LUN(s). For instance, you can include following example in your target driver configuration file: name=sd parent=lpfc target=17 lun=0; name=sd parent=lpfc target=17 lun=1; or name=sd class=scsi target=17 lun=0; name=sd class=scsi target=17 lun=1; (On x86, Solaris 2.6, you will see name=cmdk.) We use the parent keyword to ensure that Solaris specifically probes all adapters controlled by the lpfc driver for the specified targets and LUNs. Note that changes to a target drivers configuration file will only take effect after the system has been rebooted.

Target / LUN blocking

As described above, the parent=lpfc and class="scsi" entries cause the SCSI layer to probe multiple adapters, even multiple adapters across multiple drivers. There is a method where the administrator can limit the SCSI layer probing of targets and LUNs on

an adapter by adapter basis. This gives the administrator control over which targets and LUNs are seen by each initiator (Target / LUN blocking). To do this, one first needs to find the device (driver interface) number for the adapter desired. After you boot the system, with your LightPulse hardware installed, you can check the output of dmesg(1M) for a message of the following format: NOTICE: lpfcX: WWPN:10:00:00:00:c9:YY:YY:YY WWNN:10:00:00:00:c9:YY:YY:YY Where lpfcX is the interface for a specific adapter. Once you know the device number for an adapter, you can construct entries in the sd.conf or cmdk.conf file to probe a specific adapter for the desired target / LUN. The file format for this entry follows: name=sd parent=lpfc target=16 lun=0 hba=lpfc0; Note: This entry will not cancel the effect of any other class="scsi" entries for target=16 lun=0. If the user wants the SCSI layer to only probe for target=16 lun=0 on device lpfc0, the class="scsi" entries for target=16 lun=0 need to be deleted. A user can cause system problems if certain class=scsi entries are deleted. These entries are used by the SCSI adapter, so if there is a SCSI boot disk at target=0 lun=0, whose probe entry has been deleted, the system wont boot. Similarly, if any SCSI targets probe entry is deleted, that device wont operate. To guarantee the Fibre Channel and SCSI probing wont conflict, use persistent binding to assign FC devices target numbers greater then 15. Persistent binding can be used to perform target blocking, but not LUN blocking.

Configuring IP Networking (/etc/hostname.lpfc<0-N>)

When configuring for IP service you need to supply the name of the host machine that Solaris should use for a particular network interface. At installation time, if you indicated that lpfc should provide IP networking service over Fibre Channel, the installation script automatically set up /etc/hostname.lpfc0. However, if the system has more than one LightPulse attached to it, and you wish to configure more than one of those adapters for networking, you must make additional hostname.lpfc<N> files by hand (where N is the number of the LightPulse adapter being configured). Upon rebooting, Solaris will then automatically configure and set up the additional adapters for networking. Note that whatever names you specify in these hostname.lpfc<0-N> files must have valid IP addresses associated with them, via NIS or DNS or whatever hostname-toIP-address translation scheme you employ. Because lpfc loads unusually early in the boot sequence, it may be necessary for you to put the hostname to IP address translation in /etc/hosts or some other locally available file. If you add IP networking support after the initial driver installation, be sure to update the settings in lpfc.conf. Alternately, re-run the installation procedure (doing so is called an update install), which will give you the opportunity to enable IP networking support, and which will then update all the configuration file settings for you. You will have to reboot the machine for these changes to take effect.

System-Wide Configuration (/etc/system)

Normally, you should not need to modify the contents of /etc/system on behalf of lpfc. The installation procedure for the driver should take care of any required changes

for /etc/system. All of the commonly used variables are in lpfc.conf and in the target driver configuration file. Below, we discuss the small number of cases that may require you to modify /etc/system for operation of lpfc. Refer to the system(4) manual page for additional details. moddir This variable specifies a series of directories that the kernel will search for loadable device drivers at boot time. Normally, you will install lpfc into a standard directory, so no moddir variable may be needed at all. If you specify a non-standard installation directory, the lpfc installation procedure will create or update the moddir variable to include the new installation directory. If that effort fails, or if you later decide to move lpfc, you will need to create or update moddir. forceload loads the specified modules at boot time, just before mounting the root filesystem, rather than at first reference. Heres an example, which youll find in /etc/system after youve installed lpfc (assuming you enabled IP networking): forceload: drv/clone The kernel locates the modules specified in forceload by consulting moddir. If you enable IP networking for the LightPulse driver, the drv/clone module must be forceloaded for correct operation. The lpfc driver is dependent upon the clone driver. Normally, the clone driver would load itself automatically when our driver calls it the first time. However, the lpfc driver is loaded unusually early in the system boot sequence; therefore, we must force the loading of the clone driver. lomempages applies only to x86 systems. It defines the amount of memory in the kernels low memory pool. Memory in this pool is guaranteed to be suitable for DMA across all x86 platforms. The default is set by the operating system kernel to 36 pages. There are no minimum and maximum values for this variable. We strongly recommend that you do not set it fewer than 36 pages. You only need to adjust the value of lomempages if you turn on use-lomempages in lpfc.conf (See On most systems, you will not need to enable use-lomempages and so wont need to adjust the value of lomempages in /etc/system. If you must specify a value for lomempages, the value you specify should be large enough to account for all the buffers allocated by lpfc for DMA. The primary value you should consider is num-bufs in lpfc.conf. However, under no circumstance should you specify a value for lomempages less than the systems default of 36 pages.

Chapter 3 Installation, Update and Boot Instructions

When installing the device driver, you must use supported versions of the hardware and software. See the Supported Operating Systems and Supported Hardware sections. Read the Configuration section before you start the installation. To install the driver and boot the system, follow these instructions. While it is possible to install lpfc onto one or more clients from a server, that procedure is not covered here; refer to the Solaris documentation for that case. If a prior version of the lpfc device driver is already installed on the system, you must first remove the old driver using the pkgrm procedure at the end of this chapter. 1. You must be running as root. Therefore, you must login as or su to root. 2. Load the package from your distribution medium into a directory, referred to here as <where-you-put-it>. The driver will be a regular tar file, named lpfc-sparc.tar or lpfc-i386.tar. 1. cd <where-you-put-it> 2. Extract the installation image from the tar file tar xvf lpfc-sparc.tar 3. At the shell prompt, type: pkgadd -d `pwd` 4. Answer the following:
Select package(s) you wish to process (or all to process all packages). (default: all) [?,??,q]:

5. Select the number associated with lpfc for installation. If you need help, type? or ??. You will see some additional questions related to directories. Ordinarily, the defaults are sufficient. Note that a default is selected by simply typing a carriage return. 1. Answer the question:
Rebuild manual pages database for section 7d [y,n,?]:

Manual pages for lpfc are installed is section 7d of the online man pages. Normally the catman command can be run to create preformatted versions of the on-line manual from the nroff(1) input files. Each manual page is examined and those whose preformatted versions are missing or out of date are recreated. If any changes are made, catman recreates the windex database. Depending on your system, this operation can take anywhere from 1 to 10 minutes. If you type n , the catman will be skipped which will allow the installation to complete quickly. The catman command skipped will be output for the administrator to run at a later point in time, if desired. 2. The next question you will see is:
Use IP networking over Fibre Channel [y,n,?]:

If you answer n, go on to Step 5. If you answer y, you must answer the following question:
Network hostname (for first adapter) [?,q]:

The answer you type should be a valid network hostname that will be used to identify this machine on the Fibre Channel network attached via the LightPulse adapter. A valid

hostname must be associated with an IP address (in /etc/hosts) for the adapter that is unique to the adapters Fibre Channel network. 6. Continue with the installation by answering the following in the affirmative:
Do you want to continue with the installation of <lpfc> [y,n,?]:

7. The install package provides running commentary on the installation process. Be sure to examine the output for any errors or warnings. When the installation concludes, take notice of the message: SCSI: If you are using lpfc to access disks, be sure to check the configuration file of your SCSI target driver (presumably sd.conf) to ensure that the driver will probe for all of the targets/luns in your environment. 8. Finally you will be asked:
Select package(s) you wish to process (or all to process all packages). (default: all) [?,??,q]:

This is the same question your session started with. If you do not want to install additional packages, type q. The lpfc driver installation is now complete. After you quit, you will see the following message: ***IMPORTANT NOTICE *** This machine must now be rebooted in order to ensure sane operation. 9. Before you reboot the machine, consider whether you must modify any configuration files, beyond the defaults established by the installation process. 1. Inspect lpfc.conf. If necessary, change settings as you deem appropriate. However, the defaults are likely to be satisfactory. You can find lpfc.conf in the driver directory you selected in Step 5A. 2. If you have FCP targets with multiple LUNs, add configuration entries for them to /kernel/drv/sd.conf or /kernel/drv/cmdk.conf (Solaris 2.6 x86). These files were set up by the installation process to probe for some FCP targets on the loop but only for LUN 0 on some targets. If you have FCP devices with multiple LUNs or target IDs greater than 17, you will need to modify sd.conf or cmdk.conf to find them at boot time. 3. If you have more than one LightPulse adapter that will be configured for IP networking in the host, you must create an address file for each additional adapter. The installation process created /etc/hostname.lpfc0 for the first adapter. 4. On an Intel x86 host, you may need to increase the size of the low memory pool. SeeSystem Wide Configuration (/etc/system) for details. Most Intel x86 hosts do not require any adjustment. 5. If you have a SPARC machine, go to Step 6. If you have an x86 machine, read the note below and follow the accompanying directions. NOTE: On Intel x86 platforms, there may be a problem running the Emulex LightPulse adapters in conjunction with certain video drivers. Any video adapter can be run in 640x480 mode but higher resolutions may lead to an inability to access the LightPulse adapter, incorrect video operation, or both problems. For instance, you may see

messages indicating that the driver is unable to initialize the adapter, or unable to talk to the Fibre Channel, or unable to carry out commands. Another manifestation is that the entire system might freeze and have to be either reset or power cycled. There are two workarounds for this problem. One workaround is to not run the X windows system at all. For instance, you may disable desktop login:
# /usr/dt/bin/dtconfig -d

By making the desktop login change, you guarantee that X windows will never be started automatically. Note that the system must be rebooted for this change to take effect. The second workaround is to configure the window system to run in 640x480 VGA mode rather than in a higher resolution. Consult your Solaris documentation for details on changing the resolution of the windowing system. If you do not take this bug into account, when you reboot the system you may be confronted with one of the failure modes described above. 10. Reboot the system to incorporate the new driver. At the shell prompt, type the following: # sync # reboot The system will reboot to multi-user mode and the Emulex LightPulse will be available for use. If you are enabling IP networking, you may find immediately after rebooting the system in step 10 that FCP disk access works but host-to-host IP access does not work at all. The lpfc network interface isnt even configured; as Solaris boots, it will print out an error message when configuring lpfc0, the Fibre Channel network, indicating that there is no such interface. Running netstat -i will also reveal that there is no lpfc interface. All you must do is reboot the machine a second time. In certain rare cases, it may be necessary to reboot the machine a third time. Be careful, though: if netstat -i shows an lpfc interface, but the network still isnt being started, you probably have an error in your hostname.lpfc<0-N> file or some other error in your network configuration files. pkgrm procedure The driver can be removed by using pkgrm. All the lpfc files will be removed, including the file containing the driver itself. The next time the system is rebooted, the driver will not be loaded. All system resources disk space, memory, etc. will be reclaimed. Two files are actually saved when a pkgrm is performed. The file /kernel/drv/lpfc.conf is moved to /usr/tmp/lpfc.conf.pkgrm to allow a user to restore any customized parameter settings after installing a new device driver. The file /kernel/drv/sd.conf is copied to /usr/tmp/sd.conf.pkgrm because the pkgrm process deletes any lpfc specific settings in the original file. After installing a new device driver, the saved file can be used to restore all the customized disk settings. To remove the lpfc device driver, follow these instructions: 1. You must be running as root. Therefore, you must login as or su to root. 2. At the shell prompt, type:

pkgrm lpfc 3. Perform a reconfiguration reboot into single user mode by typing: reboot -- -rs 4. Type the following command to clean up the old structures: (Solaris 7 or higher) disks -C 5. Install the new lpfc device driver by performing steps 1 10 above in this chapter. When you install the driver with pkgadd, two files are saved if they exist, just in case they are needed. The file /kernel/drv/lpfc.conf is saved in /usr/tmp/lpfc.conf.pkgadd. The file /kernel/drv/sd.conf is saved in /usr/tmp/lpfc.conf.pkgadd. For further information on installing and removing drivers, consult the Solaris system administration documentation and the pkgadd(1M) and pkgrm(1M) manual pages. 4.x PCI and SBUS Driver Update procedure The driver can be updated by saving certain driver configuration files and using pkgrm. Two files are saved when a pkgrm is performed. For 4.x PCI driver, lpfc, the file /kernel/drv/lpfc.conf is moved to /usr/tmp/lpfc.conf.pkgrm to allow a user to restore any customized parameter settings after installing a new device driver. For 4.x Sbus driver, lpfs, the file /kernel/drv/lpfs.conf is moved to /usr/tmp/lpfs.conf.pkgrm. The file /kernel/drv/sd.conf is copied to /usr/tmp/sd.conf.pkgrm because the pkgrm process deletes any lpfc or lpfs specific settings in the original file depending on removing lpfc or lpfs. After installing a new device driver, the saved file can be used to restore all the customized disk settings. If you are running both 4.x PCI (lpfc) and SBUS (lpfs) driver in your system and want to update the drivers, follow these instructions: 1. Login as or su to root. 2. Save the file /etc/path_to_inst to /usr/tmp/path_to_inst.lpfc. This file is not saved when a pkgrm is performed. 3. Save the file /kernel/drv/sd.conf to /kernel/drv/sd.conf.lpfc. This step is needed because the /kernel/drv/sd.conf.pkgrm will not have the correct setting after step 4 and 5 are performed. 4. At the shell prompt, type: pkgrm lpfs 5. At the shell prompt, type: pkgrm lpfc 6. Copy the /etc/path_to_inst. file /usr/tmp/path_to_inst.lpfc back to

7.

Install the new driver package:

pkgadd d `pwd`

CAUTION: lpfs will be renamed to lpfc and the instance number of lpfs will be modified in the file /etc/path_to_inst. On screen, you can find the following messages for lpfs devices, when installation completed. Move lpfs0 to lpfc3 Move lpfs1 to lpfc4 Write down those messages, you will need them at the next step.

6. Restore any customized parameter settings in sd.conf and lpfc.conf, follow these
instructions:

Copy the file /usr/tmp/sd.conf.lpfc back to /kernel/drv/sd.conf. Merge the file /usr/tmp/lpfs.conf.pkgrm and /usr/tmp/lpfc.conf.pkgrm into /kernel/drv/lpfc.conf. Rename lpfs to lpfc in both /kernel/drv/lpfc.conf and /kernel/drv/sd.conf files. Replace old lpfs instance number to the new lpfc instance number in the file /kernel/drv/lpfc.conf. Reboot the system.

7.

4.x SBUS Driver Update procedure

If you update from the 4.x Sbus driver, you need to read through this section carefully. Otherwise, go to the General Driver Update procedure section. The driver can be updated by saving certain driver configuration files and using pkgrm. Two files are saved when a pkgrm is performed. The file /kernel/drv/lpfs.conf is moved to /usr/tmp/lpfs.conf.pkgrm to allow a user to restore any customized parameter settings after installing a new device driver. The file /kernel/drv/sd.conf is copied to /usr/tmp/sd.conf.pkgrm because the pkgrm process deletes any lpfs specific settings in the original file. After installing a new device driver, the saved file can be used to restore all the customized disk settings. To update the lpfs device driver, follow these instructions: 1. Login as or su to root.

2. Save the file /etc/path_to_inst to /usr/tmp/path_to_inst.lpfc. This file is not saved when a pkgrm is performed. 3. At the shell prompt, type: pkgrm lpfs 4. Copy the /etc/path_to_inst. 5. file /usr/tmp/path_to_inst.lpfc back to

Install the new driver package:

pkgadd d `pwd`

CAUTION: lpfs will be renamed to lpfc and the instance number of lpfs will remain the same in the file /etc/path_to_inst. On screen, you can find the following messages for lpfs devices, when installation completed. Move lpfs0 to lpfc0 Move lpfs1 to lpfc1

7. Restore any customized parameter settings in sd.conf and lpfc.conf, follow these
instructions:

Copy the file /usr/tmp/sd.conf.pkgrm back to /kernel/drv/sd.conf. Merge the file /usr/tmp/lpfs.conf.pkgrm to /kernel/drv/lpfc.conf. Rename lpfs to lpfc in both /kernel/drv/lpfc.conf and /kernel/drv/sd.conf files. 7. Reboot the system.

General Driver Update procedure

The driver can be updated by saving certain driver configuration files and using pkgrm. Two files are saved when a pkgrm is performed. The file /kernel/drv/lpfc.conf is moved to /usr/tmp/lpfc.conf.pkgrm to allow a user to restore any customized parameter settings after installing a new device driver. The file /kernel/drv/sd.conf is copied to /usr/tmp/sd.conf.pkgrm because the pkgrm process deletes any lpfc specific settings in the original file. After installing a new device driver, the saved file can be used to restore all the customized disk settings. To update the lpfs device driver, follow these instructions:

1. Login as or su to root. 2. Save the file /etc/path_to_inst to /usr/tmp/path_to_inst.lpfc. This file is not saved when a pkgrm is performed. 3. At the shell prompt, type: pkgrm lpfc 4. Copy the /etc/path_to_inst. 5. file /usr/tmp/path_to_inst.lpfc back to

Install the new driver package:

pkgadd d `pwd`

6.

Restore any customized parameter settings in sd.conf and lpfc.conf.

Copy the file /usr/tmp/sd.conf.pkgrm back to /kernel/drv/sd.conf. Merge the file /usr/tmp/lpfc.conf.pkgrm to /kernel/drv/lpfc.conf. 7. Reboot the system.

Chapter 4 - Operational Issues and Known Bugs The following operational issues and bugs were known at the time the documentation was created. An operational issue is a property of the driver alone or in conjunction with Solaris, that results in behavior worthy of being called to the attention of the system administrator. A bug is a Solaris defect, an lpfc driver defect, a host bus adapter hardware or firmware defect, or a disk drive hardware or firmware defect that may be fixed in a subsequent revision of the software or hardware in question.

Operational Issues FCP

Discovering Disk Drives: drvconfig(1M) and disks(1M) Solaris provides a utility, drvconfig(1M), intended to probe for new hardware added after the system is booted. First run drvconfig, then run disks to acquire access to new devices. Sometimes, the driver recognizes disk drives at boot time but you wont be able to use those drives because special file entries for them do not exist. In that case, run the disks(1M) command to create the special file entries in /dev/dsk and /dev/rdsk. In some cases, you may need first to run drvconfig(1M), which creates the /devices tree, and then run disks(1M). Finally, in extreme cases, do the following, as root:
# touch /reconfigure # reboot

A reconfiguration reboot will cause Solaris to rediscover all possible devices at boot time. Different SCSI Target IDs on Different Hosts A Fibre Channel target is assigned its D_ID at loop initialization time; the SCSI target ID for that target is assigned by the device driver when the device is first discovered. It is possible for the D_ID to change between one loop initialization and the next. Every time a system boots or a target is added to or removed from the Fibre Channel, the loop will be re-initialized. After a system has booted, it will maintain a constant view of the same target ID because the driver software remaps the SCSI target ID to the new D_ID on the fly. However, a second system may use a different SCSI target ID for that target. Thus, an administrator seeking to work with the same target across multiple hosts must be prepared to encounter the situation where the same Fibre Channel target is known by different SCSI target IDs. The persistent binding feature of this device driver may be used to maintain a consistent target ID on all systems.

IP
Promiscuous Mode The lpfc (IP) driver supports promiscuous mode. However, promiscuous mode on Fibre Channel works differently than you might expect compared to promiscuous mode on Ethernet. Promiscuous mode allows system administrators and others to do things such as run tcpdump and other utilities to find out what packets are being sent from or received by a node on a network. Unlike a true Ethernet driver, the lpfc driver working in promiscuous mode does not really receive all the packets going over the wire. Fibre Channel, unlike Ethernet, is not a broadcast medium. When promiscuous mode is enabled on the host machine, only the packets sent between the host machine and another machine can be

seen. Packets sent between other machines cannot be seen. This restriction is a property of Fibre Channel itself. An artifact of the drivers implementation also means that packets may not be reported in order. Sometimes a TCP acknowledgment packet can be sent back up a promiscuous STREAM before transmitted data packets are themselves sent up that STREAM. Networking throughput decreases and latency increases for all packets sent or received through lpfc while promiscuous mode is enabled. Promiscuous mode forces a data copy for each incoming packet to make a duplicate copy for the socket operating in promiscuous mode. Another side effect of promiscuous mode is an increase in memory use.

Common Problems
This section deals with common failure modes, what they mean, and how to fix them. Adapter Attach Failure Some versions of SPARC OBP (Open Boot Prom) do not properly recognize the LightPulse PCI registers. When this happens the lpfc driver will not successfully load and attach at system boot time and a WARNING message similar to the following example will be displayed:
WARNING: /pci@9,2000/pci10df,1ae5@2 (lpfc0):

Adapter Attach Failure Unable to map SLIM memory: 0xfffffff9 You are most likely to encounter this failure on Ultra Enterprise servers that are being retrofitted with Sunfire PCI I/O adapters; other Ultra machines with built-in PCI support were manufactured after this OBP problem was identified and fixed. In any case, you should immediately contact Sun to upgrade your OBP. In the meantime, there is a workaround for this problem, by adding a property to the drivers lpfc.conf configuration file. This workaround only permits the use of a single LightPulse adapter in the system; more than one adapter may not be used in conjunction with this workaround. Begin by obtaining the PCI Bus and Device numbers for the LightPulse. The PCI Bus and Device numbers can be obtained by examining the OBP properties for your device before booting. In our example, the device is identified in the WARNING message. Use the device information to navigate the OBP device properties tree to obtain the desired information. At the ok prompt from OBP, type the following (your input is underlined, the information you seek is underlined and boldfaced, and the rest is output from OBP):
{0} ok cd /pci@9,2000/pci10df,1ae5 {0} ok .properties assigned-addresses 83 80 10 10 00 00 power-consumption 00000000 00e4e1c0 reg 00801000 00000000 03801010 00000000 03801018 00000000 01801020 00000000 name pci10df,1ae5 66mhz-capable 00 00 00 00 udf-supported 00 00 00 00 fast-back-to-back 00 00 00 01 devsel-speed 00 00 00 01 class-code 00 0c 04 00 00 00 00 00 20 00 00 00 00 00 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000ff8 000000f8 00000100

interrupts max-latency min-grant revision-id device-id vendor-id {0} ok

00000001 00 00 00 00 00 00 00 00 00 00 00 1a 00 00 10

00 08 02 e5 df

The values returned by OBP for the reg property, in the locations indicated by the boldfaced and underlined text in the above example, are the sought-after PCI Bus and Device numbers. Of the four digits reported by OBP, the first two are the PCI Bus number and the second two are the Device number. So, in our example, the PCI Bus number is 80 and the Device number is 10. (These values are in hexadecimal.) These values may very well be different on your machine. Now, add a reg property to your lpfc.conf file, of the following form (weve left an example for you, commented out, in the installed lpfc.conf):
reg = 0x00BBDD00, 0x00000000, 0x00000000, 0x00000000, 0x00000000, # PCI 0x02BBDD10, 0x00000000, 0x00000000, 0x00000000, 0x00001000, # SLIM 0x02BBDD18, 0x00000000, 0x00000000, 0x00000000, 0x00000100; # CSRs

In place of BB in the above example, use the PCI Bus number you obtained from OBP; in place of DD in the above example, use the Device number you obtained from OBP. In our example, we would add the following reg property to lpfc.conf (the BB and DD values we inserted are underlined and boldfaced):
reg = 0x00801000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, # PCI 0x02801010, 0x00000000, 0x00000000, 0x00000000, 0x00001000, # SLIM 0x02801018, 0x00000000, 0x00000000, 0x00000000, 0x00000100; # CSRs

Adding this information to the lpfc.conf file will override the erroneous reg property created by OBP and allow the lpfc driver to be loaded and attached. No additional changes should be needed. However, as we have already noted, this workaround only permits the use of a single LightPulse in the system. Because the PCI Bus and Device numbers are now hard coded in the lpfc.conf file, all LightPulse adapters will be assigned the same Bus and Device numbers, which could not possibly work correctly. To operate more than one LightPulse in the same system, you have no alternative but to obtain an OBP upgrade. For more information about the OBP PCI register sets, please refer to the IEEE 1275 PCI Bus Bindings specification, and to the Solaris pci(4) manual page.

Bugs
In this section of the release notes, we list bugs we think you might encounter in practice. Platform-Independent The following issues are thought to be independent of the host platform processor type. Disk drive failure modes Below, we describe some disk drive failure modes that we have encountered in practice. Drive light turns out Typically, the drive light does not turn on again under its own power. Maybe the drive will come back to life when a command to it times out and the driver tries to abort the command; or maybe it will come back to life at the next loop event; or maybe not. Try contacting the drive manufacturer for new firmware to fix the problem.

LIP type F8 Some disk drive firmware does not recover from a LIP type F8 in the way expected by other Fibre Channel devices; the result can be a loop full of devices that are all hopelessly confused. Usually, the best that can be done for recovering from this situation is to reboot all the hosts involved and power cycle all the disk drives involved. Unfortunately, it may be difficult to realize that an F8 LIP has happened: subsequent loop events may obscure the relevant log messages on the console or in the system log. The Emulex Digital Hub will bypass devices sending LIP F8, and keep your loop up and running. Drive Firmware Update After Labeling Isnt Reported If the drive firmware is updated after the disk is labeled by Solaris, the new firmware revision isn't reflected in the Solaris disk label. Solaris stores the firmware revision number in the disk's ASCII name in the Solaris disk label; that's what gets displayed by the format command. If you update the drives firmware, you have to do an explicit inquiry command from within format to see the change, re-labeling doesn't help. Maybe re-formatting the disk would do the trick, but that's a lengthy process. Note that on Solaris/SPARC with the sd driver, disks labeled by Solaris/x86 aren't recognized and are reported as having "corrupt" labels; in that case, Solaris will actually issue an inquiry under its own power, so ironically in this instance Solaris/SPARC will accurately report the updated firmware revision. fsck Through Block Device Fails Using fsck against the block- instead of the character-special device can result in complaints about unreadable blocks and then messages about partially allocated inodes, unknown file types and other unusual problems. Damage to the slice will result. Using the character-special device which is what you are supposed to use always works correctly. Fibre Channel Network Doesn't Operate After pkgadd and reboot Immediately after doing pkgadd and rebooting, lpfc isnt configured. An additional reboot is required to cure the problem. Sometimes, you may even have to reboot a third time. If you follow the installation directions given in these release notes, you wont have any trouble with this problem. lpfc_fc_buf_dma_alloc: lpfcN cannot map RSP dma buffer The device driver can run out of memory allocating command buffers when there are many targets on the loop across ALL loops on a single host. The workaround is to reduce the values of lun-queue-depth and lpfcNtM-lun-throttle, so that fewer commands are allocated per LUN. Also, reducing the number of devices in sd.conf helps.

Intel x86-Specific
Scatter / gather list overflow Solaris on x86 systems can pass down more scatter / gather list entries than the driver said it could handle. This is a Solaris x86 bug that has not been fixed by SUN. If you get this error, try adding more memory to your system. Window System Cant Be Run At High Resolution There is a known problem wherein the window system cant be run at resolutions exceeding 640x480 in conjunction with the LightPulse. When this problem occurs, the lpfc driver will begin complaining that it is unable to communicate with the LightPulse and/or unable to initiate or complete commands.

While this problem is under active investigation, we do not have enough information to tell you whether all video adapters are affected or only some; nor do we know whether those video adapters that are affected will always display the same symptoms in conjunction with the LightPulse. Refer to Operational Issues for additional details.

SPARC Specific
OBP Fails to Recognize LightPulse Correctly SPARC systems with older Open Boot Prom revisions may fail to recognize the LightPulse adapter. The lpfc driver will print out warning messages about being unable to configure the LightPulse. You must upgrade the OBP firmware or employ a workaround. See Operational Issues for details.

Chapter 5 - Diagnostic utility - lputil

The required files for the common utility are automatically loaded when the Emulex IP/SCSI driver files for Solaris are installed. Enter this path to start the utility:
/usr/sbin/lpfc/lputil

After the lputil goes through its start-up procedure, the Main menu displays: Main Menu This table describes each option. Values are in hexadecimal format. Menu Description Selection 1. List Adapters This option displays a list of host adapters discovered in the system. 2. Adapter This option displays host adapter characteristics: Information PCI configuration parameters, adapter revision levels, wakeup parameters, IEEE address, loop map, status and counters, link status and adapter configuration parameters. 3. Firmware This option is used to upgrade firmware or Maintenance download PCI configuration data. Acceptable file formats are: absolute firmware image (.awc), program image (.prg), and download with checksum (.dwc). Each file is hardware specific (LP7000, LP8000, etc.), but can be used on supported platforms (AIX, Solaris, Windows NT). PCI configuration data is contained in .cfl files, with can be used across any supported platform for type of supported host adapter. 4. Reset Adapter This option can be used during troubleshooting to isolate one adapter from several installed. In most cases, the upgrade procedure performs an automatic restart of the host adapter.

Update PCI Base Host Adapter Firmware 1. Obtain a copy of the upgrade file. Upgrade files are available on the Emulex ftp server at ftp://ftp.emulex.com/pub/fibrechannel. From the common utility Main menu, 2. Select 3, Firmware Maintenance. For PCI base adapter, this menu displays FIRMWARE MAINTENANCE MENU 1. Load Firmware Image 2. Display AWC File Contents 3. Display Flash Load List 4. Display RAM Load List 5. Load PCI Configuration File 6. Disable Boot BIOS 3. Select 1, Load firmware image. 4. Enter the new firmware image file name. This file downloads to the FLASH ROM of the host adapter. The board should automatically reset. 5. Upon completion, press 0 to return to Main menu. 6. Select 2, Adapter Information. 7. Select 2, Adapter Revision Levels. Verify that the new revision level is displayed.

Update PCI Base Host Adapter FCode 1. Obtain a copy of the upgrade file. Upgrade files are available on the Emulex ftp server at ftp://ftp.emulex.com/pub/fibrechannel. From the common utility Main menu, 2. Select 3, Firmware Maintenance. This menu displays FIRMWARE MAINTENANCE MENU 1. Load Firmware Image 2. Display AWC File Contents 3. Display Flash Load List 4. Display RAM Load List 5. Load PCI Configuration File 6. Disable Boot BIOS 3. Select 1, Load firmware image. 4. Enter the new FCode image file name. This file downloads to the FLASH ROM of the host adapter. The board should automatically reset. 5. Upon completion, press 0 twice to exit the program. Note: All adapters MUST have an FCode image, or none of them. You cannot add FCode to just one adapter in a multi-adapter system. 6. If your controller did not have an FCode image previously, your Device Driver Instances will be different because FCode adapters use lpfc to identify each adapter instance while non-FCode adapters use fibre-channel or the PCI device / vendor id. If the driver instances change, for each adapter, it will make existing lpfc.conf configuration settings (like persistent binding) invalid. To preserve any instance specific adapter information, execute the following script:
/usr/sbin/lpfc/convert_path_lpfc /etc/path_to_inst

7. Reboot the system for the changes to take effect. Update SBUS Base Host Adapter Firmware 1. Obtain a copy of the upgrade file. Upgrade files are available on the Emulex ftp server at ftp://ftp.emulex.com/pub/fibrechannel. From the common utility Main menu, 2. Select 3, Firmware Maintenance. This menu displays FIRMWARE MAINTENANCE MENU 1. Load Firmware Image 2. Display AWC File Contents 3. Display Flash Load List 4. Display RAM Load List 5. Load PCI Configuration File 6. Load FCode Image 7. Display Current FCode Revision 3. Select 1, Load firmware image.

4. Enter the new firmware image file name. This file downloads to the FLASH ROM of the host adapter. The board should automatically reset. 5. Upon completion, press 0 to return to Main menu. 6. Select 2, Adapter Information. 7. Select 2, Adapter Revision Levels. Verify that the new revision level is displayed.

Update Host Adapter FCode 1. Obtain a copy of the upgrade file. Upgrade files are available on the Emulex ftp server at ftp://ftp.emulex.com/pub/fibrechannel. From the common utility Main menu, 2. Select 3, Firmware Maintenance. This menu displays FIRMWARE MAINTENANCE MENU 1. Load Firmware Image 2. Display AWC File Contents 3. Display Flash Load List 4. Display RAM Load List 5. Load PCI Configuration File 6. Load FCode Image 7. Display Current FCode Revision 3. Select 7, Load FCode Image. 4. Enter the new FCode image file name. This file downloads to the FLASH ROM of the host adapter. The board should automatically reset. 5. Upon completion, select 8, Display Current FCode Revision Verify that the new FCode revision level is displayed. (This operation might take a few seconds to complete.) 6. Reboot the system for the changes to take effect

Chapter 6 - Error Messages

By default, error / warning messages are printed to the console and error logged. If the configuration parameter, log-only, is set, messages are only error logged. Some messages are only recorded if the log-verbose configuration parameter is turned on. With the verbose parameter, you can set a bit mask to record specific types of verbose messages: 0x1 ELS events 0x2 Device Discovery events 0x4 Mailbox Command events 0x8 Miscellaneous events 0x10 Link Attention events 0x20 IP events 0x40 FCP events 0x80 Node table events The following section describes each error / warning message and indicates any user actions needed. In addition, the message status (ERROR, WARNING, PANIC) and (always / verbose) and driver source code location are also given. Error messages are of the format:

LpfcN: Num : Message Data: Variables Where N is the interface number Num is a unique 3-digit error code Message is the Error Message Variables are 4 message dependent data variables

Platform Independent errors

These error codes range from 001 to 099. lpfcN: 1 : Stray ELS completion Received an ELS command completion without issuing a corresponding ELS Command (based on the iotag field in the CMD_ELS_REQUEST_CR iocb). The IOCB command and iotag used to match are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcelsb.c lpfcN: 2 : ELS [command response] completion error An ELS command completed with a status error in the IOCB. The IOCB command, IOCB status, IOCB parameter error, as well as the Port ID this ELS command was directed to are recorded. This is only a warning. It could mean The Fibre Channel device on the network is not responding. The Fibre Channel device is not an FCP target. The Fibre Channel driver will automatically retry this ELS command if needed. ACTION: If the ELS command is a PLOGI or PRLI, and the destination PortID is not an FCP Target, no action is needed. Otherwise, check physical connections to Fibre Channel network and the state the remote PortID is in. WARNING ELS verbose fcelsb.c lpfcN: 3 : Device Discovery completion error This indicates an uncorrectable error was encountered during device (re)discovery after a link up. Fibre Channel devices will not be accessible if this message is displayed. The location within the driver is recorded. ACTION: Reboot system. If problem persists, contact customer support. Run with verbose mode on for more details. ERROR always fcelsb.c lpfcN: 4 : Device Discovery completes This indicates successful completion of device (re)discovery after a link up. The location within the driver is recorded. ACTION: none needed. WARNING DISCOVERY verbose fcelsb.c lpfcN: 5 : Unknown ELS [command response iocb] An ELS command, ELS response, or ELS ring IOCB are not supported or have an illegal value. The illegal / unknown value is recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcelsb.c

lpfcN: 6 : FARP-[REQ RSP] received A FARP ELS command was received from a Fabric. The command originators PortID is recorded. ACTION: none needed. WARNING IP verbose fcelsb.c lpfcN: 7 : FLOGI completes successfully A Fabric Login completed successfully. The assigned PortID, R_A_TOV and E_D_TOV are recorded. ACTION: none needed. WARNING DISCOVERY verbose fcelsb.c lpfcN: 8 : PRLI completes successfully A PRLI to a FCP target completed successfully. target PortID and FCP SCSI ID (pan, sid) are recorded. ACTION: none needed. WARNING FCP / DISCOVERY verbose fcelsb.c lpfcN: 9 : FAN received A FAN ELS command was received from a Fabric. The assigned PortID and internal driver state are recorded. ACTION: none needed. WARNING DISCOVERY verbose fcelsb.c lpfcN: 10 : RSCN received A RSCN ELS command was received from a Fabric. The internal driver state and size of the RSCN payload is recorded. ACTION: none needed. WARNING DISCOVERY verbose fcelsb.c lpfcN: 11 : Unknown Identifier in RSCN payload Typically the identifier in the RSCN payload specifies a domain, area or a specific N_Port ID. If neither of these are specified, a warning will be recorded. The value of the illegal identifier will be recorded. ACTION: Potential problem with Fabric. Check with Fabric vendor. WARNING always fcelsb.c lpfcN: 12 : ELS Retry failed If an ELS command fails, it may be retried up to 3 times. This message will be recorded if the driver gives up retrying a specific ELS command. The value of the ELS command and the Remote PortID will be recorded. ACTION: If the ELS command is a PRLI, and the destination PortID is not an FCP Target, no action is needed. Otherwise, check physical connections to Fibre Channel network and the state the remote PortID is in. WARNING ELS verbose fcelsb.c lpfcN: 13 : Service Level Interface selected Either a CONFIG_PORT (SLI2) or PART_SLIM (SLI1) mailbox command was issued. The SLI selected, 1 or 2, and the virtual address of the SLIM will be recorded. ACTION: none needed. WARNING MISC verbose fcmboxb.c lpfcN: 14 : Memory Buffer Pool is below low water mark

A driver memory buffer pool is low on buffers. The buffer pool number and low water mark are recorded. ACTION: none needed. WARNING MISC verbose fcmemb.c lpfcN: 15 : Memory Buffer Pool is out of buffers A driver memory buffer pool is exhausted. The buffer pool number is recorded. ACTION: Configure more resources for that buffer pool. If problems persist report these errors to customer service. ERROR always fcmemb.c lpfcN: 16 : Cannot find virtual address for mapped buffer The driver cannot find the specified buffer in its mapping table. Thus is cannot find the virtual address needed to access the data. The mapped buffer address and ring number are recorded. ACTION: If problems persist report these errors to customer service. ERROR always fcmemb.c lpfcN: 17 : Ran out of Node Table entries The driver maintains a node table with one entry for every device it needs to communicate with on the Fibre Channel network. The number of table entries and the PortID for the device that failed are recorded. This message will be a warning / verbose if the driver can free up some unused node table entries. ACTION: Reduce the number of devices in your Fibre Channel network. If problems persist report these errors to customer service. ERROR always fcrpib.c lpfcN: 18 : Device Discovery Started Device discovery / rediscovery after FLOGI or FAN has started. The location within the driver is recorded. ACTION: none needed. WARNING DISCOVERY verbose fcrpib.c lpfcN: 19 : Stray FCP Completion Received an FCP command completion without issuing a corresponding FCP Command (based on the iotag field in the FCP iocb). The IOCB command, IOCB status, IOCB parameter and iotag used to match are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcscsib.c lpfcN: 20 : FCP completion error An FCP command completed with a status error in the IOCB. The IOCB status, the IOCB parameter error, as well as the PortID this FCP command was directed to are recorded ACTION: If there are many errors to one device, check physical connections to Fibre Channel network and the state the remote PortID is in. ERROR always fcscsib.c lpfcN: 21 : Rcv Ring: out of posted buffers If the ring number is 1 (IP ring) this usually means the system is out of TCP/IP buffers. The ring number and missed buffer count are recorded ACTION: Try allocating more IP buffers (STREAMS buffers or mbufs) of size 4096 and/or increasing post-ip-buf configuration parameter. Reboot the system. ERROR IP verbose fcscsib.c

lpfcN: 22 : NameServer Rsp buffer allocation failed We received a NameServer Response and we need a buffer to put it in. This usually means the system is out of TCP/IP buffers. ACTION: Try allocating more IP buffers (STREAMS buffers or mbufs) of size 4096 and reboot the system. ERROR always fcscsib.c lpfcN: 23 : Post buffer for IP ring fails The driver cannot post a buffer to the IP ring. This usually means the system is out of TCP/IP buffers. The missed buffer count is recorded ACTION: Try allocating more IP buffers (STREAMS buffers or mbufs) of size 4096 and reboot the system. ERROR always fcscsib.c lpfcN: 24 : Command ring timeout An IOCB command was posted to a ring and did not complete within a timeout based on ratov. The ring number and the IOCB command are recorded. ACTION: This error could indicate a software driver or firmware problem. If no I/O is going through the adapter, reboot the system. If these problems persist, report the errors to customer service. ERROR always fcscsib.c lpfcN: 25 : Mailbox command timeout A Mailbox command was posted to the adapter and did not complete within 30 seconds. the Mailbox command and status are recorded. ACTION: This error could indicate a software driver or firmware problem. If no I/O is going through the adapter, reboot the system. If these problems persist, report the errors to customer service. ERROR always fcscsib.c lpfcN: 26 : FAN timeout We received a link up event without the login bit set, so the driver waits E_D_TOV for the Fabric to send a FAN. If no FAN if received, a FLOGI will be sent after the timeout. ACTION: None needed. The driver recovers from this with a FLOGI to the Fabric. WARNING always fcscsib.c lpfcN: 27 : NameServer Registration timeout Our registration request to the Fabric is not acknowledged within ratov. ACTION: Check Fabric configuration. The driver recovers from this and continues with device discovery. WARNING always fcscsib.c lpfcN: 28 : Nameserver Query timeout Node Authentication timeout Node Discovery timeout A NameServer Query to the Fabric or discovery of reported remote NPorts is not acknowledged within R_A_TOV. ACTION: Check Fabric configuration. The driver recovers from this and continues with device discovery. WARNING always fcscsib.c lpfcN: 29 : pt2pt timeout

This indicates a PLOGI has not been received, within R_A_TOV, after a successful FLOGI, which indicates our topology is point to point with another N_Port. Typically this PLOGI is used to assign a N_Port ID. ACTION: None Needed, driver will recover by configuring N_Port ID as 0. WARNING always fcscsib.c lpfcN: 30 : SCSI timeout An FCP IOCB command was posted to a ring and did not complete within ULP timeout seconds. The IOCB command, N_Port ID and SCSI ID are recorded. ACTION: If no I/O is going through the adapter, reboot the system; otherwise check the state of the target in question. WARNING FCP verbose fcscsib.c lpfcN: 31 : Link [Up Down] Event received [ALPA map] Either a link up or link down link event is received. It is also possible for multiple link events to be received together. If this happens, the current event number, last event number received, ALPA granted, and number of entries in the loop init LILP ALPA map are recorded. For a Link Up, an ALPA map message is also recorded if LINK_EVENT verbose mode is set. Each ALPA map message contains 16 ALPAs. ACTION: If numerous link events are happening, check physical connections to Fibre Channel network. WARNING always / LINK_EVENT fcscsib.c lpfcN: 32 : Unknown Mailbox completion An unsupported or illegal Mailbox command completed. The unknown value is recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcscsib.c lpfcN: 33 : SCSI Link reset The SCSI layer has determined the link needs to be reset. A LIP is sent to restart loop initialization. ACTION: None Needed. WARNING always fcscsib.c lpfcN: 34 : Bring Adapter online The driver has received a request to bring the adapter online. This may happen when running lputil. ACTION: None Needed. WARNING always fcscsib.c lpfcN: 35 : Bring adapter offline The driver has received a request to bring the adapter offline. This may happen when running lputil. ACTION: None Needed. WARNING always fcscsib.c lpfcN: 36 : Abort Target I/O: clearq This is displayed if the driver internally detects that it needs to abort all I/O to a specific device. This results in a Fibre Channel ABTS to abort the I/O in process. The target N_Port ID, SCSI ID, and LUN are recorded. ACTION: Check state of target in question. WARNING FCP verbose fcscsib.c

lpfcN: 37 : Abort Target I/O: issue_abort This is displayed if the SCSI layer detects that it needs to abort all I/O to a specific device. This results in an FCP Task Management command to abort the I/O in progress. The target N_Port ID, SCSI ID, and LUN are recorded. ACTION: Check state of target in question. WARNING always fcstratb.c lpfcN: 38 : FCP command failed This indicates a target has sent us a FCP RSP error. Because of the amount of information that needs to be recorded, two back to back messages are generated. The first records the IOCB information, the IOCB cmd, Iotag, status, and parameter error field. The second records FCP information, N_Port ID, SCSI ID, LUN, and FCP response status bytes. ACTION: Check the state of the target in question. WARNING FCP verbose fcscsib.c lpfcN: 39 : Cannot issue FCP command This indicates we no longer have a valid login with a target. The target N_Port ID, SCSI ID, and LUN are recorded. ACTION: Check the state of the target in question. WARNING always fcstratb.c lpfcN: 40 : No room on IP xmit queue This message indicates the system is generating IOCB commands to be processed, faster than the adapter can process them. ACTION: Check the state of the link. If the link is up and running, reconfigure the xmit queue size to be larger. Note, a larger queue size may require more system IP buffers. If the link is down, check physical connections to Fibre Channel network. WARNING IP verbose fcxmitb.c lpfcN: 41 : Stray XmitSequence completion Received an XMIT_SEQUENCE IOCB completion without issuing a corresponding XMIT_SEQUENCE Command (based on the iotag field in the XMIT_SEQUENCE_CR iocb). The IOCB command and iotag used to match are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcxmitb.c lpfcN: 42 : Xmit sequence completion error A XMIT_SEQUENCE command completed with a status error in the IOCB. The IOCB status, the IOCB iotag, the IOCB parameter error, as well as the PortID this command was directed to are recorded. The driver attempts to recover by creating a new exchange to the remote device. ACTION: If there are many errors to one device, check physical connections to Fibre Channel network and the state the remote PortID is in. WARNING IP verbose fcxmitb.c lpfcN: 43 : Stray CreateXRI completion Received a CREATE_XRI command completion without issuing a corresponding CREATE_XRI Command (based on the iotag field in the CREATE_XRI_CR iocb). The IOCB command and iotag used to match are recorded.

ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcxmitb.c lpfcN: 44 : Adapter failed to initialize This indicates the host adapter failed to initialize properly. The adapter status register is recorded. ACTION: This error usually indicates a hardware problem with the adapter, run diagnostics. ERROR always lp6000.c lpfcN: 45 : Adapter initialization failed, Mailbox command This indicates the host adapter failed to initialize properly because a mailbox command failed. The mailbox command and mailbox status values are recorded. ACTION: This error usually indicates a hardware problem with the adapter, run diagnostics. ERROR always lp6000.c lpfcN: 46 : RUN_BIU_DIAG failed This indicates the host adapter failed to initialize properly because a PCI bus DMA test failed. ACTION: This error usually indicates a hardware problem with the adapter, run diagnostics. ERROR always lp6000.c lpfcN: 47 : WorldWide PortName doesn't conform to IP Profile In order to run IP, the WorldWide PortName must be of type IEEE (NAA = 1). This message is display if the adapter WWPN doesnt conform to the standard. The NAA is recorded. ACTION: Turn off the network-on configuration parameter or has a different WWPN assigned. ERROR always lp6000.c lpfcN: 48 : Enable interrupt handler failed This error occurs if the driver cannot assign an interrupt vector to the host adapter. ACTION: Check configuration of adapter in system. ERROR always lp6000.c lpfcN: 49 : Adapter Hardware Error This error occurs if we get a FireFly error interrupt and the status register indicates a nonrecoverable error. . The adapter status register is recorded. ACTION: This error usually indicates a hardware problem with the adapter, run diagnostics. ERROR always lp6000.c lpfcN: 50 : Stray Mailbox Interrupt Received a Mailbox command completion and the mailbox is not owner by the host. The Mailbox command and status are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always lp6000.c lpfcN: 51 : [29,xx] Check condition received

This error could indicate a problem with a FCP target. The check condition response info fields are recorded. A 29,xx check condition is only recorded in verbose mode. ACTION: Check state of FCP target ERROR always lp6000.c lpfcN: 52 : [Xmit Received] ELS [command response] This error could indicate a problem with a FCP target.. The check condition response info fields are recorded. A 29,xx check condition is only recorded in verbose mode. ACTION: No action needed. WARNING ELS verbose fcelsb.c

SOLARIS specific errors

General SOLARIS error code range from 100 to 109. SOLARIS LAN specific error codes range from 110 to 119. SOLARIS FCP specific error codes range from 120 to 139. lpfcN: 100 : Attach failure: XXXXX The driver attach routine failed for reason XXXXX. The assigned dip and attach routine cmd are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLARIS.c lpfcN: 101 : Detach failure: XXXXX The driver detach routine failed for reason XXXXX. The assigned dip and attach routine cmd are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLARIS.c lpfcN: 102 : Config error: XXXXX Parameter XXXXX in the driver conf file has a bad value. The bad value and new value are recorded. ACTION: Check the driver .conf file, usually located in /kernel/drv/lpfc.conf ERROR always fcSOLARIS.c lpfcN: 103 : fc_malloc: XXXXX The driver memory allocation routine failed for reason XXXXX. Various memory allocation parameters are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLARIS.c lpfcN: 104 : fc_bufmap: XXXXX The driver memory mapping routine failed for reason XXXXX. Various memory mapping parameters are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLARIS.c lpfcN: 110 Dropped inbound packet: no STREAM The message indicates a packet was destined for a LLC/SNAP type that was not configured. The LLC/SNAP type field is recorded.

ACTION: This error could indicate a STREAMs configuration problem. ERROR IP verbose fcSOLlan.c lpfcN: 120 LUN address out of range The message indicates the SCSI LUN address out of range. The SCSI ID, LUN, and maximum LUN allowed are recorded. ACTION: Check the sd.conf or cmdk.conf file, usually located in /kernel/drv directory. ERROR always fcSOLfcp.c

lpfcN: 121 Override default targetX SCSI options The message indicates the default SCSI options for target X has been overridden. The SCSI ID, new SCSI option, and default SCSI option is recorded. ACTION: No action needed. WARNING FCP verbose fcSOLfcp.c lpfcN: 122 Override default SCSI tag age limit The message indicates the default SCSI tag age limit has been overridden. The new tag age and default tag age is recorded. ACTION: No action needed. WARNING FCP verbose fcSOLfcp.c lpfcN: 123 Override default SCSI reset delay The message indicates the default SCSI reset delay has been overridden. The new reset delay and default reset delay are recorded. ACTION: No action needed. WARNING FCP verbose fcSOLfcp.c lpfcN: 124 Exceeded max s/g size The scatter/gather elements received by the driver exceeded the number (64) it can handle. The SCSI ID, LUN, the number of elements we support, and number of elements we got from upper layer are recorded. ACTION: This error could indicate a Solaris problem. To workaround a user can issue a smaller I/Os, or install more memory. If problems persist report these errors to customer service. ERROR always fcSOLfcp.c lpfcN: 125 DMA allocation error The message indicates an error for allocating a DMA memory. The malloc status error is recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLfcp.c lpfcN: 128 Invalid FCP Response Code received The message indicates an invalid FCP Response Code has received. The SCSI ID, LUN, and FCP task mgmt response status (rspinfo3) is recorded. ACTION: This error could indicate a software driver or target problem. If problems persist report these errors to customer service. ERROR always fcSOLfcp.c lpfcN: 129 FCP Read Check Error

The message indicates the adapter detects corrupted and missing frames, which only happens when an entire sequence is dropped. The IOCB cmd, FCP response SCSI status (rspinfo2) and FCP task mgmt response status (rspinfo3) is recorded. ACTION: Check link connection to target. WARNING FCP verbose fcSOLfcp.c lpfcN: 130 SCSI Auto Sense is not enabled The target driver can either issue a SCSI Request Sense command when error occurred or enable the AutoSense capability through driver interface routine. The SCSI ID and LUN are recorded. ACTION: No action needed.. WARNING FCP verbose fcSOLfcp.c lpfcN: 131 Cant copy ARQ data, size mismatch The message indicates the SCSI ARQ status buffer greater than SCB length. The SCSI ID, LUN, SCB length, and length of our buffer are recorded. ACTION: This error could indicate a software driver or firmware problem. If problems persist report these errors to customer service. ERROR always fcSOLfcp.c

Appendix A - Glossary
ALPA Arbitrated Loop Physical Address; the address of a node on Fibre Channel when running in an Arbitrated Loop topology DDI Device Driver Interface DKI Driver Kernel Interface DLPI Data Link Provider Interface DMA Direct Memory Access; the ability of some devices to send data directly to/from memory without intervention by the host processor. EDO Extended Data Out HBA Host Bus Adapter HBA driver A driver responsible for managing a host bus adapter; transports commands and responses between a target driver and a SCSI or FCP target IP Internet protocol ISA Instruction Set Architecture JBOD Just a Bunch Of Disks. A cabinet, power supply, and disks. No controller or fancy logic. LILP Loop Initialization Loop Position LIRP Loop Initialization Report Position lpfc LightPulse Fibre Channel host bus adapter driver as a whole; which consists of an IP and a FCP portion. LUN Logical Unit Number OBP Open Boot PROM PCI Peripheral Component Interconnect SCSI Small Computer System Interface Solaris A flavor of Unix manufactured by SunSoft target driver A driver responsible for generating commands for and interpreting responses from a target; these commands are sent via the HBAdriver for a device on the SCSI bus, for instance a disk, tape, or CD-ROM WWNN WorldWide Node Name, used to uniquely identify a Fibre Channel device. WWPN WorldWide Port Name, used to uniquely identify a Fibre Channel device.