Module 6:

Configuring IP Network Multipathing

Limitations of network interfaces:

Network interfaces are exposed to failure because
they connect to network cables and hardware components in the form of switches or
hubs. Failure of any of these interfaces results in network failure, even if the
NIC that is in place does not fail.

IPMP:
By using IPMP, you can configure one or more physical interfaces into
an IP multipathing group, or IPMP group. After configuring IPMP, the system
automatically monitors the interfaces in the IPMP group for failure. If an
interface in the group fails or is removed for maintenance, IPMP automatically
migrates, or fails over, the failed interface's IP addresses. The recipient of
these addresses is a functioning interface in the failed interface's IPMP group.
The failover feature of IPMP preserves connectivity and prevents disruption of any
existing 741 connections. Additionally, IPMP improves overall network performance
by automatically spreading out network traffic across the set of interfaces in the
IPMP group. This process is called load spreading.

Solaris IPMP Components:

Solaris IPMP involves the following software:
■ The in.mpathd daemon
■ The /etc/default/mpathd coniguration file
■ ifconfig options for IPMP configuration

Multipathing Daemon, in.mpathd:

The in.mpathd daemon detects interface failures, and then implements various
procedures for failover and failback. After in.mpathd detects a failure or a
repair, the daemon sends an ioctl to perform the failover or failback. The ip
kernel module, which implements the ioctl, does the network access failover
transparently and automatically.
The in.mpathd daemon detects failures and repairs by
sending out probes on all the interfaces that are part of an IPMP group. The
in.mpathd daemon also detects failures and repairs by monitoring the RUNNING lag
on each interface in the group.

Probe-based IPMP Configuration:

Probe-based failure detection for IPMP uses test addresses to detect
failures, and notify the networking subsystem.
Probe-based IPMP Requirements:
l The Solaris 8 10/00 OS, as a minimum, must be installed.
l Unique MAC addresses must be configured on each network interface.
l Multiple network adapter interfaces must be connected on each subnet.
l An IPMP group name must be assigned to interfaces.
l A test address is assigned to an interface.
l Additional hosts or devices must exist on the same subnet.

Interface Failure Detection and Repair:

The in.mpathd daemon can detect both the failure and the repair of an
interface by:
l Sending ICMP echo requests and receiving ICMP echo replies through the
interface
l Monitoring the internal IFF_RUNNING flag on the interface
To detect the failure or repair of interfaces that belong to the IPMP group,the
in.mpathd daemon sends ICMP echo requests from the test addresses the IPMP
interfaces to targets connected to the local network. The in.mpathd daemon
determines which targets to probe dynamically. If five consecutive probes do not
receive replies, the interface is considered failed. Adjust the failure detection
time by editing the FAILURE_DETECTION_TIME variable from the default value of
10,000 milliseconds (10 seconds) in the /etc/default/mpathd file. When responses to
the ICMP echo requests are not received and a specific time period has elapsed, the
physical interface is considered failed. The IP address that is associated with
the failed address is moved to a new logical interface associated with another
physical interface in the same IPMP group.

Configuring Probe-based IPMP by Using Configuration Files :

To configure probe-based IPMP chek following steps first:
1. Verify the Solaris OS release. (cat /etc/release)
2. Configure unique MAC addresses. ( using eeprom or ifconfig command)
3. Define IP addresses. ( cat /etc/inet/hosts)
4. Configure the interfaces.(cat /etc/hostname.sige0 and cat /etc/hostname.qfe1)
5. Reboot the system. ( init 6)
6. View the interface configuration. ( using ifconfig –a command)

Configuring Probe-based IPMP on the Command Line:

To configure IPMP, complete the following steps, which are described in greater
detail in the next sections.
1. Verify the Solaris OS release. (cat /etc/release)
2. Configure unique MAC addresses. (eeprom and ifconfig)
3. Configure IP addresses. (cat /etc/hosts)
4. Configure the hme0 interface as part of an IPMP group.
# ifconfig hme0 group mpgrp-one && ifconfig -a
5. Configure a test address for the hme0 interface.
# ifconfig hme0 addif 192.168.1.51 deprecated netmask + \ broadcast + -failover up
# ifconfig –a ( to check the changes)
6. Configure the qfe1 interface as part of the same IPMP group.
# ifconfig qfe1 plumb sys11-data-qfe1 netmask + broadcast +
# ifconfig qfe1 group mpgrp-one up
#ifconfig -a
7. Configure a test address for the qfe1 interface.
# ifconfig qfe1 addif 192.168.1.71 deprecated netmask + \ broadcast + -failover up
#ifconfig -a
Start the in.mpathdDaemon to Monitor the Interfaces
# cat /etc/default/mpathd
If the TRACK_INTERFACES_ONLY_WITH_GROUPS variable is set to yes, the ifconfig
command’s group option starts the in.mpathd daemon automatically.
*The in.mpathd daemon is started by the svc:network/net-init SMF service:
# grep in[.]mpathd /lib/svc/method/net-init
To start in.mpathd daemon run following command through root:
# /sbin/in.mpathd

8. View the interface configuration.(ifconfig –a command)

Link-based IPMP Configuration :

Link-based failure detection for IPMP uses the network
interface kernel driver to detect failures and notify the networking subsystem.
Link based IPMP requirements:
1) Solaris 9 12/02 OS, at a minimum, must be installed.
2) l Network interfaces must use any of the following drivers:
hme,eri,ce,bfe etc
3) Unique MAC addresses must be configured on each of the interfaces.
4) An IPMP group name must be assigned to interfaces.
Configuring Link-based IPMP by Using Configuration Files :
1) Ifconfig –a
2) Cat /etc/release
3) eeprom or ifconfig command for unique MAC Address.\
4) Cat /etc/inet/hosts

5) # vi /etc/hostname.hme0
# vi /etc/hostname.hme1
6) Init 6
View the Link-based IPMP Configuration:
# ifconfig –a
# pgrep –f1 mpathd ( to verify that mpathd daemon is running)
Messages to the console (and to /var/adm/messages)from in.mpathd indicate that the
system is configured for link-based IPMP, rather than for probe-based IPMP.

Verify Link-based IPMP Operation :

To verify the system’s IPMP configuration, the if_mpadm command can be used.
Messages are sent to the console and to /var/adm/messages that indicate any
failovers or failbacks which occur.

Take the hme0 interface offline to force a failover:

# if_mpadm -d hme0
Then see failover message in /var/adm/messages or use dmesg command to see this.
#ifconfig –a
Reattach the hme0 interface, to force a failback:
# if_mpadm -r hme0
# ifconfig –a

Configure a Single IPMP Group on the Command Line:

To create a singleton IPMP group, assign a
multipath group name to the interface:
# ifconfig hme0 group singleton ( here singleton is the group name)
# ifconfig –a (to check)

Configure a Single IPMP Group at System Boot:

To create a singleton IPMP group at
system boot, ensure that the interface configuration file contains the group option
and the IPMP group name:
# cat /etc/hostname.hme0
sys11 group singleton up
Viewing IPMP Operation:
To verify the system’s failover configuration, or to change the operational status
of IPMP interfaces, use the if_mpadm command.
# if_mpadm -d hme0
Aug 4 14:00:38 sys11 in.mpathd[535]: Successfully failed over from NIC hme0 to
NIC qfe1
The message indicates that the failover was successful.
#ifconfig –a
To reattach an offline interface, type the command:
# if_mpadm -r hme0
Aug 4 14:02:09 sys11 in.mpathd[535]: Successfully failed back to NIC hme0
The message indicates that the fail back was successful.
# ifconfig –a
Troubleshooting an IPMP Configuration:
Carefully read messages in the /var/adm/messages file
or in the console window to take the proper troubleshooting steps when you
configure andtest the IPMP.
# ifconfig –a ( to check the things are alright)
To configure a test interface, use the ifconfig command:
# ifconfig hme0 addif 192.168.1.51 deprecated netmask + \
> broadcast + -failover up
Created new logical interface hme0:1
Setting netmask of hme0:1 to 255.255.255.0
#
The in.mpathd daemon reports that it can now perform failure detection. Be aware
that more than one interface is required to provide effective failover. To view
the interface configuration, use the ifconfig command:
# ifconfig –a
Both the physical and logical interfaces are configured properly.