Cisco MDS Inter-VSAN Routing

Abstract: This page will give the details about inter-vsan routing in Cisco MDS switches and its
specifics with respect to ViPR controller.

What is Inter-VSAN Routing?

Traditionally, VSANs are an abstraction to group together, logically, storage ports and host initiators to allow communication to happen between
those ports and initiators. VSANs can span multiple switches when inter-connected and multiple VSANs can be configured on the same switch to
logically separate out these entities based on business needs and other factors. In the traditional sense, no inter-VSAN communication was
possible. But with the introduction of inter-vsan routing (aka IVR), storage ports and initiators can talk to each other (in other words, routing is
made possible between these entities that are in different VSANs).

Fore more information on IVR please refer : http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/nx-os/configuration/guides/ivr

/fm_ivr/fm_ivr/ivrbasic.html

Prerequisites:
For IVR routing to happen, the switches need to support IVR and have the ivr feature enabled on them. Please refer to Cisco documentation for
switch/Fabric OS version that support IVR.

Transit VSANs need to be configured between IVR switches. For more information on transit VSANs and its best practices, please refer to the
above IVR documentation from Cisco.

Topologies:
We will consider 3 different topologies here.

1. No IVR switches in the configuration.

2. Only IVR switches in the configuration
3. Mix of non-IVR and IVR switches in the configuration. The term core-edge configuration is sometimes referred to in this context with the
IVR switches forming the core and non-IVR switches on the edges.

ViPR controller support:

Following topologies have been implemented and tested in ViPR and is supported as of skywalker release.
Ingestion of IVR zones is not yet currently supported and is planned for the upcoming release (as of now that is Leia release). COP-3105
3 has been filed to track that effort.

Scenario 1: No IVR switches in the configuration

In this case, there are multiple switches connected to each other with Inter-switch links (aka ISLs), but none of the switches are capable of doing
inter VSAN routing.

{code}
Because the switches are not IVR capable, no routing is possible between say, VSAN3 on switch 1 to VSAN12 on switch 2.

Also, no routing is possible between VSAN2 and VSAN3 on switch1.

Scenario 2: IVR only configuration

All the participating switches in this configuration are IVR capable. In this configuration, technically, any VSAN can route to any other VSAN
provided all the switches have been correctly configured and there are valid transit-VSANs between the switches.
Things to note:

1. Both Switch1 and Switch2 are IVR capable Cisco switches

2. Switch1 and Switch2 are interconnected and VSAN 1000 is the transit VSAN between the two switches that carries the traffic from one
VSAN in one switch to another VSAN in another switch in an IVR configuration.
3. In switch 1, routing is possible between VSAN2 and VSAN3 and it doesnt use the transit VSAN since all the participating switches are
local to the switch.
4. Routing is also possible between VSAN2 on switch1 and VSAN12 on switch2. Internally, VSAN1000 is used to carry the traffic.
5. show ivr-vsan topology : This CLI command lists all the switches in the IVR configuration and all the participating VSANs on those
switches.

Scenario 3: Mixed configuration

IVR and non-IVR capable switches connected to each other. an example of such a configuration is shown below.
Things to note:

1. Switch1 and Switch2 are IVR capable switches. Switch3 and Switch4 are not IVR capable.
2. VSAN1000 is transit vsan between the IVR capable switches.
3. VSANs on IVR switches are routable to each other.
4. Storage ports/initiators from switch 3 and switch 4 and only on VSAN1000 can route to other VSANs on the IVR switches.
5. VSAN such as VSAN100/VSAN101 on switch3 are not routable to any other VSAN because they reside entirely on a non-IVR switch.
same applies for VSAN200 and VSAN201 on Switch4. show ivr-vsan topology command does not list any non-IVR switches.

show ivr-vsan topology command is only valid when executed against an IVR capable switch. ViPR relies on this command to fetch all the VSANs
information on IVR capable switch and uses this information to determine which VSANs are routable to other VSANs.

Testing with ViPR controller:

Scenario 1: No IVR switches in the configuration

1. Add networks to the virtual array (Only the selected networks will be added)
2. Create Block Volume for a Host on Non IVR env - same network
3. Create Block Volume for a Host on Non IVR env - different network
4.
 3.
4. Create Block Volume for Host connected to multiple networks (considering path)

Scenario 2: IVR only configuration

1. Add networks to the virtual array (The routed networks will be added)
2. Create Block Volume for a Host on IVR env - same network
3. Create Block Volume for a Host on IVR env - different network single switch
4. Create Block Volume for a Host on IVR env - different network across switch
5. Create Block Volume for a Host for IVR env - Host connected to 2 networks and array in other network
6. Create Block Volume for a Host for IVR env - Host connected to 2 networks and array only in 1 of them

Scenario 3: Mixed configuration

1. Add networks to the virtual array (If the networks in non IVR switches are selected, only the selected networks will be added. If the
networks in IVR switches are selected, the routed networks will be added)
2. Create Block Volume for a Host on Mixed IVR env - same network
3. Create Block Volume for a Host/Cluster and mount on Mixed IVR env - different network in routednetwork
4. Create Block Volume for a Host on Mixed IVR env - different network not in routednetwork
5. Create Block Volume for a Host for Mixed IVR env - Host connected to 2 networks