Cookbook

V
Virtual Connect FlexFab

F
bric Cookbo
ook
PPart number c02

2617100
FFirst edition Novvember 2010
Legal notices
Never leverage legal and notice information. For the most current requirements, refer to HP Standard 004-0 Legal Notices
for Product Documentation, Promotional Materials, and Software and Media located at
http://standards.inet.cpqcorp.net/smc/hpstd/hpstds.htm
Contents
Purpose ................................................................................................................................................. 5
Introduction to Virtual Connect FlexFabric ................................................................................................. 6
Tunneled VLAN and Mapped VLANS ..................................................................................................... 10
Single Domain/Enclosure Scenarios ....................................................................................................... 13
Overview .............................................................................................................................................. 13
Requirements .................................................................................................................................... 13
Scenario 1 Simple vNet with Active/Standby Uplinks and FCoE Windows 2008 R2 .............................. 14
Overview .............................................................................................................................................. 14
Requirements .................................................................................................................................... 14
Installation and configuration................................................................................................................... 16
Switch configuration .......................................................................................................................... 16
Review ................................................................................................................................................. 23
Results Windows 2008 R2 Networking Examples ............................................................................... 24
Results Windows 2008 R2 SAN Connectivity..................................................................................... 28
Summary .............................................................................................................................................. 29
Scenario 2 Share Uplink Sets with Active/Active uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE
.......................................................................................................................................................... 30
Overview .............................................................................................................................................. 30
Requirements .................................................................................................................................... 30
Review ................................................................................................................................................. 42
Summary .............................................................................................................................................. 48
Scenario 3 Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE
Boot from SAN .................................................................................................................................... 49
Overview .............................................................................................................................................. 49
Requirements .................................................................................................................................... 49
Review ................................................................................................................................................. 62
Summary .............................................................................................................................................. 68
Scenario 4 Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows Hyper-V - Ethernet
and FCoE SAN .................................................................................................................................... 69
Overview .............................................................................................................................................. 69
Requirements .................................................................................................................................... 69
Review ................................................................................................................................................. 84
Summary .............................................................................................................................................. 96
Purpose 3
Scenario 5 Share Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - vSphere - Ethernet and FCoE
SAN ................................................................................................................................................... 97
Overview .............................................................................................................................................. 97
Requirements .................................................................................................................................... 97
Review ............................................................................................................................................... 112
Results vSphere Networking Examples............................................................................................. 112
Results vSphere SAN Connectivity .................................................................................................. 115
Summary ............................................................................................................................................ 116
Scenario 6 Tunneled VLANs and Share Uplink Set with Active/Active Uplinks - vSphere Ethernet and FCoE
SAN ................................................................................................................................................. 117
Overview ............................................................................................................................................ 117
Requirements .................................................................................................................................. 117
Installation and configuration................................................................................................................. 119
Switch configuration ........................................................................................................................ 119
Review ............................................................................................................................................... 130
Results vSphere Networking Examples............................................................................................. 131
Results vSphere SAN Connectivity .................................................................................................. 134
Summary ............................................................................................................................................ 135
Appendix A Scenario-based CISCO Command Line Reference ............................................................. 136

Appendix B Scenario-based ProCurve Command Line Reference ........................................................... 139
Appendix C: Acronyms and abbreviations ............................................................................................ 142
Appendix D: Useful VC CLI Command sets ............................................................................................ 144
Purpose 4
Purpose
The purpose of this Virtual Connect FlexFabric Cookbook is to provide users of Virtual Connect with a
better understanding of the concepts and steps required when integrating HP BladeSystem and Virtual
Connect FlexFabric components into an existing network.
The scenarios in this Cookbook vary from simplistic to more complex while covering a range of
typical building blocks to use when designing Virtual Connect FlexFabric solutions. Although these
scenarios are shown individually, some scenarios could be combined to create a more complex and
versatile Virtual Connect environment, however, keeping in mind the difference between mapped and
tunneled VLANs, discussed later in this paper are mutually exclusive.
The scenarios as written are meant to be self-contained configurations and do not build on earlier
scenarios, with this you may find some repetition or duplication of configuration across scenarios.
This is not meant to be a complete or detailed guide to Virtual Connect FlexFabric, but is intended to
provide the reader with some valid examples of how Virtual Connect FlexFabric could be deployed
within their environments. Many additional configurations or scenarios could also be implemented.
Please refer to the following section for additional reference material on Virtual Connect, Flex-10 and
FlexFabric.
Purpose 5
Introduction to Virtual Connect FlexFabric

Virtual Connect is an industry standard-based implementation of server-edge virtualization. It puts an
abstraction layer between the servers and the external networks so the LAN and SAN see a pool of
servers rather than individual servers (Figure 1). Once the LAN and SAN connections are physically
made to the pool of servers, the server administrator uses Virtual Connect management tools (Virtual
Connect Enterprise Manager or Virtual Connect Manager) to create an Interconnect modules
connection profile for each server.
Virtual Connect FlexFabric is an extension to Virtual Connect Flex-10 which leverages the new Fibre
Channel over Ethernet (FCoE) protocols. By leveraging FCoE for connectivity to existing Fibre
Channel SAN networks, we can reduce the number of switch modules and HBAs required within the
server blade. This in turn further reduces cost, complexity, power and administrative overhead.
It is assumed that the reader has experience with and is familiar with Virtual Connect Flex-10. For
additional information on Virtual Connect and Flex-10, please review the documents below. The
Chapter on Flex-10 in the Virtual Connect Ethernet Cookbook, listed below, will be an excellent
primer for this paper.
Additional Virtual Connect Reference Material

Link to HP Virtual Connect technology site, provides a great deal of reference information on Virtual
connect, Flex-10 and FlexFabric.
http://h18000.www1.hp.com/products/blades/virtualconnect/
Link to HP Virtual Connect technology for the HP BladeSystem c-Class, 4th edition when available
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00814156/c00814156.pdf
Link to HP Virtual Connect for c-Class BladeSystem Setup and Installation Guide
http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c01732252/c01732252.pdf
Link to HP Flex-10 technology
Link to HP BladeSystem Network Reference Architecture - FlexFabric and VMware vSphere 4
http://h20195.www2.hp.com/v2/GetPDF.aspx/c02505638.pdf
Link to Virtual Connect User, Setup and CLI Guides
http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportM
anual&lang=en&cc=us&docIndexId=64180&taskId=101&prodTypeId=3709945&prodSeriesId=379
4423
HP Virtual Connect FlexFabric Solutions Recipe
Introduction to Virtual Connect FlexFabric 6
Virtual Connect Ethernet Cookbook

Virtual Connect can be used to support both Ethernet and Fibre Channel connections. The Virtual
Connect Ethernet Cookbook is provided with basic Virtual Connect configurations for both 1Gb and
10Gb solutions, whereas the Virtual Connect FlexFabric Cookbook provides additional scenarios that
leverage the new FCoE protocols. If you are new to Virtual Connect it is highly recommended that
you also review the existing Virtual Connect Ethernet Cookbook
Virtual Connect Ethernet (with Flex-10) Channel Cookbook
(www.hp.com/go/blades)
Virtual Connect Fibre Channel Cookbook

Virtual Connect can be used to support both Ethernet and Fibre Channel connections; however, this
guide is focused completely on the Ethernet configuration.
For Fibre Channel connectivity, please refer to the Virtual Connect Fibre Channel Cookbook
(www.hp.com/go/blades)
Virtual Connect 3.15 Firmware Release

Virtual Connect firmware version 3.15 is the initial release to provide Fibre Channel over Ethernet
(FCoE) support within Virtual connect. FCoE support is provided within the FlexFabric Modules
discussed in the document.
Please refer to the Virtual Connect 3.15 Release note for further details on new features and resolved
issues.
NOTE: When VC 8Gb 20-port FC module or VC FlexFabric 10Gb/24-port module Fibre Channel
uplink ports are configured to operate at 8Gb speed and connecting to HP B-series (Brocade) Fibre
Channel SAN switches, the minimum supported version of the Brocade Fabric OS (FOS) is v6.3.1
and v6.4.x. In addition, a fill word on those switch ports must be configured with option Mode 3 to
prevent connectivity issues at 8Gb speed.
On HP B-series (Brocade) FC switches use command portCfgFillWord (portCfgFillWord <Port#>
<Mode>) to configure this setting:
Mode
Link Init/Fill Word
Mode 0
IDLE/IDLE
Mode 1
ARBF/ARBF
Mode 2
IDLE/ARBF
Mode 3
If ARBF/ARBF fails use IDLE/ARBF
Although this setting only affects devices logged in at 8G, changing the mode is disruptive regardless
of the speed the port is operating at. The setting is retained and applied any time an 8G device logs
in. Upgrades to FOS v6.3.1 or v6.4 from prior releases supporting only modes 0 and 1 will not
change the existing setting, but a switch or port reset to factory defaults with FOS v6.3.1 or v6.4 will
be configured to Mode 0 by default. The default setting on new units may vary by vendor. Please use
portcfgshow CLI to view the current portcfgfillword status for that port.
Modes 2 and 3 are compliant with FC-FS-3 specifications (standards specify the IDLE/ARBF behavior
of Mode 2 which is used by Mode 3 if ARBF/ARBF fails after 3 attempts). For most environments,
Brocade recommends using Mode 3, as it provides more flexibility and compatibility with a wide
range of devices. In the event that the default setting or Mode 3 does not work with a particular
device, contact your switch vendor for further assistance. When connecting to Brocade SAN Switches
at 8Gb, portCfgFillWord must be set to Mode 3 If ARBF/ARBF fails use IDLE/ARBF. In order to
use Mode 3, FOS v6.3.1 or v6.4.x or better is required. Virtual Connect FlexFabric Hardware
Overview
Virtual Connect FlexFabric is the first Data Center Bridging (DCB) and Fibre Channel over Ethernet
(FCoE) solution introduced into the HP BladeSystem portfolio. It provides 24 line rate ports, FullDuplex 240Gbps bridging, single DCB-hop fabric. As shown in Image 1, there are 8 faceplate
ports. Ports X1-X4 are SFP+ transceiver slots only; which can accept a 10Gb or 4Gb/8Gb SFP+
transceiver. Ports X5-X8 are SFP and SFP+ capable which can accept 1Gb or 10Gb Ethernet
connections, and do not support Fibre Channel transceivers.
Virtual connect FlexFabric Uplink Port Mappings

In it is important to note how the external uplink ports on the FlexFabric module are configure. The
graphic below outlines the type and speed each port can be configured as.
Ports X1 X4; Can be configured as 10Gb Ethernet or Fibre Channel, FC speeds supported =
2Gb, 4Gb or 8Gb using 4Gb or 8Gb FC SFP modules, please refer to the FlexFabric Quick
Spec for a list of supported SFP modules
Ports X5 X8: Can be configured as 1Gb or 10Gb Ethernet

Ports X7 X8; Are also shared as internal cross connect
Uplink ports X1-X4 support 0.55m length DAC as stacking or uplink
Uplink Ports X5-X8 support 0.57m length DAC as stacking or uplink
Note; 5m DAC cables are supported on all ports with FlexFabric, in addition, 7m DAC cables are
also supported on ports X5 through X8.
Note: The CX-4 port provided by the Virtual Connect Flex-10 and legacy modules has been
depreciated.
Note: Even though the Virtual Connect FlexFabric module supports Stacking, stacking only applies to
Ethernet traffic. FC uplinks cannot be consolidated, as it is not possible to stack the FC ports, nor
provide a multi-hop DCB bridging fabric today.
Figure 1 - FlexFabric Module

e port configuration, speedss and types
Note: Even thou

ugh the Virtua
al Connect Fle
exFabric moduules support S
Stacking, stacking only app
plies
to Ethernet
E
traffic. FC uplinks cannot be consolidated, ass it is not posssible to stack the FC ports.
Figure 2 - FlexN
NIC Connectio
ons It is imp
portant to notee that Physicall Function two
o (pf2) can be
e
con
nfigured as Eth
hernet, FCoE or
o iSCSI. Phy
ysical Functionns 1, 3 and 4 are assigned
d as Ethernet o
only
Introductiion to Virtual C
Connect FlexFab
bric 9
Tunneled VLAN and Mapped VLANS

Virtual Connect provides two Ethernet networks connection methods. Both of these connection types are
discussed within the following scenarios.
Tunnel vs. Map VLAN tags setting

It is important to note that the behavior of both vNets and Shared Uplink Sets is dependent on whether
VLAN Tunnel or Map VLAN Tags is set. Server VLAN Tagging Support, as configured in the
Advanced Ethernet Settings tab of Virtual Connect is a Domain wide configuration.
If Virtual Connect is set to Tunnel Mode, you can do the following;
Create a Shared Uplink Set which can support several VLANs up to the publish limits.
Create a vNet which can support both TAGGED and UNTAGGED frames, if tagged the host
system will need to interpret those tags
These VLANs can be presented to a Server NIC, one at a time No multiple VLANS supported,
frames are presented to the NIC untagged
If Virtual Connect is set to Map VLAN Tags Mode, you can do the following;
Create a Shared Uplink Set (the behavior of a SUS changes and now provides the ability to
connect multiple networks to a NIC) which can support several VLANs up to the publish limits.
These VLANs can be presented to a Server NIC, as either a single Network (where VC will
remove the tags and present an untagged frame to the NIC), or as multiple Networks, where
VC will present all frames with their VLAN tags, in which case the host system will need to
interpret the tags (one network could be configured as untagged)
Create a vNet (the behavior of a vNet also changes) a vNet can now only support
UNTAGGED frames, which means a vNet could then only support ONE VLAN/network
vNet
There are two modes of operation for the vNet. The first is a simple vNet that will pass untagged
frames. The second is a vNet tunnel which will pass tagged frames for one or many VLANs.
The vNet is a simple network connection between one or many server NICs to one or many uplink ports
A vNet could be used to connect a single VLAN, no tagging, to one or many server NICs, if this
network is part of a VLAN, by configuring the upstream switch port as an access or untagged port, by
extension, any server connected to this vNet would reside in that VLAN, but would not need to be
configured to interpret the VLAN tags.
A tunneled vNet will pass VLAN tagged frames, without the need to interpret or forward those frames
based on the VLAN tag. Within a tunneled vNet the VLAN tag is completely ignored by Virtual
Connect and the frame is forwarded to the appropriate connection (server NIC[s] or uplinks) depending
on frame direction flow. In this case, the end server would need to be configured to interpret the VLAN
tags. This could be a server with a local operating system, in which the network stack would need to
be configured to understand which VLAN the server was in, or a virtualization host with a vSwitch
supporting multiple VLANs.
The tunneled vNet has no limit to the number of VLANs it can support.
10
Benefits of a vNet
If no VLAN support is required, support for a single specific VLAN being presented as untagged or
many VLANs need to be presented to the server a vNet is a very simple network to configure and
manage within Virtual Connect.
A vNet can be utilized in one of two ways, a simple vNet, used to pass untagged frames and a
tunneled vNet. A tunneled vNet can be used to pass many VLANs without modifying the VLAN tags,
functioning as a transparent VLAN Pass-Thru module.
Shared Uplink Set (SUS)

The SUS provides the ability to support VLAN tagging and forward frames based on the VLAN tags of
those frames. The SUS connects one or many server NICs to one or many uplink ports. A SUS would
be configured for the specific VLANs it will support. If support for additional VLANs is required, those
VLANs need to be configured within the SUS.
When connecting a server NIC to a network within a SUS, there are two choices provided. The key
difference between these two options is the state in which the frame is passed to the server NIC;
1.
Select a single network which would be mapped to a specific VLAN.

If a single network is selected, the frames will be presented to the server NIC WITHOUT a
VLAN tag. In this case the host operating system does not need to understand which VLAN it
resides in. When the server transmits frames back to VC, those frames will not be tagged,
however; Virtual Connect will add the VLAN tag and forward the frame onto the correct VLAN.
2.
Select multiple networks which would provide connectivity to several VLANs.

The Map VLAN Tags feature provides the ability to use a Shared Uplink Set to present multiple
networks to a single NIC. If you select Multiple Networks when assigning a Network to a
server NIC, you will have the ability to configure multiple Networks (VLANS) on that server
NIC. At this point VC tags ALL the packets presented to the NIC unless the Native check box
is selected for one of the networks, in which case packets from this network (VLAN) will be
untagged, and any untagged packets leaving the server will be placed on this Network
(VLAN).
With Mapped VLAN Tags, you can create a Shared Uplink Set that contains ALL the VLANs
you want to present to your servers, then present only ONE network (the one associated with
the VLAN we want the server NIC in) to the Windows, LINUX or the ESX Console NIC, then
select Multiple Networks for the NIC connected to the ESX vSwitch and select ALL the networks
that we want presented to the ESX host vSwitch. The vSwitch will then break out the VLANs and
present them to the guests. Using Mapped VLAN Tags minimizes the number of uplinks
required.
In order to utilize the Multiple Networks feature of Virtual Connect, the Map VLAN Tags
feature, needs to be turned on under the Ethernet Settings/Advanced tab within the Virtual
Connect manager or the Virtual Connect CLI.
Benefits of a SUS
A Shared Uplink Set can be configure to support both tagged and un-tagged network traffic to a server
NIC, which simplifies the overall configuration and minimizes the number of uplink cables required to
support the network connections.
11
Virtual Connect VLAN Support SUS - Restrictions and limitations

Shared Uplink Sets provide administrators with the ability to distribute VLANs into discrete and
defined Ethernet Networks (vNet.) These vNets can then be mapped logically to a Server Profile
Network Connection allowing only the required VLANs to be associated with the specific server NIC
port. This also allows the flexibility to have various network connections for different physical
Operating System instances (i.e. VMware ESX host and physical Windows host.)
The following Shared Uplink Set rules apply per domain:
320 VLANs per Virtual Connect Ethernet module. Note; Every VLAN on every uplink counts
towards the 320-VLAN limit. If a Shared Uplink Set is comprised of multiple uplinks *from the
same module*, each VLAN on that Shared Uplink Set is counted multiple times towards 320
limit.
128 VLANs per Shared Uplink Set (64 VLAN Support is provided in VC firmware older than
2.30)
28 VLANs per Ethernet server profile connection regardless of physical vs. FlexNIC connections.
On Flex-10 adapters, this amounts to 28 x 4 = 112 VLANs per physical 10Gb port. Note; only
applies when in Mapped VLAN mode.
The above configuration rules apply only to a Shared Uplink set. If support for a larger numbers
of VLANs is required, the VC domain can be set to Tunneled VLAN mode and a VLAN Tunnel
con be configured to support a large number of VLANs.
Please see the Virtual Connect 3.15 Release Notes for future details.
12
S
Single
e Dom
main/E
Enclosure Sccenarios
O
Overviiew
This Cookbook will
w provide sevveral configurration scenariios of Virtual C
Connect FlexFFabric, using a
Sing
gle HP BladeS
System c7000
0 enclosure wiith two Virtuall Connect FlexxFabric modu
ules installed in
Bays 1 and 2. Virtual
V
Connecct FlexFabric also
a supports M
Multi-Enclosurre stacking, fo
or up to 4
encllosures, which
h provides a single
s
point off managemennt and can furrther reduce ca
able connectivity
requ
uirements. Ea
ach scenario will
w provide an overview off the configura
ation, show how to comple
ete
that configuration
n and include both GUI and
d CLI (scripted
d) methods. W
Where possib
ble, examples for
Win
ndows and/or VMware vSp
phere will also
o be provided
d.
R
Requirem
ments
This Cookbook will
w utilize a sin
ngle HP Blade
eSystem c700
00 enclosure w
with TWO Virrtual Connect
Flex
xFabric modules installed in
n I/O Bays 1 and 2 and a BL460c G7 h
half height Bla
adeSystem Servers
in se
erver Bay 1. The server wiill connect to the
t FlexFabricc models in Ba
ays 1 and 2, through with two
10G
Gb FlexFabric adapter ports. Each Flex
xFabric Adaptter port suppo
orts Ethernet and iSCSI or FFibre
odule in Bay 1 and Port 2 will
Cha
annel over Ethernet (FCoE). Port 1 will connect to the FlexFabric mo
connect to the Fle
exFabric modu
ule in Bay 2.
Pairr of managed Ethernet switcches, as well as a pair of B
Brocade Fibree Channel switches are
connected to the FlexFabric mo
odules.
It is assumed thatt a Virtual Con
nnect Domain
n has been creeated either th
hrough the GU
UI or a CLI script
and no VC Netw
works, uplink se
ets or Server Profiles have been created. Virtual Con
nnect scripting
g
examples are pro
ovided in App
pendix C.
c700
00 enclosure with
w Half Heightt G7 BladeSystem servers and
d two Virtual Co
onnect FlexFab
bric modules in
Interrconnect module
e bays 1& 2.
Figu
ure 3 - c700
00 enclosure with
w Half Heig
ght G7 BladeS
System serverss and two Virrtual Connect
Flex
xFabric modules in Interconnect module bays
b
1& 2
Single Domain/Enclosure Scenario

os
13
Scenario 1 Simple vNet with Active/Standby

Uplinks and FCoE Windows 2008 R2
Overview
This simple configuration uses the Virtual Connect vNet along with FCoE for SAN connectivity. When
VLAN mapping is not required, the vNet is the simplest way to connect Virtual Connect to a network
and server. In this scenario, the upstream network switch connects a network to a single port on each
FlexFabric module. In addition, Fibre Channel uplinks will also be connected to the FlexFabric
modules to connect to the existing Fibre Channel infrastructure.
No special upstream switch configuration is required as the switch is in the factory default
configuration, typically configured as an Access or untagged port on either the default VLAN or a
specific VLAN, either way, Virtual Connect does not receive VLAN tags.
When configuring Virtual Connect, we can provide several ways to implement network fail-over or
redundancy. One option would be to connect TWO uplinks to a single vNet; those two uplinks
would connect from different Virtual Connect modules within the enclosure and could then connect to
the same or two different upstream switches, depending on your redundancy needs. An alternative
would be to configure TWO separate vNets, each with a single, or multiple, uplink configured. Each
option has its advantages and disadvantages. We will review the first option in this scenario.
In addition, several vNets can be configured to support the required networks to the servers within the
BladeSystem enclosure. These networks could be used to separate the various network traffic types,
such as iSCSI, backup, VMotion from production network traffic.
This scenario will also leverage the Fibre Channel over Ethernet (FCoE) capabilities of the FlexFabric
modules. Each Fibre channel fabric will have one uplink connected to each of the FlexFabric
modules.
Requirements
This scenario will support both Ethernet and fibre channel connectivity. In order to implement this
scenario, an HP BladeSystem c7000 enclosure with one or more server blades and TWO Virtual
Connect FlexFabric modules, installed in I/O Bays 1& 2 are required. In addition, we will require
ONE or TWO external Network switches. As Virtual Connect does not appear to the network as a
switch and is transparent to the network, any standard managed switch will work with Virtual
Connect. The Fibre Channel uplinks will connect to the existing FC SAN fabrics. The SAN switch
ports will need to be configured to support NPIV logins. One uplink from each FlexFabric module will
be connected the existing SAN fabrics.
Scenario 1 Simple vNet with Active/Standby Uplinks and FCoE Windows 2008 R2 14
Figu
ure 4 - Phys
sical View; Shows a sing
gle Ethernet up
plink from Porrt X5 on Module 1 to Port 1 on
the first
f
network switch,
s
a single uplink from
m Port X5 on M
Module 2 to Port 1 on the ssecond network
swittch.
The SAN fabric is also conneccted as Module 1 port X1 t o SAN switch
h/fabric A and module 2 p
port
X1 to SAN switch/fabric B.
Scenario 1 Simple vNet with

w Active/Sta ndby Uplinks a
and FCoE Windows 2008 R
R2 15
Figure 5 - Logical View; Shows

S
a single
e Ethernet upllink from Port X5 on Module 1 on the firrst
netw
work switch and a single uplink from Porrt X5 on Mod ule 2 to the seecond networrk switch. Botth
Ethe
ernet uplinks are
a connected
d to the same vNet, vNet-PR
ROD. In addition, SAN Fa
abric FCoE_A
con
nnects to the existing
e
SAN Fabric
F
A throu
ugh port X1 o
on Module 1 ( Bay 1) and FC
CoE_B connects to
the existing SAN Fabric B thro
ough port X1 on
o Module 2 (Bay 2).
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
nd B provide a summary of the command
ds required to
o configure the
e switch in eitther a
Ciscco IOS or a PrroCurve network infrastructture. The conffiguration info
ormation provided in the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
are configured as ACCESS or unta gged ports, eeither presenting the Defaullt
VLAN or a specific
s
VLAN
N and will be forwarding
f
unntagged frames
As an altern
native, if the sw
witch ports we
ere configured
ports and forw
warding multiple
d as TRUNK p
VLANS, Virttual Connect would
w
forward
d those taggeed frames to th
he host NICs configured fo
or this
network, however; the Virtual Connect domain wou ld need to bee configured in
n Tunnel Mo
ode
et would need
d to be configured for VLAN
N Tunneling. The connecte
ed host would then
and the vNe
need to be configured
c
to interpret those VLAN tags..
This scenario assumes the switch port is con

nfigured as ann Access port (not tagging),, the Virtual
nnect domain could be configured for eth
her Tunneled or Mapped V
VLAN mode, h
however, the vvNet
Con
is NOT
N
configure
ed for VLAN Tunneling
T
and the frames a re presented tto Virtual Con
nnect as untag
gged
The network switcch port should
d be configure
ed for Spanni ng Tree Edgee as Virtual Co
onnect appears to
the switch
s
as an access
a
device
e and not another switch. B
By configuring
g the port as S
Spanning Tree
Edge, it allows th
he switch to pllace the port into
i
a forward
ding state mucch quicker tha
an otherwise, this
allow
ws a fail-over link or newerr connected port
p to come o
online much quicker.
The SAN switch ports
p
connectiing to the Flex
xFabric modu le must be co nfigured to acccept NPIV lo
ogins.
V
VC CLI commands
In addition to the GUI many off the configura
ation settings within VC can be also be accomplished
d via
a CLI command set.
s
In order to
t connect to VC
V via a CLI, open an SSH
H connection tto the IP addrress of
the active
a
VCM. Once logged
d in, VC proviides a CLI wit h help menus. The Virtual Connect CLI g
guide
R2 16
also provides many useful examples. Throughout this scenario the CLI commands to configure VC for
each setting are provided.
Configuring the VC module
Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1.
Physically connect Port X1 on the FlexFabric module (Bay 1) to switch port in SAN Fabric A
Physically connect Port 1 of the second Network switch to Port X5 of the VC module in Bay 2, if
you have only one network switch, connect VC port 1 (Bay 2) to an alternate port on the same
switch. This will NOT create a network loop and Spanning Tree is not required
Physically connect Port X1 on the FlexFabric module (Bay 2) to switch port in SAN Fabric B
Configuring Fast MAC Cache Failover

When an uplink on a VC Ethernet Module that was previously in standby mode becomes active, it
can take several minutes for external Ethernet switches to recognize that the c-Class server blades must
now be reached on this newly active connection.
Enabling Fast MAC Cache Failover forces Virtual Connect to transmit Ethernet packets on newly
active links, which enables the external Ethernet switches to identify the new connection (and
update their MAC caches appropriately). This transmission sequence repeats a few times at the
MAC refresh interval (five seconds is the recommended interval) and completes in about one
minute.
Configuring the VC Module for VLAN Tunneling and Fast Mac Cache Fail-over
via GUI (Ethernet settings)
Virtual Connect tunnel mode can be enabled to provide the ability to tunnel an unlimited number of
tagged VLANs through a Virtual Connect vNet. This can be very using in a network environment that
utilizes a large number of VLANs. Note; for this imply scenario, the Domain mode could be configure
for either Map VLAN tags or Tunnel mode.
Enable Tunnel VLAN Tags within Virtual Connect
On the Virtual Connect Manager screen, Left pane, click Ethernet Settings, Advanced Settings
Select Tunnel VLAN Tags
Select Fast MAC Cache Fail-over with a refresh of 5
Select Apply
Configuring the VC Module for VLAN Tunneling and Fast Mac Cache Fail-over
via CLI (Ethernet settings)
The following command can be copied and pasted into an SSH based CLI session with Virtual
Connect;
# Set Advanced Ethernet Settings to "Tunnel VLAN Tags" and Enable Fast MAC cache fail-over
set enet-vlan vlantagcontrol=Tunnel (click apply)
Click the Other tab
set mac-cache Enabled=True Refresh=5 (click apply)
Figu
ure 6 - Configure Tunnel Mode
M
(under Ethernet
E
Settinngs Advanceed Settings)
Figure 7 - Set Fa
ast MAC Cacche (under Eth
hernet Settingss Advanced Settings - Oth
her)

R2 18
Defining a new vNet via GUI

Create a vNet and name it vNet-PROD
Login to Virtual Connect, if a Domain has not been created, create it now, but cancel out of the
configuration wizards after the domain has been created.
On the Virtual Connect Manager screen, click Define, Ethernet Network to create a vNet
Ether the Network Name of vNet-PROD
Note; Do NOT select any of the options (ie; Smart Link, Private Networks etc.)
Select Add Port, then add the following ports;
Enclosure 1, Bay 1, Port X5
Leave Connection Mode as Auto

Select Apply
Note: By connecting TWO Uplinks from this vNet we have provided a redundant path to the network.
As each uplink originates from a different VC module, one uplink will be Active and the second will
be in Standby. This configuration provides the ability to lose an uplink cable, network switch or
depending on how the NICs are configured at the server (teamed or un-teamed), even a VC module.
Note: Smart Link In this configuration Smartlink should NOT be enabled. Smartlink is used to turn
off downlink ports within Virtual Connect, if ALL available uplinks to a vNet are down. We will use
Smartlink in a later scenario.
Defining a new vNet via CLI

The following command(s) can be copied and pasted into an SSH based CLI session with Virtual
Connect
# Create the vNet "vNet-PROD" and configure uplinks as discussed above
add Network vNet-PROD
add uplinkport enc0:1:X5 Network=vNet-PROD speed=auto
add uplinkport enc0:2:X5 Network=vNet-PROD speed=auto
set network vNet-PROD SmartLink=Disabled
Figure 8 - Defin
ne Ethernet Network (vNet--PROD). Note : The Port Status and Conn
nected to
info
ormation. If th
he connected switch supporrts LLDP, the cconnected to information sh
hould be displlayed
as below
b
D
Defining a new (FCoE) SAN Fabric via
a GUI
Crea
ate a Fabric and
a name it FCoE_A
On the Virtu
ual Connect Manager
M
scree
en, click Definne, SAN Fabric to create th
he first Fabric
Ether the Ne
etwork Name of FCoE_A
Select Add Port,
P
then add
d the following
g ports;
Select Apply
y
Crea
ate a second Fabric and na
ame it FCoE_
_B
On the Virtu
ual Connect Manager
M
scree
he second Fab
bric
Ether the Ne
etwork Name of FCoE_B
Select Add Port,
P
then add
d the following
g ports;
Select Apply
y
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
n be copied and pasted into
o an SSH bassed CLI sessio
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
E_A and FCoE_B and conffigure uplinks as discussed above
Add
d fabric FCoE_
_A Bay=1 Ports=1
Add
d fabric FCoE_
_B Bay=2 Porrts=1

R2 20
Figu
ure 9 FCoE
E SAN fabricss configured with
w a single 4
4Gb uplink peer fabric. No
ote the bay an
nd
portt numbers on the right
Figu
ure 10 FCo
oE SAN Serve
er Connection
ns (After serveer profiles havve been create
ed, assigned and
servvers powered on)
D
Defining a Server Profile with
h NIC and
d FCoE Co
onnectionss, via GUII
E
Each server NIC
N will connect to a spe
ecific networrk.
On the Virtual Co
onnect Manag
ger screen, cliick Define, Seerver Profile to
o create a Serrver Profile
Create a serrver profile ca

alled App-1
In the Netwo
ork Port 1 dro
op down box, select vNet--PROD
In the Netwo
ork Port 2 dro
op down box, select vNet--PROD
Expand the FCoE Connecctions box, for Bay 1, selecct FCoE_A forr Bay 2, selectt FCoE_B
N or iSCSI Connection
Do not confiigure FC SAN
In the Assign
n the Profile to
o a Server Bay, select Bay 1 and apply
Prior to applying the profile, ensure that the server in Bayy 1 is currentlyy OFF
Notte: you should
d now have a server profile
e assigned to Bay 1, with 2 Server NIC connections.
NIC
Cs 1&2 should
d be connected to network vNet_PROD a
and FCoE SAN
N fabrics FCo
oE_A and FCo
oE_B.

R2 21
Defining a Server Profile with

h NIC and
d FCoE Co
onnectionss, via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
# Create and Asssign Server Prrofile App-1 to
o server bay 1
add
d profile App-1
1
set enet-connectio
e
on App-1 1 px
xe=Enabled Network=vNe
N
et-PROD
set enet-connectio
e
on App-1 2 px
xe=Disabled Network=vNe
N
et-PROD
set fcoe-connectio
f
on App-1:1 Fa
abric=FCoE_A
A SpeedType=
=4Gb
set fcoe-connectio
f
on App-1:2 Fa
abric=FCoE_B
B SpeedType=
=4Gb
assign profile App-1 enc0:1
Figu
ure 11 - Define Server Profile
e (App- 1)
Notte: The speed of the NIC an
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that
the FCoE connecttions are assig

gned to the tw
wo SAN fabriccs created ea
arlier and use ports LOM:1--b and
LOM
M:2-b.

R2 22
Figu
ure 12 - Serve
er Profile View Bay
B 1
R
Review
w
In th
his scenario we
w have create
ed a simple vNet
v
utilizing uuplinks originating from the
e each FlexFa
abric
Mod
dule, by doing so we proviide redundant connections out of the Virrtual Connect domain, whe
ere
one
e uplink will be
e active and the
t alternate uplink
u
will be in standby. W
We create tw
wo FCoE SAN
Fab
brics, utilizing a single uplin
nk each.
We
e created at se
erver profiles, with two NIC
Cs connected to the same vvNet, which provides the ab
bility
to sustain a link or
o module failure and not lo
ose connectio
on to the network; this confiiguration also
o
gua
arantees that ALL
A server to server
s
communications withhin the enclosuure will remaiin inside the
enclosure. Altern
natively, we could have cre
eated a two v Net and assig
gned uplinks ffrom one mod
dule to
each vNet, providing an Activve/Active uplink scenario.
The FCoE SAN fa
abric connects to each SAN
N fabric over a single uplin
nk per module
e.
Add
ditional uplinkks could be ad
dded to eitherr the San fabrrics or the Etheernet networkks, which could
incrrease performance and/or availability.
R2 23
Results Windo
ows 200
08 R2 Network
N
king Exa
amples
We have successsfully configured FlexFabricc with a simplee vNet and reedundant SAN
N fabrics. W
We
have
e created a se
erver profile to
o connect to the
t vNet with TWO NICs a
and the SAN fabrics using the
FCo
oE connectionss created with
hin the profile.
Although both Eth
hernet and Fib
bre channel connectivity is provided by the CNA ada
apter used the
e in
G7 servers; each capability (LA
AN and SAN
N) is provided by a different component o
of the adapter,
they
y appear in the server as in
ndividual netw
work and SAN
N adapters.
Figu
ure 13 - Exam
mple of Emulex's OneManager Utility (formerrly known as H BA Anywhere).. Note that therre are
3 Ethernet persona
alities and one FCoE
F
personaliity per port, as configured in tthe server profile.
The following gra

aphics show a Windows 2008 R2 serveer with TWO FlexNICs configured at 6G
Gb.
You
u will also notiice that Wind
dows believes there are 6 N
NICs within thhis server. Ho
owever, only TTWO
NIC
Cs are currently configured within FlexFa
abric, the extra
a NICs are offfline and cou
uld be disabled. If
we did not requirre SAN conne
ectivity on thiss server, the FFCoE connectiions could be deleted and the
servver would then
n have 8 NIC
C ports availab
ble to the OS..
Note: the BL465cc G7 and BL6
685c G7 utiliz
ze a NC551i chipset, wherreas the BL46
60c G7, BL620c
G6 and BL680c G7
G utilizes an
n NC553i chiipset.
Figu
ure 14 - Wind
dows 2008 R2 Network Conn
nections (2 Connnections Activee
Notte: The NICs that

t
are not co
onfigured with
hin VC will ap
ppear with a red x as not cconnected. Yo
ou
can go into Netw
work Connectiions for the Windows
W
2008
8 server and Disable any N
NICs that are not
R2 24
on
currrently in use. Windows asssigns the NICs as NIC 1-6,, whereas threee of the NICs will reside o
LOM
M:1 and three
e on LOM:2. You may nee
ed to refer to tthe FlexFabricc server profile
e for the NIC MAC
add
dresses to verify which NIC is which.
Figu
ure 15 - Wind
dows 2008 R2 Extra Network Connections Disabled
ure 16 - Wind
nection Status
Figu
Note: In windowss 2003 the NIC

N speeds ma
ay not be show
wn accuratelyy when speed
ds are configured in
100
0MB incremen
nts above 1Gb
b. IE: if a NIC
C is configureed for 2.5Gb it will be disp
played in Win
ndows
200
03 as a 2Gb NIC.
N
Window
ws 2008 doe
es not have thiis limitation.

R2 25
ure 17 - Windows 2008 R2, Device Manager,

M
SIX NICs are sho
own, however,, we have only
Figu
conffigured two of the NICs and two FCoE HBAs.
H
The following gra

aphics provide
es an example
e of a Windo
ows 2008 R2 server with TW
WO NICs
connected to the network, initia
ally each NIC
C has its own TCP/IP addreess, alternative
ely, both NIC
Cs
coulld be teamed to provide NIC fail-over re
edundancy. If an active up
plink or netwo
ork switch were to
fail, Virtual Connect would faill-over to the sttandby uplinkk. In the event of a Virtual Connect FlexFFabric
mod
dule failure, th
he servers NIC
C teaming software would see one of th
he NICs go offfline, assumin
ng it
wass the active NIC, NIC teaming would fail-over to the sstandby NIC.
Figu
ure 18 - Both NICs
N
for Profile
e App-1are connected to the nnetwork through
h vNet-PROD

R2 26
NIC Team
ming
If higher availability is desired, NIC teaming
g in Virtual Co
onnect works the same wa
ay as in standa
ard
netw
work configura
ations. Simply, open the NIC
N teaming U
Utility and con
nfigure the avvailable NICs for
team
ming. In this example,
e
we have only TW
WO NICs avaiilable, so seleecting NICs fo
or teaming will be
quite
e simple. However, if multtiple NICs are
e available, ennsure that thee correct pair o
of NICs is tea
amed.
You will note the BAY#-Port# indication with
hin each NIC.. You would typically TEAM a NIC from
m Bay
1 to
o Bay 2 for ex
xample.
The following gra
aphics provide
e an example
e of a Window
ws 2008 R2 sserver with TW
WO NICs team
med
and connected to
o the network. In the event of an Uplink or switch failure, VC will fa
ail-over to the
e
stan
ndby uplinks, if a VC FlexFa
abric module were to fail, tthe NIC teaming software w
would fail-ove
er to
the standby
s
NIC.
Figu
ure 19 Team
m both NICs, ussing the HP Nettwork Configurration Utility
Figu
ure 20 - Both NICs
N
for Profile
e App-1are team
med and conneected to the nettwork through vvNet-PROD
Variious modes ca
an be configu
ured for NIC Teaming,
T
suchh as NFT, TLB etc. Once th
he Team is cre
eated,
you can select the
e team and ed
dit its propertiies. Typicallyy, the default ssettings can b
be used.

R2 27
Figu
ure 21 - View Network Co
onnections NIC
C Team #1 W
Windows
Figu
ure 22 - Both NICs are teamed and connecct to the networrk with a common IP Address
R
Results Windo
ows 200
08 R2 SAN
S
Co
onnectiviity
Figu
ure 23 - Durin
ng server boot, you can see that Port 1 of thee FlexHBA is co
onnected to an MSA1000 SAN
N LUN

R2 28
Figu
ure 24 - Wind
dows 2008 R2 Disk Administrator. Note; tha
at D: is the SAN
N attached volu
ume
S
Summa
ary
We presented a Virtual
V
Conne
ect Network sccenario by creeating a simp
ple vNet, we included a dual
path
h SAN fabric for storage co
onnectivity.
When VC profile App-1 is app
plied to the se
erver in bay1 and is powerred up, it has one NIC
connected throug
gh each FlexFa
abric module connected to vNet-PROD, which conn
nects to the ne
etwork
infra
astructure thro
ough the 10Gb uplinks. The
ese NICs coulld now be con
nfigured as in
ndividual NICss with
theirr own IP addrress or as a pa
air of TEAMED NICs. Eitheer NIC could be active. A
As a result, thiss
servver could acce
ess the networrk through eith
her NIC or eitther uplink ca
able, dependin
ng on which iis
activve at the time. Each NIC iss configured for
f 6Gb of neetwork bandw
width and each FCoE port iss
conffigured for 4G
Gb of SAN ba
andwidth.
Add
ditional NICs could
c
be added within Flex
xFabric, by si mply powerin
ng the server o
off and addin
ng up
to a total of 6 NIC
Cs, the NIC speed can then be adjusted
d accordingly to suit the needs of each N
NIC.
If ad
dditional or le
ess SAN band
dwidth is required, the speeed of the SAN
N connection ccan also be
adju
usted.
As additional
a
servers are adde
ed to the enclo
osure, simply create additio
onal profiles, or copy existting
proffiles, configure
e the NICs for vNet-PROD and the SAN fabrics and a
apply them to
o the appropriate
servver bays.

R2 29
Scenario 2 Shared Uplink Sets with

Active/Active uplinks and 802.3ad (LACP) Windows - Ethernet and FCoE
Overview
This scenario will implement the Shared Uplink Set (SUS) to provide support for multiple VLANs. In
this scenario, the upstream network switches connect a shared uplink set to two ports on each
FlexFabric modules, LACP will be used to aggregate those links.
As multiple VLANs will be supported in this configuration, the upstream switch ports connecting to the
FlexFabric modules will be configured to properly present those VLANs. In this scenario, the
upstream switch ports will be configured for VLAN trunking/VLAN tagging.
redundancy. One option would be to connect TWO uplinks to a single virtual connect network; those
two uplinks would connect from different Virtual Connect modules within the enclosure and could then
connect to the same upstream switch or two different upstream switches, depending on your
redundancy needs. An alternative would be to configure TWO separate Virtual Connect networks,
each with a single, or multiple, uplinks configured. Each option has its advantages and
disadvantages. We will review the second option in this scenario.
In addition, several Virtual Connect Networks can be configured to support the required networks to
the servers within the BladeSystem enclosure. These networks could be used to separate the various
network traffic types, such as iSCSI, backup, VMotion from production network traffic.
modules. Each Fibre channel fabric will have two uplinks connected to each of the FlexFabric
modules.
Requirements
ports will need to be configured to support NPIV logins. Two uplinks from each FlexFabric module
will be connected the existing SAN fabrics.
Scenario 2 Shared Uplink Sets with Active/Active uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE 30
Figure 25 - Phy
ysical View; Shows two Ethe
ernet uplinks from Ports X5
5 and X6 on M
Module 1 to Ports 1
and
d 2 on the firstt network swittch and two Ethernet
E
uplinkks from Ports X
X5 and X6 on
n Module 2 to
o Ports
1 and 2 on the second networrk switch. The
e SAN fabricss are also con
nnected redun
ndantly, with TTWO
upliinks per fabricc, from ports X1
X and X2 on
n module 1 to Fabric A and
d ports X1 and
d X2 to Fabricc B.
Scenario
o 2 Shared Up
plink Sets with Active/Active
A
uplinks
u
and 80 2.3ad (LACP) - Windows - Eth
hernet and FCo
oE 31
Figure 26 - Log
gical View; the
e server blade
e profile is connfigured with Four FlexNIC
Cs and 2 FlexH
HBAs.
NIC
Cs 1 and 2 arre connected to
t VLAN_101-x, NICs 3 annd 4 are conn
nected to multtiple networks
VLA
AN_102-x thro
ough VLAN_105-x, which are
a part of thee Shared Upliink Sets, VLAN
N-Trunk-1 and
d
VLA
AN-Trunk-2 resspectively. Th
he VLAN-Trunkks are conneccted, at 10Gb
b, to a networrk switch, thro
ough
Portts X5 and X6 on each FlexFFabric Module
e in Bays 1 a nd 2. In add
dition, SAN Fa
abric FCoE_A
A
con
e
SAN Fabric
F
A throu
ugh port X1 o
on Module 1 ( Bay 1) and FC
CoE_B connects to
the existing SAN Fabric B thro
ough port X1 on
o Module 2 (Bay 2).
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
ds required to
o configure the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
are configured as VLA
AN TRUNK po
orts (tagging) to support VLLANs 101 thro
ough
105. All fra
ames will be forwarding to Virtual Conneect with VLAN
N tags. Optio
onally, one VLLAN
could be configured as (D
Default) untagged, if so, theen a correspo
onding vNet w
within the Shared
w
be configured and se
et as Default.
Uplink Set would
Note: when adding additionall uplinks to the

e SUS, if the a
additional uplinks are conn
necting from the
me FlexFabric module to the
e same switch,, in order to eensure all the uplinks are acctive, the swittch
sam
portts will need to
o be configure
ed for LACP within
w
the samee Link Aggreg
gation Group..
The network switcch port should
d be configure
ed for Spanni ng Tree Edgee as Virtual Co
onnect appears to
the switch
s
as an access
a
device
e and not another switch. B
By configuring
g the port as S
Spanning Tree
Edge, it allows th
he switch to pllace the port into
i
a forward
ding state mucch quicker tha
an otherwise, this
allow
ws a fail-over link or newerr connected port
p to come o
online much quicker.
The SAN connecttion will be made
m
with redu
undant conneections to each
h Fabric. SAN
N switch portss
connecting to the
e FlexFabric module
m
must be
e configured tto accept NPIIV logins.
V
VC CLI commands
Man
ny of the conffiguration settiings within VC
C can also bee accomplisheed via a CLI co
ommand set. In
orde
er to connect to VC via a CLI,
C open an SSH
S
connectio
on to the IP ad
ddress of the a
active VCM. Once
logg
ged in, VC pro
ovides a CLI with
w help men
nus. Through this scenario the CLI comm
mands to confiigure
VC for each settin
ng will also be
e provided.
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 32
Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1
Physically connect Ports X1 and X2 on the FlexFabric module Bay 1 to switch port in SAN Fabric A

Physically connect Port 1 of the second Network switch to Port X5 of the VC module in Bay 2
Physically connect Port 1 of Network switch 1 to Port X6 on the VC module in Bay 2, if you have only one
network switch, connect VC port 1 (Bay 2) to an alternate port on the same switch. This will NOT create a
network loop and Spanning Tree is not required
Physically connect Ports X1 and X2 on the FlexFabric module Bay 2 to switch port in SAN Fabric B

minute.
Configuring the VC Module for Map VLAN Tags and Fast Mac Cache Fail-over
Note: Flex-10 has also provided additional controls when configuring VC for mapped VLAN tags
(Multiple Networks over a single link) support. These features provide the ability to set a Custom or
Preferred network speed value for each NIC. These are VC domain settings and when configured
will limit the maximum configurable speed of a NIC. In order to enable this capability, Mapped
VLAN Tags must be enabled.
Enable Map VLAN Tags within Virtual Connect
Select Map VLAN Tags (click apply)
Optionally, select a preferred/Maximum link speed
Select apply
Select the Other tab
Select Apply
Connect
# Set Advanced Ethernet Settings to "Map VLAN Tags" and set "Force server connections" to
disabled
set enet-vlan vlantagcontrol=map sharedservervlanid=false
set mac-cache Enabled=True Refresh=5
Note: Do not set a Preferred or

o Max speed
d for this scen ario. This ex ample is provvided for referrence
only
y. For the purrpose of this scenario,
s
we will
w not be co nfiguring the custom valuess for Preferred
d Link
e.
or Maximum
M
Con
nnection spee
eds. Howeverr, the CLI com
mmands are prrovided below
w for reference
Sele
ecting the Set a Custom vallue for Preferrred Link Conn ection Speed and/or Set a Customer va
alue
for Maximum Link Connection Speed, Sets the
t respectivee speed for alll connections using multiple
e
netw
works to 500M
Mb and maximum connection speed to 2
2.5Gb.
# Se
et Advanced Ethernet Settin
ngs to a Prefe
erred speed off 500Mb and
d a Max Spee
ed of 2500Mb
b
set enet-vlan
e
PrefSpeedType=C
Custom PrefSp
peed=500 Ma
axSpeedType=Custom MaxxSpeed=2500
0
Figu
ure 27 - Ethern
net Settings - Map
M VLAN Tagss (under Etherrnet Settings Advanced Se
ettings)
Figure 28 - Sett Fast MAC Cache (under Ethernet

E
Settinngs Advanceed Settings - O
Other)
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 34
Defining a new Shared Uplink Set (VLAN-Trunk-1)

Connect Port 1 of FlexFabric module 1 to Port 1 on switch 1
Create a SUS named VLAN-Trunk-1 and connect it to FlexFabric Port X1 on Module 1
On the Virtual Connect Home page, select Define, Shared Uplink Set
Insert Uplink Set Name as VLAN-Trunk-1
Select Add Port, then add the following port;
Add Networks as follows (to add a network, right click on the grey bar under the Associate
Networks (VLAN) header, the select ADD;
VLAN101-1 = VLAN ID=101
Enable SmartLink on ALL networks
Click Apply
Optionally, if one of the VLANs is configured as Default/untagged, on that VLAN only, set
Native to Enabled

Click Apply
Native to Enabled
Defining a new Shared Uplink Set via CLI

The following script can be used to create the first Shared Uplink Set (VLAN-Trunk-1)
Connect
# Create Shared Uplink Set "VLAN-Trunk-1" and configure uplinks
add uplinkset VLAN-Trunk-1
add uplinkport enc0:1:X5 Uplinkset=VLAN-Trunk-1 speed=auto
# Create Networks VLAN101-1 through VLAN104-1 for Shared Uplink Set "VLAN-Trunk-1"
add network VLAN101-1 uplinkset=VLAN-Trunk-1 VLanID=101
Set Network VLAN101-1 SmartLink=Enabled
The following script can be used to create the Second Shared Uplink Set (VLAN-Trunk-2)
Figu
ure 29 - Define
e a Shared Upllink Set (VLAN-TTrunk-1) and ad
dd VLANs
Notte: The Conn

nected to field in the grap
phic above dissplays the MA
AC address an
nd port number of
the switch
s
this uplink is conneccted to. This information is provided thro
ough LLDP disscover and is not
available in all sw
witch productss. This inform
mation can be very helpful w
when determining which
swittches and ports VC is conne
ected to.
Figu
ure 30 - Active
e / Active Sharred Uplink Set
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 37
Figu
ure 31 - Assocciated Networkks tab
Sum
mmary Notte: In this scen
nario we have
e created two independentt Share Uplinkk Sets (SUS), e
each
orig
ginating from the opposite FlexFabric
F
Mo
odules, by do
oing so we pro
ovide the ability to create
separate and red
dundant conne
ections out of the Virtual Co
onnect domaiin. When we
e create the se
erver
proffiles, you will see how the NICs will con
nnect to VLAN
Ns accessed th
hrough the opposite VC mo
odule,
which provides th
he ability to create an Activve / Active up
plink scenario
o. Alternatively, we could h
have
crea
ated a single SUS and assigned both setts of these upllink ports to th
he same SUS,, however, this
wou
uld have provvided an Activve/Standby up
plink scenario
o, example beelow.
Figu
ure 32 - Exam
mple of an Activve/Standby Sha
ard Uplink Set
D
a GUI
Crea
ate a Fabric and
a name it FCoE_A
On the Virtuual Connect Manager
M
scree
he first Fabric
Ether the Neetwork Name of FCoE_A
Select Add Port,
P
then add
d the following
g ports;
2
Select the Advanced Settiings
Select Automa
atic Login Re-Distribution
Select Applyy
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 38
Crea
ame it FCoE_
_B
M
scree
he second Fab
bric
Ether the Neetwork Name of FCoE_B
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
Add
d fabric FCoE_
_A Bay=1 Ports=1,2 LinkDiist=Auto
Add
d fabric FCoE_
_B Bay=2 Porrts=1,2 LinkDisst=Auto
Figu
ure 33 - SAN
N Login Distriibution Setting
g
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 39
Figu
ure 34 FCo
oE SAN fabrics configured
d with two 4G
Gb uplinks perr fabric. Note
e the bay and port
num
mbers on the riight
Figu
ure 35 FCo
oE SAN Serve
er Connection
ed, assigned and
D
Defining a Server Profile
We will
w create a se
erver profile with
w two serve
er NICs.
Each server NIC will connect to a specific nettwork.
n menu, selectt Define, then Server Profilee

On the main
alled App-1
In the Netwo
ork Port 1 dro
op down box, select VLAN1
101-1
In the Netwo
ork Port 2 dro
101-2
In the Assign
n Profile to Se
erver Bay box, locate the Seelect Location
n drop down a
and select Bayy 1,
they apply
Notte: you should
NIC
Cs 1&2 should
d be connected to networkss VLAN101-1 and VLAN10
01-2 and FCo
oE SAN fabriccs
FCo
oE_A and FCo
oE_B.
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 40
D
Defining a Server Profile via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
# Create Se
erver Profile App-1
A
add profile App-1
A
set enet-conn
nection App-1
1 1 pxe=Enab
bled Network=
=VLAN101-1
set enet-conn
nection App-1
1 2 pxe=Disab
bled Networkk=VLAN101-2
2
set fcoe-connection App-1
1:1 Fabric=FC
CoE_A Speed
dType=4Gb
1:2 Fabric=FC
CoE_B SpeedTType=4Gb
poweroff server 1
assign profile App-1 enc0
0:1
Figu
ure 36 - Define a Server Proffile (App-1)
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that

gned to the tw
LOM
M:2-b.
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 41
Scenario
Figu
ure 37 - Serve
er Profile View Bay 1
R
Review
w
In th
his scenario we
w have create
ed Two Share
ed Uplink Setss (SUS), providing support for 5 VLANs.
Uplinks originatin
ng from the ea
ach FlexFabric Module connnect to each SUS, by doin
ng so we provvide
redu
undant conne
ections out of the
t Virtual Co
onnect domainn. As multiplee uplinks are u
used for each SUS,
we have also leveraged LACP to improved uplink perform
mance. In this scenario, alll uplinks will be
active. We create
e two FCoE SAN
S
Fabrics.
We
e created a server profile, with
w two NICss connected to
o the same VLA
AN, VLAN_101-1 and
VLA
AN_101-2, wh
hich provides the ability to sustain a linkk or module fa
ailure and not lose connectiion to
the network. VLA
AN101-1 and VLAN101-2 are
a configureed to support V
VLAN 101, frrames will be
pressented to the NIC(s) withou
ut VLAN tags (untagged), thhese two NIC
Cs are connectted to the sam
me
VLA
AN, but taking
g a different path
p
out of the
e enclosure.
Add
ditionally, FCo
oE port 1 is co
onnected to SAN fabric FC
CoE_A and FC
CoE SAN portt is connected
d to
SAN
N Fabric FCoE_B, providing a multi-path
hed connected
d to the SAN.
The FCoE SAN fa
N fabric over a single uplin
nk per module
e.
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 42
R
Results Windo
ows 200
08 R2 Network
N
king Exa
amples
We have successsfully configured FlexFabricc with a simplee vNet and reedundant SAN
N fabrics. W
We
have
e created a se
erver profile to
o connect to the
t vNet with TWO NICs a
and the SAN fabrics using the
FCo
oE connectionss created with
hin the profile.
Although both Eth
hernet and Fib
apter used the
e in
AN and SAN
of the adapter,
they
ndividual netw
work and SAN
N adapters.
Figu
ure 38 - Exam
mple of Emulex's OneManage
er Utility (formerrly known as H BA Anywhere). Note that therre are
3 Ethernet persona
F
The following gra

Gb.
You
owever, only TTWO
NIC
abric, the extra
uld be disabled. If
servver would then
n have 8 NIC
C ports availab
ble to the OS..
685c G7 utiliz
60c G7, BL620c
G6 and BL680c G7
G utilizes an
n NC553i chiipset.
Figu
ure 39 - Wind
nections (2 Connnections Activee)
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 43

t
are not co
onfigured with
hin VC will ap
ou
can go into Netw
W
2008
NICs that are not
on
LOM
M:1 and three
e for the NIC MAC
add
Figu
ure 40 - Wind
Figu
ure 41 - Wind
nection Status

N speeds ma
ay not be show
100
0MB incremen
nts above 1Gb
b. IE: if a NIC
played in Win
ndows
200
03 as a 2Gb NIC.
N
Window
ws 2008 doe
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 44
Figu
M
SIX NICs are sho
H
The following gra

aphics provide
es an example
e of a Windo
WO NICs
ally each NIC
ely, both NIC
Cs
plink or netwo
ork switch were to
mod
dule failure, th
he servers NIC
ng it
Figu
ure 43 - Both NICs
N
for Profile
h vNet-PROD
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 45
N
NIC Team
ming
g in Virtual Co
ay as in standa
ard
netw
work configura
N teaming U
Utility and con
team
e
we have only TW
or teaming will be
quite
of NICs is tea
amed.
m Bay
1 to
o Bay 2 for ex
xample.
The following gra
aphics provide
e an example
e of a Window
WO NICs team
med
and connected to
o the network. In the event of an Uplink or switch failure, VC will fa
ail-over to the
e
stan
ndby uplinks, if a VC FlexFa
abric module were to fail, tthe NIC teaming software w
would fail-ove
er to
the standby
s
NIC.
Figu
ure 44 Team
Figu
ure 45 - Both NICs
N
for Profile
e App-1are team
med and conneected to the nettwork through vvNet-PROD
Variious modes ca
an be configu
T
he Team is cre
eated,
you can select the
e team and ed
be used.
Figu
onnections NIC
C Team #1 W
Windows
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 46
Figu
R
Results Windo
ows 200
08 R2 SAN
S
Co
onnectiviity
Figu
ure 48 - Durin
N LUN
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 47
Figu
ure 49 - Wind
at D: is the SAN
N attached volu
ume
S
Summa
ary
V
Conne
ect Network sccenario by creeating two sh
hared uplink sets (SUS), eacch
SUS
S is connected
d with TWO active
a
uplinks, both SUS ca
an actively pa
ass traffic. We
e included a d
dual
path
onnectivity.
plied to the se
connected throug
gh FlexFabric module 1 (connected to V
VLAN_101-1), the second NIC is conne
ected
through FlexFabriic module 2 (cconnected to VLAN_101-2 ). Each NIC i s configured at 6Gb. Thesse
NIC
Cs could now be configured
d as individua
al NICs with thheir own IP ad
ddress or as a pair of TEAM
MED
NIC
Cs. Either NIC
C could be acttive. As a ressult, this serveer could accesss the networkk through eithe
er
NIC
C or either uplink, dependin
ng on which NIC
N is active a
at the time. E
Each NIC is co
onfigured for 6Gb
of network bandw
width and eacch FCoE port is configured for 4Gb of SAN bandwidth.
Add
ditional NICs could
c
ng the server o
off and addin
ng up
to a total of 6 NIC
NIC.
If ad
dditional or le
ess SAN band
adju
usted.
As additional
a
servers are adde
ed to the enclo
e the NICs for vNet-PROD and the SAN fabrics and a
apply them to
o the appropriate
servver bays.
Scenario
o 2 Shared Up
A
uplinks
u
hernet and FCo
oE 48
Scenario 3 Shared Uplink Set with

Active/Active Uplinks and 802.3ad (LACP) Windows - Ethernet and FCoE Boot from SAN
Overview
FlexFabric modules, LACP will be used to aggregate those links. This scenario is identical to Scenario
2, however, scenario 3 also provides the steps to configure a windows 2008 R2 server to Boot to
SAN using the FC0E connections provided by FlexFabric. When using Virtual Connect/FlexFabric in
a Boot to SAN implementation, no custom or special HBA configuration is required. The HBA
configuration is controlled by Virtual Connect and maintained as part of the server profile. Once the
server profile has been configured and applied to the server bays, the controller will be configured on
the next and subsequent boot. When we later configure the server profile, we will also configured
the HBA boot parameters.
modules.
Requirements
Scenario 3 Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows - Ethernet and FCoE Boot from
SAN 49
Figure 50 - Phy
ysical View; Shows two Ethe
ernet uplinks from Ports X5
5 and X6 on M
Module 1 to Ports 1
and
d 2 on the firstt network swittch and two Ethernet
E
X5 and X6 on
n Module 2 to
o Ports
1 and 2 on the second networrk switch. The
e SAN fabricss are also con
nnected redun
ndantly, with TTWO
upliinks per fabricc, from ports X1
X and X2 on
n module 1 to Fabric A and
d ports X1 and
d X2 to Fabricc B.
Scenario 3 Shared Uplink Set with Active

e/Active Uplinkks and 802.3ad
d (LACP) - Windows - Ethernett and FCoE Boo
ot from
SAN
N 50
Figure 51 - Log
gical View; the
e server blade
e profile is connfigured with TWO FlexNICs and 2
Flex
xHBAs. NICs 1 and 2 are connected
c
to VLAN_101-x
V
which are pa
art of the Sharred Uplink Setts,
VLA
AN-Trunk-1 and VLAN-Trunkk-2 respectively. The VLAN
N-Trunks are cconnected, at 10Gb, to a
netw
work switch, through
t
Ports X5 and X6 on
n each FlexFa
abric Module in Bays 1 and
d 2. The FCo
oE
2 on each FleexFabric modu
SAN
N connectionss are connecte
ed through po
orts X1 and X2
ule. In additio
on,
SAN
N Fabric FCoE_A connects to the existing SAN Fabricc A through p
port X1 on Mo
odule 1 (Bay 1
1) and
FCo
oE_B connectss to the existin
ng SAN Fabricc B through p
port X1 on Mo
odule 2 (Bay 2
2).
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
ds required to
o configure the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
AN TRUNK po
orts to supporrt tagging on V
VLANs 101
through 105
5. All frames will be forwa
arding to Virtuual Connect w
with VLAN tags. Optionallyy, one
VLAN could be configure
ed as (Default)) untagged, iff so, then a co
orresponding vNet within th
he
Shared Uplink Set would be configured
d and set as Default.
Note: when adding the additio

onal uplinks to
o the SUS, if tthe additionall uplinks are cconnecting fro
om the
me Flex-10 module to the sa
ame switch, in order to ensuure all the upllinks are activve, the switch ports
sam
will need to be co
onfigured for LACP within the
t same Link Aggregation Group.
m
with redu
h Fabric. SAN
N switch portss
connecting to the
e FlexFabric module
m
must be
V
VC CLI commands
Man
ommand set. In
orde
C open an SSH
S
connectio
on to the IP ad
ddress of the a
active VCM. Once
logg
ged in, VC pro
ovides a CLI with
w help men
mands to confiigure
VC for each settin
ng will also be
e provided.

ot from
SAN
N 51


minute.
Select apply
Select Apply
Connect
disabled
SAN 52

o Max speed
only
s
we will
d Link
e.
or Maximum
M
Con
nnection spee
w for reference
Sele
alue
e
netw
works to 500M
2.5Gb.
# Se
ngs to a Prefe
d a Max Spee
ed of 2500Mb
b
set enet-vlan
e
PrefSpeedType=C
Custom PrefSp
peed=500 Ma
0
Figu
ure 52 - Ethern
net Settings - Map
ettings)

E
Other)

ot from
SAN
N 53

Connect Ports X5 and X6 of FlexFabric module 1 to Port 1 on switch 1
Create a SUS named VLAN-Trunk-1 and connect it to FlexFabric Ports X5 and X6 on Module 1
Enable Smartlink on ALL networks
Click Apply
Native to Enabled

Connect Ports X5 and X6 of FlexFabric module 2 to Ports X5 and X5 on switch 2
Enable smartlink on ALL networks
Click Apply

Native to Enabled
Connect
SAN 54

SAN 55
Figu
ure 54 - Define
dd VLANs
Notte: The Conn

AC address an
nd port number of
the switch
s
available in all sw
ected to.
Figu
ure 55 - Active

ot from
SAN
N 56
Figu
Summ
mary Note:: In this scenario we have created
c
two inndependent Share Uplink S
Sets (SUS), each
origin
nating from the
e opposite Fle
exFabric Modules, by doing
g so we proviide the abilityy to create sep
parate
and re
edundant con
nnections out of
o the Virtual Connect
C
dom ain. When w
we create the server profiless, you
will se
ee how the NICs will conne
ect to VLANs accessed
a
thro
ough the oppo
osite VC modu
ule, which pro
ovides
the ab
bility to create
e an Active / Active uplink scenario. Altternatively, w
we could have created a sin
ngle
SUS and
a assigned both sets of th
hese uplink po
orts to the sam
me SUS, howeever, this wou
uld have proviided
an Acctive/Standby
y uplink scenario, example below.
Figu
ure 57 - Exam
ard Uplink Set
D
a GUI
Crea
ate a Fabric and
a name it FCoE_A
M
scree
he first Fabric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Apply
y

ot from
SAN
N 57
Crea
ame it FCoE_
_B
M
scree
he second Fab
bric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
Add
d fabric FCoE_
Add
d fabric FCoE_
Figu
ure 58 - SAN
g
Figu
ure 59 FCo
d with a singlee 4Gb uplink per fabric. N
Note the bay a
and

ot from
SAN
N 58
Figu
ure 60 FCo
oE SAN Serve
er Connection
ed, assigned and
D
We will
w create a se
erver profile with
w two serve
er NICs.

On the main
alled App-1_
_BfS
In the Netwo
ork Port 1 dro
101-1
In the Netwo
ork Port 2 dro
101-2
eters box und
der the FCoE configuration box
Select the FFibre Channell Boot Parame
Sele
ect PORT 1 an
nd click on the drop down under SAN Boot and se
elect Primary
Clic
ck on the Targ
get Port Name
e field and ennter the SAN ccontroller ID
Clic
ck on the LUN
N field and enter the boot LU
UN number, w
which is typiccally 1
Clic
ck on PORT 2 and click on the drop dow
wn under SA
AN Boot and select
Se
econdary
Clic
ck on the Targ
get Port Name
e field and ennter the SAN ccontroller ID
Clic
ck on the LUN
N field and enter the boot LU
UN number, w
which is typiccally 1
Clic
ck Apply
In the Assign
n Profile to Se
n drop down a
and select Bayy 1,
they apply
Notte: you should
NIC
Cs 1&2 should
d be connected to networkss VLAN101-1 and VLAN10
01-2 and FCo
oE SAN fabriccs
FCo
oE_A and FCo
oE_B.

ot from
SAN
N 59
D
The following com
mmand(s) can
on with Virtuall
Con
nnect
# Create Se
erver Profile App-1
A
add profile App-1
A
set enet-conn
nection App-1
1 1 pxe=Enab
bled Network=
=VLAN101-1
set enet-conn
nection App-1
1 2 pxe=Disab
2
1:1 Fabric=FC
CoE_A SpeedTType=4Gb Bo
ootPriority=Primary
BootPort=50
0:08:05:F3:00:00:58:11 BootLun=1
B
1:2 Fabric=FC
CoE_B SpeedTType=4Gb Bo
ootPriority=Se
econdary
BootPort=50
0:08:05:F3:00:00:58:12 BootLun=1
B
poweroff server 1
0:1
Note: you
u will need to
t locate the
he WWN forr the contro
oller you are
e booting to
o
and substtitute the ad
ddresses ab
bove.
Figu
ure 61 - Define a Server Proffile (App-1), inccluding Boot to SAN configura
ation
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that

gned to the tw
LOM
M:2-b.

ot from
SAN
N 60
Figu
ure 62 - Configure Boot from
m SAN Controlller information
Figu
ure 63 - Serve
er Profile View Bay 1

ot from
SAN
N 61
Review
In this scenario we have created Two Shared Uplink Sets (SUS), providing support for 5 VLANs.
Uplinks originating from the each FlexFabric Module connect to each SUS, by doing so we provide
redundant connections out of the Virtual Connect domain. As multiple uplinks are used for each SUS,
we have also leveraged LACP to improved uplink performance. In this scenario, all uplinks will be
active. We create two FCoE SAN Fabrics.
We created a server profile, with two NICs connected to the same VLAN, VLAN_101-1 and
VLAN_101-2, which provides the ability to sustain a link or module failure and not lose connection to
the network. VLAN101-1 and VLAN101-2 are configured to support VLAN 101, frames will be
presented to the NIC(s) without VLAN tags (untagged), these two NICs are connected to the same
VLAN, but taking a different path out of the enclosure.
Additionally, FCoE port 1 is connected to SAN fabric FCoE_A and FCoE SAN port is connected to
SAN Fabric FCoE_B, providing a multi-pathed connected to the SAN.
We configured the server profile for Boot from SAN, connected a SAN LUN to the server, installed
and booted Windows 2008 R2 from the SAN LUN.
The FCoE SAN fabric connects to each SAN fabric over a pair of uplinks per module. SAN logins
are distributed across the multiple paths.
Results Windows 2008 R2 Networking Examples

We have successfully configured FlexFabric with a simple vNet and redundant SAN fabrics. We
have created a server profile to connect to the vNet with TWO NICs and the SAN fabrics using the
FCoE connections created within the profile.
Although both Ethernet and Fibre channel connectivity is provided by the CNA adapter used the in
G7 servers; each capability (LAN and SAN) is provided by a different component of the adapter,
they appear in the server as individual network and SAN adapters.
SAN 62
Figu
ure 64 - Exam
mple of Emulex's OneManager Utility (formerrly known as H BA Anywhere).. Note that therre are
3 Ethernet persona
F
The following gra

Gb.
You
owever, only TTWO
NIC
abric, the extra
uld be disabled. If
servver would then
n have 8 NIC
C ports availab
ble to the OS..
685c G7 utiliz
60c G7, BL620c
G6 and BL680c G7
G utilizes an
n NC553i chiipset.
Figu
ure 65 - Wind

t
are not co
onfigured with
hin VC will ap
ou
can go into Netw
W
2008
NICs that are not
on
LOM
M:1 and three
e for the NIC MAC
add

ot from
SAN
N 63
Figu
ure 66 - Wind
Figu
ure 67 - Wind
nection Status

N speeds ma
ay not be show
100
0MB incremen
nts above 1Gb
b. IE: if a NIC
played in Win
ndows
200
03 as a 2Gb NIC.
N
Window
ws 2008 doe

ot from
SAN
N 64
Figu
M
SIX NICs are sho
H
The following gra

aphics provide
es an example
e of a Windo
WO NICs
ally each NIC
ely, both NIC
Cs
plink or netwo
ork switch were to
mod
dule failure, th
he servers NIC
ng it
Figu
ure 69 - Both NICs
N
for Profile
h vNet-PROD

ot from
SAN
N 65
N
NIC Team
ming
g in Virtual Co
ay as in standa
ard
netw
work configura
N teaming U
Utility and con
team
e
we have only TW
or teaming will be
quite
of NICs is tea
amed.
m Bay
1 to
o Bay 2 for ex
xample.
The following gra
aphics provide
e an example
e of a Window
WO NICs team
med
and connected to
o the network. In the event of an Uplink or switch failure, the SUS w
will lose
connection to the network; Sma
artLink will altter the NIC teeaming softwa
are to this eve
ent, by turning
g the
servver NIC port off,
o causing the NIC teamin
ng software to
o fail-over to th
he alternate N
NIC.
Figu
ure 70 Team
Figu
ure 71 - Both NICs
N
for Profile
e App-1are team
med and conneected to the nettwork through V
VLAN-Trunk-x, e
either
path
h could be activve

ot from
SAN
N 66
Variious modes ca
an be configu
T
he Team is cre
eated,
you can select the
e team and ed
be used.
Figu
onnections NIC
C Team #1 W
Windows
Figu
R
Results Windo
ows 200
08 R2 SAN
S
Co
onnectiviity
Figu
ure 74 - Durin
N LUN

ot from
SAN
N 67
Figu
ure 75 - Wind
at D: is the SAN
N attached volu
ume
S
Summa
ary
V
Conne
SUS
S is connected
d with TWO active
a
uplinks, both SUS ca
an actively pa
ass traffic. We
e included a d
dual
path
h FCoE SAN fabric
f
for storage connectivvity and boot to SAN.
plied to the se
connected throug
gh FlexFabric module 1 (connected to V
VLAN_101-1), the second NIC is conne
ected
through FlexFabriic module 2 (cconnected to VLAN_101-2 ). Each NIC i s configured at 6Gb. Thesse
NIC
Cs could now be configured
d as individua
al NICs with thheir own IP ad
ddress or as a pair of TEAM
MED
NIC
Cs. Either NIC
C could be acttive. As a ressult, this serveer could accesss the networkk through eithe
er
NIC
C or either uplink, dependin
ng on which NIC
N is active a
at the time. E
Each NIC is co
onfigured for 6Gb
of network bandw
width and eacch FCoE port is configured for 4Gb of SAN bandwidth.
Add
ditional NICs could
c
ng the server o
off and addin
ng up
to a total of 6 NIC
NIC.
If ad
dditional or le
ess SAN band
adju
usted.
As additional
a
servers are adde
ed to the enclo
e the NICs for the approprriate VLAN, SA
AN fabrics an
nd apply them
m to the
app
propriate serve
er bays.

ot from
SAN
N 68

Active/Active Uplinks and 802.3ad (LACP) Windows Hyper-V - Ethernet and FCoE SAN
Overview
modules.
Requirements
Scenario 4 Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - Windows Hyper-V - Ethernet and FCoE
SAN 69
Figure 76 - Phy
ysical View; Shows
S
two Eth
hernet uplinks from Ports X5
5 and X6 on M
Module 1 to PPorts
1 and 2 on the fiirst network sw
witch and two
o Ethernet upliinks from Portts X5 and X6 on Module 2 to
Portts 1 and 2 on the second network
n
switch
h. . The SAN
N fabrics are a
also connecte
ed redundantlyy,
with
h TWO uplinkks per fabric, from ports X1 and X2 on m
module 1 to Fa
abric A and p
ports X1 and X
X2 to
Fab
bric B.
Scenario 4 Shared Uplin

nk Set with Activve/Active Uplin
nks and 802.3a
ad (LACP) - Wiindows Hyper-V
V - Ethernet and
d FCoE
SAN
N 70
Figure 77 - Log
gical View; the server blade
e profile is co
onfigured with
h Four FlexNIC
Cs and 2
Flex
xHBAs. NICs 1 and 2 are connected
c
to VLAN_101-x,
V
, NICs 3 and 4 are conneccted to multiplle
netw
works VLAN_102-x through
h VLAN_105--x, which are part of the Sh
hared Uplink S
Sets, VLAN-Trrunk-1
and
d VLAN-Trunk--2 respectively
y. The VLAN--Trunks are co
onnected, at 10Gb, to a ne
etwork switch,
thro
ough Ports X5 and X6 on ea
ach FlexFabric Module in B
Bays 1 and 2. The FCoE S
SAN connectio
ons
are connected through ports X1
X and X2 on each FlexFab
bric module. In addition, S
SAN Fabric
FCo
oE_A connectss to the existin
ng SAN Fabriic A through p
port X1 on Mo
odule 1 (Bay 1) and FCoE__B
con
e
SAN Fabric
F
B throu
ugh port X1 onn Module 2 (B
Bay 2).
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
ds required to
o configure the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
AN TRUNK po
VLANs 101
through 105
he

onal uplinks to
om the
me Flex-10 module to the sa
ame switch, in order to ensuure all the upllinks are activve, the switch ports
sam
will need to be co
onfigured for LACP within the
t same Link Aggregation Group.
m
with redu
h Fabric. SAN
N switch portss
connecting to the
e FlexFabric module
m
must be
V
VC CLI commands
Man
ommand set. In
orde
C open an SSH
S
connectio
on to the IP ad
ddress of the a
active VCM. Once
logg
ged in, VC pro
ovides a CLI with
w help men
mands to confiigure
VC for each settin
ng will also be
e provided.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 71


minute.
Select apply
Select Apply
Connect
disabled
SAN 72

o Max speed
only
s
we will
d Link
e.
or Maximum
M
Con
nnection spee
w for reference
Sele
alue
e
netw
works to 500M
2.5Gb.
# Se
ngs to a Prefe
d a Max Spee
ed of 2500Mb
b
set enet-vlan
e
PrefSpeedType=C
Custom PrefSp
peed=500 Ma
0
Figu
ure 78 - Ethern
net Settings - Map
ettings)

E
Other)

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 73

Click Apply

Native to Enabled

Click Apply

Native to Enabled
SAN 74

Connect
SAN 75
Figu
ure 80 - Define
dd VLANs
Notte: The Conn

AC address an
nd port number of
the switch
s
available in all sw
ected to.
Figu
ure 81 - Active

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 76
Figu
Sum
nario we have
each
orig
F
Mo
odules, by do
oing so we pro
separate and red
dundant conne
e create the se
erver
nnect to VLAN
Ns accessed th
odule,
which provides th
plink scenario
have
crea
wou
plink scenario
o, example beelow.
Figu
ure 83 - Exam
ard Uplink Set
D
a GUI
Crea
ate a Fabric and
a name it FCoE_A
M
scree
he first Fabric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 77
Crea
ame it FCoE_
_B
M
scree
he second Fab
bric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
Add
d fabric FCoE_
Add
d fabric FCoE_
Figu
ure 84 - SAN
g
Figu
ure 85 FCo
d with a singlee 4Gb uplink per fabric. N
Note the bay a
and

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 78
Figu
ure 86 FCo
oE SAN Serve
er Connection
ed, assigned and
D
We will
w create a se
erver profile with
w two serve
er NICs.

On the main
alled App-1
In the Netwo
ork Port 1 dro
101-1
Set the port speed to Cusstom at 1Gb
In the Netwo
ork Port 2 dro
101-2
Left click on either of Port 1 or Port 1 in

n the Ethernett Connections box, and sele
ect ADD netw
work
(add two ad
dditional netw
work connectio
ons)
In the Netwo
ork Port 3 dro
op down box, select Multip
ple Networks
Configure for
f networks VLAN102-1
V
th
hrough VLAN 105-1
Leave the network
n
speed
d as Auto
In the Netwo
ork Port 4 dro
ple Networks
Configure for
V
th
hrough VLAN 105-1
Leave the network
n
speed
d as Auto
In the Assign
n Profile to Se
n drop down a
and select Bayy 1,
they apply
Notte: you should
NIC
Cs 1&2 should
d be connected to networkss VLAN101-x,, NICs 3&4 sh
hould be conn
nected to netw
works
VLA
AN102-x throu
ugh VLAN105
5-x and FCoE SAN fabrics FCoE_A and FCoE_B.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 79
D
The following com
mmand(s) can
on with Virtuall
Con
nnect
# Create Se
erver Profile App-1
A
add profile App-1
A
set enet-conn
nection App-1
1 1 pxe=Enab
bled Network=
=VLAN101-1 SpeedType=Custom Speed1000
set enet-conn
nection App-1
1 2 pxe=Disab
2 SpeedType=
=Custom Spee
ed1000
add enet-connection App
p-1 pxe=Disab
bled
add server-p
port-map App--1:3 VLAN_10
03-1 VLanId=
=103
add server-p
04-1 VLanId=
=104
add server-p
05-1 VLanId=
=105
add enet-connection App
p-1 pxe=Disab
bled
add server-p
03-2 VLanId=
=103
add server-p
04-2 VLanId=
=104
add server-p
05-2 VLanId=
=105
1:1 Fabric=FC
CoE_A Speed
dType=4Gb
1:2 Fabric=FC
CoE_B SpeedTType=4Gb
poweroff server 1
0:1
Figu
ure 87 - Define a Server Proffile (App-1)
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that

gned to the tw
LOM
M:2-b.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 80
Figu
ure 88 - Configure Customerr NIC Speed - move
m
the sliderr left/right to co
onfigure NIC sp
peed.
Figu
ure 89 - Configure NIC Port 3 for Multiple Networks - VLA
AN102-1 throu gh VLAN105-1
1

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 81
Figu
ure 90 - Configure NIC Port 4 for multiple Networks
N
- VLA
AN102-2 throug
gh VLAN105-2
2

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 82
Figu
ure 91 - Serve
er Profile View Bay
B 1
Figu
ure 92 - By ho
overing over the Multiple Networks page, a listing of the connected Nettworks is displa
ayed.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 83
Review
active. We also create two FCoE SAN Fabrics.
Network Ports 3 and 4 were added, these NICs will be connected to Multiple Networks and each
NIC will then be configured for networks VLAN102-x through VLAN105-x. As these networks are
tagging, frames will be presented to the server with VLAN tags. NICs 3 and 4 will be teamed and
connected to a Hyper-V virtual switch. VLAN tagged frames for these networks will be forwarded to
the Virtual switch and then passed on to the appropriate Virtual Machine.
Results Windows 2008 R2 Networking Examples

We have successfully configured FlexFabric with a simple vNet and redundant SAN fabrics. We
have created a server profile to connect to the vNet with TWO NICs and the SAN fabrics using the
FCoE connections created within the profile.
SAN 84
Figu
ure 93 - Exam
mple of Emulex's OneManage
er Utility (formerrly known as H BA Anywhere). Note that therre are
3 Ethernet persona
F
The following gra

aphics show a Windows 2008 R2 serveer with FOUR FlexNICs con
nfigured, two at
1Gb
b and two at 5Gb. You will also notice that Window
ws believes theere are 6 NIC
Cs within this
servver. Howeverr, only four NICs are curren
ntly configured
d within FlexFFabric, the exttra NICs are o
offline
and
d could be disabled. If we did not require SAN conneectivity on thiss server, the FFCoE connectiions
could be deleted and the serve
er would then have 8 NIC ports availab
ble to the OS.
685c G7 utiliz
ze an NC551 i chipset, wheereas the BL460c G7, BL620c
G6 and BL680c G7
G utilize an NC553i chip
pset.
Figu
ure 94 - Wind

t
are not co
onfigured with
hin VC will ap
ou
can go into Netw
W
2008
8 R2 server and disable an
ny NICs that a
are
not currently in usse. Windowss assigns the NICs
N
as NIC 1-6, whereass three of the N
NICs will resid
de on
LOM
M:1 (a,c &d)a
and three on LOM:2
L
(a,c &d).
&
You may need to referr to the FlexFa
abric server prrofile
for the
t NIC MAC
C addresses to
o verify which NIC is whichh.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 85
Figu
ure 95 - Wind
nections (2 Connnections Disab
bled)
Figu
ure 96 - Wind
nection Status

N speeds ma
ay not be show
100
0MB incremen
nts above 1Gb
b. IE: if a NIC
played in Win
ndows
200
03 as a 2Gb NIC.
N
Window
ws 2008 doe

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 86
Figu
M
SIX NICs are sho
conffigured four of
o the NICs an
nd two FCoE HBAs.
H
The following gra

aphics provide
es an example
e of a Windo
ows 2008 R2 server with fo
our NICs conn
nected
to th
he network, in
nitially each NIC
N has its ow
wn TCP/IP add
dress, alternattively, NICs could be teamed to
provvide NIC fail-o
over redundancy. In this sccenario we w
will create two teams, one fo
or the management
netw
work (VLAN 101) and one for the Virtual guest netwo rks (VLANs 102 through 105). If an acctive
o the standbyy uplink. In the
uplink or networkk switch were to fail, Virtual Connect wo
ould fail-over to
even
nt of a Virtual Connect Flex
xFabric modulle failure, the servers NIC teaming softw
ware would se
ee
one of the NICs go
g offline, asssuming it was the active NI C, NIC teamiing would faill-over to the
stan
ndby NIC.
Figu
ure 98 Two NICs for Profile
e App-1are con
nnected to the nnetwork through
h VLAN-101 an
nd two NICs arre
conn
nected through VLAN 104.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 87
N
NIC Team
ming
Befo
ore configuring NIC teamin
ng, ensure tha
at the Hyper-V
V role has beeen added, then
n install the la
atest
verssion of the HP ProLiant Netw
work Teaming
g Software (veersion 10.10) and run the u
utility. Notice
e the
NIC
C Names that were set earliier. You can also highlightt each NIC an
nd select prop
perties to veriffy the
MAC address of the NIC and compare it to
o the MAC ad dress shown in the VC manager, before
e
crea
ating the team
m.
NIC
C teaming in Virtual
V
Connecct works the sa
ame way as iin standard neetwork config
gurations. Sim
mply,
open the NIC tea
aming Utility and
a configure the availablee NICs for teaming. In this example, we
e have
FOU
UR NICs availlable and will create two te
eams. The firsst team, using
g the two 1Gb
b NICs will be
e used
for access
a
to and management of the local operating
o
sysstem. You will note the BAY
Y#-Port# indiccation
with
hin each NIC. Select the NICs
N
that are indicated as B
Bay 1 Port 1/1 and Bay 2 Port 1/1 as the
NIC
Cs for the first team.
The second team will be used to provide acccess to the guuest VMs. Select the NICs that are indiccated
as Bay
B 1 Port 1/3 and Bay 2 Port 1/3 as th
he NICs for thhe second tea
am.
The following gra
aphics provide
e an example
e of a Window
WO NIC team
ms and
connected to the network. In the event of an Uplink or sw
witch failure, tthe SUS will lo
ose connectio
on to
the network; Sma
artLink will alte
er the NIC tea
aming softwarre to this even
nt, by turning the server NIC
C port
off, causing the NIC
N teaming software
s
to fail-over to the a
alternate NIC.
Notte: The Hyper-V role must be
b added beffore the NIC tteaming softw
ware is installe
ed or configurred. If
the HP NIC teaming Utility (NC
CU) is installed, prior to thee Hyper-V rolee being added
d, uninstall the
NCU by opening
g the property of one of the existing Loca
al Area Conneections, selectt the HP Netw
work
Con
nfiguration Utility and clickk Uninstall. NCU
N
version 1
10.10 or laterr should be installed after th
he
Hyp
per-V role has been added.
Figu
ure 99 - Uninsstall the HP NC
CU, reinstall the NCU (version 10.10 or laterr), after the Hyp
per-V role has b
been
adde
ed.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 88
Figu
ure 100 Tea
am both NICs, using
u
the HP Network Configuuration Utility Note the NIC numbers on the
e Left
and the Bay Port # on the right. You
Y typically would team the ssame port numb
ber to the oppo
osite bay numb
ber.
Figu
ure 101 The
e four NICs con
nfigured within the
t profile (App
p-1) are teamed
d in pairs and connected to th
he
netw
work. Note thatt each Team levverages one NIC from Bay 1 and 1 NIC from Bay 2.
Variious modes ca
an be configu
T
suchh as NFT or TLLB. Once the
e Team is crea
ated,
you can select the
e team and ed
be used. How
wever,
the team
t
that will be used for the
t VM Guests will need to
o have Promisccuous mode e
enabled, if VLA
AN
tagg
ging pass-thro
ough is require
ed.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 89
Figu
ure 102 - View
w Network Connections
C
NIC
N Team #1 (m
management) TTeam #2 Hyperr-V Guests
Figu
ure 103 - If VLAN pass-throu
ugh is required,, Promiscuous m
mode must be eenabled on the NIC Team that will
be connected
c
to the
e Hyper-V virtua
al switch.
Note
e: VLAN Promisscuous is a featture that was ad
dded to this ve rsion of the HPP NCU. The pu
urpose of VAL
Prom
miscuous is to allow VLAN tagged frames to pass through thhe NIC teaming
g software. If V
VLAN Promiscuous is
not enabled,
e
VLAN
N tagged framess will be dropp
ped. In earlier vversions of the HP NCU, in orrder to support VLAN
tagg
ging, an Etherne
et connection needed
n
to be de
efined for eachh VLAN. VLAN promiscuous p
provides the ab
bility of
supp
porting many VLANs, without the
t need to deffine each in thee NCU.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 90
Figu
ure 104 - Both
h teamed and connect
c
to the network,
n
each cconfigured for DHCP and rece
eived an IP Address.
Connection 8 is TEA
AM 1 and Con
nnection 9 is Te
eam 2 which w
will connect to th
he Hyper-v Virtu
ual Switch
C
Configurin
ng Hyper-V
V Virtual Network
N
On the Windowss 2008 R2 serrver, verify tha
at the Hyper-V
V role has beeen added and
d that HP NCU
U
verssion 10.10 or later has bee
en installed an
nd both NIC tteams as desccribed above have been
crea
ated. Open th
he Hyper-V manager conso
ole in Window
w 2008 R2 an
nd click on Viirtual Networkk
Man
nager. Selectt New Virtual Network, in the
t create Virt
rtual Network box, select E
External and
d click
ADD
D.
Figu
ure 105 - Cre
eate a new virtu
ual network (ste
ep 1)
Crea
ate the Virtual network for the guest Virtu
ual Machines . Name the vvirtual networrk as VM
Netw
work, select TEAM #2 as the network connection
c
(w
which is config
gured for VLAN
N tagging on
de-select Allow
VLA
ANs 102-105). Optionally,, if this networrk will not be used to mana
age the host, d
w
man
nagement ope
erating system
m to share this network ada pter. Click ap
pply to create
e the virtual
netw
work, a warning box will appear and the
e network ada
apter may resset as the virtu
ual network iss
crea
ated.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 91
Figu
ure 106 - Cre
eate the virtual network
n
(step 2)
2
C
Creating a Virtual Machine
M
and
a conne
ecting to th
he virtual network
Click NEW in the
t Hyper-V manager
m
and select
s
Virtual Machine to ccreate a new vvirtual machin
ne. In
the Virtual
V
Machine settings bo
ox, click on the network ad
dapter and configure it to use the VM
Netw
work created earlier, click on enable virtual LAN ideentification a
and input the V
VLAN ID 102
2.
This system will th
hen be conneccted to VLAN 102, click O
OK to apply the settings. Th
he next step w
will be
to in
nstall an opera
ating system in
i the virtual machine.
m

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 92
Figu
ure 107 - Con
nfigure the virtu
ual machine to function
f
on VLA
AN 102
TTesting VM
M connecttivity
Afte
er installing Windows
W
2008
8 R2 onto the VM, we can move the VM
M from VLAN 102 through VLAN
105
5, by simply changing
c
the VM
V ID that is configured onn the Networkk Adapter witthin the VM
Setttings page.
Figu
ure 108 - VM
M2 has received
d an IP address on VLAN 102
2

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 93
By simply
s
changiing the VLAN ID indicated in the Settingss of the Netw
work Adapter ffor this VM, w
we
can change the VLAN
V
that thiss VM connectss to.
Figu
ure 109 - Movve VM2 to VLA
AN 104 by changing the VLAN
N ID used for th
his VM.
Figu
ure 110 - VM2 has now rece
eived an IP add
dress on VLAN 104.

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 94
R
Results Windo
ows 200
08 R2 SAN
S
Co
onnectiviity
Figu
ure 111 - Durring server boott, you can see that
t
Port 1 of thhe FlexHBA is cconnected to an
n MSA1000 SA
AN
LUN
Figu
ure 112 - Win
ndows 2008 R2 Disk Adminisstrator. Note; tthat D: is the SA
AN attached vo
olume

nks and 802.3a
V - Ethernet and
d FCoE
SAN
N 95
Summary
We presented a Virtual Connect Network and SAN scenario by creating two shared uplink sets
(SUS), each SUS is connected with TWO active uplinks; both SUS can actively pass traffic. We
included a dual path SAN fabric for storage connectivity.
When VC profile App-1 is applied to the server in bay1 and is powered up, it has one NIC
connected through FlexFabric module 1 (connected to VLAN_101-1), the second NIC is connected
through FlexFabric module 2 (connected to VLAN_101-2). Each NIC is configured at 1Gb. These
NICs are teamed and will be used to manage the Hyper-v host. Either NIC or path could be active.
The second pair of NICs are also teamed and the team is configured in promiscuous mode to support
multiple tagged VLANs (102-105). Each of these NICs is configured for 5Gb. Either NIC could be
active. As a result, this server could access the network through either NIC or either uplink,
depending on which NIC is active at the time. This host is also configured for FCoE based SAN
access and connects to a SAN LUN to store the Guest VMs. Each FCoE port is configured for 4Gb of
SAN bandwidth.
Additional NICs could be added within FlexFabric, by simply powering the server off and adding up
to a total of 6 NICs, the NIC speed can then be adjusted accordingly to suit the needs of each NIC.
If additional or less SAN bandwidth is required, the speed of the SAN connection can also be
adjusted. If the FCoE SAN connections are not required, these could be deleted, in which two
additional NIC ports would then be made available to the host.
As additional servers are added to the enclosure, simply create additional profiles, or copy existing
profiles, configure the NICs for LAN and SAN fabrics as required and apply them to the appropriate
server bays and power the server on.
We then added the Hyper-V role to the server and created a VM guest. The guest was configured for
VLAN 102, and then was later moved to VLAN 104, by simply changing the VLAN id as configured
in the VM settings tab. Additional VLANs can be configured within the Shared Uplink Set and
presented to the NIC team supporting the guests; those VLANs would then be made available to
Hyper-V manager and any VM guests.
SAN 96

Active/Active Uplinks and 802.3ad (LACP) vSphere - Ethernet and FCoE SAN
Overview
modules.
Requirements
Scenario 5 Shared Uplink Set with Active/Active Uplinks and 802.3ad (LACP) - vSphere - Ethernet and FCoE SAN 97
Figure 113 - Physical View; Shows two Ethernet

E
X5 and X6 on
n Module 1 to
o Ports
witch and two
n
switch
h. . The SAN
N fabrics are a
also connecte
ed redundantlyy,
with
h TWO uplinkks per fabric, from ports X1 and X2 on m
module 1 to Fa
abric A and p
ports X1 and X
X2 to
Fab
bric B.
Scenario 5 Shared Uplink Set with Activve/Active Uplin

nks and 802.3a
ad (LACP) - vSp
phere - Ethernett and FCoE SAN
N
98
Figure 114 - Lo
ogical View; the
t server blade profile is cconfigured with SIX FlexNIC
C and 2 FlexH
HBAs.
NIC
nected to VLAN_102-x and
d
NIC
C3 5 and 6 arre connected to multiple ne
etworks VLAN _103-x throug
gh VLAN_105
5-x, which are
e part
of th
he Shared Up
plink Sets, VLA
AN-Trunk-1 an
nd VLAN-Trunkk-2 respectiveely. The VLAN
N-Trunks are
con
nnected, at 10
0Gb, to a netw
work switch, through Ports X
X5 and X6 on
n each FlexFa
abric Module in
Bay
ys 1 and 2. The
T FCoE SAN
N connections are connecteed through po
orts X1 and X2
2 on each
Flex
xFabric module. In addition, SAN Fabric FCoE_A co nnects to the existing SAN Fabric A thro
ough
port X1 on Modu
ule 1 (Bay 1) and
a FCoE_B connects
c
to thhe existing SA
AN Fabric B th
hrough port X1 on
Module 2 (Bay 2).
2
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
ds required to
o configure the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
AN TRUNK po
VLANs 101
through 105
he

onal uplinks to
om the
me FlexFabric module to the
e same switch,, in order to eensure all the uplinks are acctive, the swittch
sam
portts will need to
o be configure
ed for LACP within
w
the samee Link Aggreg
gation Group..
m
with redu
h Fabric. SAN
N switch portss
connecting to the
e FlexFabric module
m
must be
V
VC CLI commands
Man
ommand set. In
orde
C open an SSH
S
connectio
on to the IP ad
ddress of the a
active VCM. Once
logg
ged in, VC pro
ovides a CLI with
w help men
mands to confiigure
VC for each settin
ng will also be
e provided.
Scenario 5 Shared Uplink Set with Activve/Active Uplin

nks and 802.3a
ad (LACP) - vSp
phere - Ethernett and FCoE SAN
N
99


minute.
Select apply
Select Apply
Connect
disabled

o Max speed
only
s
we will
d Link
e.
or Maximum
M
Con
nnection spee
w for reference
Sele
alue
e
netw
works to 500M
2.5Gb.
# Se
ngs to a Prefe
d a Max Spee
ed of 2500Mb
b
set enet-vlan
e
PrefSpeedType=C
Custom PrefSp
peed=500 Ma
0
ernet Settingss Advanced S
Figu
ure 115 - Ethe
ernet Settings - Map
M VLAN Tag
gs (under Ethe
Settings)
Figure 116 - Set Fast MAC Cache (underr Ethernet Setttings Advancced Settings - Other)

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
101

Connect Ports X5 and X6 of FlexFabric module 1 to Ports and 2 on switch 1
Click Apply

Native to Enabled

Connect Ports X5 and X6 of FlexFabric module 2 to Ports 1 and 2 on switch 2
Click Apply

Native to Enabled

Connect
Figu
ure 117 - Define a Shared Uplink Set (VLAN
N-Trunk-1) and add VLANs
Notte: The Conn

AC address an
nd port number of
the switch
s
available in all sw
ected to.
Figu
ure 118 - Actiive / Active Shared Uplink Se
et

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
104
Figu
ure 119 - Asso
ociated Networks tab
Sum
nario we have
each
orig
F
Mo
odules, by do
oing so we pro
separate and red
dundant conne
e create the se
erver
nnect to VLAN
Ns accessed th
odule,
which provides th
plink scenario
have
crea
wou
plink scenario
o, example beelow.
Figu
ure 120 - Example of an Acttive/Standby Shard Uplink Seet
D
a GUI
Crea
ate a Fabric and
a name it FCoE_A
M
scree
he first Fabric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
105
Crea
ame it FCoE_
_B
M
scree
he second Fab
bric
Select Add Port,
P
then add
d the following
g ports;
2
Select Automa
Select Applyy
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
Add
d fabric FCoE_
Add
d fabric FCoE_
Figu
ure 121 - SAN
N Login Distribu
ution Setting
Figu
ure 122 FCo
oE SAN fabricss configured with a single 4Gb
b uplink per fa bric. Note the bay and port
numbers on the right

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
106
Figu
ure 123 FCo
oE SAN Server Connections (A
After server pro
ofiles have been
n created, assig
gned and serve
ers
pow
wered on)
D
We will
w create a se
erver profile with
w two serve
er NICs.

On the main
alled ESX-1
In the Netwo
ork Port 1 dro
101-1
In the Netwo
ork Port 2 dro
101-2
Left click on either of Port 1 or Port 2 in

n the Ethernett Connections box, then rig
ght click and sselect
ADD networrk (add four additional
a
netw
work connectiions)
In the Netwo
ork Port 3 dro
102-1
Set the port speed to Cusstom at 1.5Gb
b
In the Netwo
ork Port 4 dro
102-2
Set the port speed to Cusstom at 1.5Gb
b
In the Netwo
ork Port 5 dro
ple Networks
Configure for
V
th
hrough VLAN 105-1
Leave the network
n
speed
d as Auto
In the Netwo
ork Port 6 dro
ple Networks
Configure for
V
th
hrough VLAN 105-1
Leave the network
n
speed
d as Auto
In the Assign
n Profile to Se
n drop down a
and select Bayy 2,
they apply
Notte: you should
NIC
Cs 1&2 should
d be connected to networkss VLAN101-x,, NICs 3&4 sh
hould be conn
nected to
VLA
AN102-x (whicch will be use
ed for VMotion
n) NICs 5&6 should be con
nnected to networks VLAN103-x
through VLAN105-x and FCoE
E SAN fabricss FCoE_A and
d FCoE_B.
d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
107
D
The following com
mmand(s) can
on with Virtuall
Con
nnect
# Create
C
Server Profile ESX-1 with
w 6 NICs and
a 2 FCoE S
SAN connectio
ons
add
d profile ESX-1
1
set enet-connectio
e
on ESX-1 1 px
xe=Enabled Network=VLAN
N
N101-1 SpeeedType=Custo
om Speed-100
00
set enet-connectio
e
on ESX-1 2 px
xe=Disabled Network=VLA
N
AN101-2 SpeeedType=Custo
om Speed-1000
add
d enet-connecttion ESX-1 pxe
e=Disabled Network=VLAN
N
om Speed-150
00
add
e=Disabled Network=VLAN
N
om Speed-150
00
add
e=Disabled
add
d server-port-m
map ESX-1:5 VLAN_103-1
V
VLanId=103
V
add
d server-port-m
V
VLanId=104
V
add
d server-port-m
V
VLanId=105
V
add
e=Disabled
add
d server-port-m
V
VLanId=103
V
add
d server-port-m
V
VLanId=104
V
add
d server-port-m
V
VLanId=105
V
set fcoe-connectio
f
on ESX-1:1 Fa
abric=FCoE_A
A SpeedType=
=4Gb
set fcoe-connectio
f
on ESX-1:2 Fa
abric=FCoE_B
B SpeedType=
=4Gb
pow
weroff server 1
assiign profile ESX
X-1 enc0:2
Figu
ure 124 - Defiine a Server Pro
ofile (ESX-1)
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that

gned to the tw
LOM
M:2-b.

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
108
Figu
ofile (ESX-1)
Notte: The speed of the NIC and SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note
e that
the FCoE connections are assigned to the tw

wo SAN fabriics created ea
arlier and use ports LOM:1--b
and
d LOM:2-b.
Figu
ure 126 - Con
nfigure Custom NIC Speed - move
m
the slider left/right to co
onfigure NIC sp
peed.

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
109
Figu
ure 127 - Con
nfigure NIC Port 3 for Multiple
e Networks - VLLAN103-1 thro
ough VLAN105
5-1
Figu
ure 128 - Con
nfigure NIC Port 4 for multiple
e Networks - VLLAN102-2 thro
ough VLAN105-2

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
110
Figu
ure 129 - Servver Profile View
w Bay 1
Figu
ure 130 - By hovering over the
t Multiple Networks
N
pagee, a listing of th
he connected N
Networks is disp
played.

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
111
Review
active. We also create two FCoE SAN Fabrics.
Network Ports 3 and 4 were added, these NICs will be connected to Multiple Networks and each
NIC will then be configured for networks VLAN102-x through VLAN105-x. As these networks are
tagging, frames will be presented to the server with VLAN tags. NICs 3 and 4 will be connected to
the same vSwitch, which will provide load balancing across both NICs, as well as fail-over
capabilities. VLAN tagged frames for these networks will be forwarded to the vSwitch and then
passed on to the appropriate Virtual Machine.
Results vSphere Networking Examples

We have successfully configured FlexFabric with a Share Uplink Set, supporting 5 VLANs and
redundant SAN fabrics. We have created a server profile to connect to the various vNet with SIX
NICs and the SAN fabrics using the FCoE connections created within the profile.
The following graphics show an ESX 4.0 server with SIX FlexNICs configured, two at 1Gb (console),
two at 1.5Gb (VMotion) and two at 3.5Gb (Guests port groups). If we did not require SAN
connectivity on this server, the FCoE connections could be deleted and the server would then have 8
NIC ports available to the OS. In addition, if we did not want FCoE connectivity and instead wanted
to leverage iSCSI, we could delete the FCoE connected and re-create those connects as iSCSI
connections, with offload and optionally iSCSI boot.
Note: the BL465c G7 and BL685c G7 utilize a NC551i chipset, whereas the BL460c G7, BL620c
G6 and BL680c G7 utilizes an NC553i chipset. This scenario leverages a BL465c G7 server.
ure 131 ESX

X 4.0 Network Connections
Figu
Figu
ure 132 - ESX
X 4.0 networkin
ng - three vSwittches configureed. (Note the N
NIC speeds)

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
113
Figu
ure 133 - Con
nfiguring the vS
Switch for multiple port groupss / VLANs
Figu
ure 134 - VM
M1 configured fo
or VLAN 105
Figu
ure 135 VM
M1 on VLAN 10
05

d (LACP) - vSph
here - Ethernet and FCoE SAN
N
114
R
Results vSphere SAN Connectivity
Figu
ure 136 - Durring server boo
ot, you can see that Port 1 of thhe FlexHBA is cconnected to an MSA1000 SA
AN
LUN
N

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
115
Figu
ure 137 ESX
X storage config
guration, the Sh
hared Storage LUN is provideed through the FFCoE connectio
ons to
the SAN.
S
S
Summa
ary
V
Conne
SUS
S is connected
d with TWO active
a
uplinks; both SUS ca
an actively pa
ass traffic. We
e included a d
dual
path
onnectivity.
When VC profile ESX-1 is app
plied to the serrver in bay2 a
and is powereed up, it has o
one NIC conn
nected
through FlexFabriic module 1 (cconnected to VLAN_101- 1), the secon
nd NIC is con
nnected throug
gh
Flex
xFabric module 2 (connecte
ed to VLAN_101-2). Each N
NIC is configuured at 1Gb. These NICs are
connected to the console vSwittch. The seco
ond pair of N ICs are conneected to the se
econd vSwitch
h,
whicch is configured for VMotio
on and is connected to VLA
AN102-x throuugh NICs 3 a
and 5 which a
are
conffigured at 1.5
5Gb. The lastt pair of NICss 5 and 6, aree connected to
o the third vSw
witch, which iis
conffigured to sup
pport VLANs 103
1
through 105.
1
This hosst is also conf igured for FC
CoE based SAN
acce
ess and conne
ects to a SAN
N LUN to store
e the Guest VM
Ms. Each FCo
oE port is con
nfigured for 4G
Gb of
SAN
N bandwidth.
Add
ditional NICs could
c
ng the server o
off and addin
ng up
to a total of 6 NIC
NIC.
If ad
dditional or le
ess SAN band
adju
usted. If the FCoE
F
SAN connections are not required,, these could be deleted, in
n which two
add
ditional NIC ports would the
en be made available
a
to thhe host.
As additional
a
servers are adde
ed to the enclo
e the NICs for LAN and SA
AN fabrics as required and
d apply them tto the approp
priate
servver bays and power
p
the serrver on.

d (LACP) - vSph
here - Ethernet a
and FCoE SAN
N
116
Scenario 6 Tunneled VLANs and Shared

Uplink Set with Active/Active Uplinks - vSphere
Ethernet and FCoE SAN
Overview
This scenario will implement a Shared Uplink Set (SUS) to provide support for individual VLANs for
management connectivity and a Tunneled vNet to support many VLANs. In this scenario, the
upstream network switches connect a shared uplink set to one port on each FlexFabric module, a
second switch port connects to each FlexFabric module to support the large number of VLANs.
disadvantages. We will review the first option in this scenario.
modules.
Requirements
Scenario 6 Tunneled VLANs and Shared Uplink Set with Active/Active Uplinks - vSphere Ethernet and FCoE SAN 117
Figure 138 - Physical View; Shows two Ethernet

E
X5 and X6 on
n Module 1 to
o Ports
witch and two
n
switch
h. The uplinkss from each m
module will support differen
nce
VLA
ANs. The SAN
N fabrics are connected red
dundantly, wiith TWO uplin
nks per fabricc, from ports X
X1
and
d X2 on module 1 to Fabricc A and ports X1 and X2 to
o Fabric B.
Scenario 6 Tunneled VLA

ANs and Shared
d Uplink Set witth Active/Activve Uplinks - vSp
phere Ethernet a
and FCoE SAN
N
118
Figure 139 - Lo
ogical View; the
t server blade profile is cconfigured with SIX FlexNIC
C and 2 FlexH
HBAs.
NIC
nected to VLAN_102-x, which
are part of the Sh
hared Uplink Sets, VLAN-Trrunk-1 and VLLAN-Trunk-2 reespectively. IIn addition, N
NIC 5
and
d 6 are conne
ected to VLAN_Many_x, wh
hich are vNet Tunnels. All uuplinks are co
onnected at 10
0Gb,
to a network swittch, through Ports
P
X5 and X6
X on each Fl exFabric Mod
dule in Bays 1 and 2. The FCoE
SAN
N connectionss are connecte
ed through po
orts X1 and X2
2 on each FleexFabric modu
ule. SAN Fab
bric
FCo
oE_A connectss to the existin
ng SAN Fabriic A through p
ports X1 and X
X2 on Module
e 1 (Bay 1) and
FCo
oE_B connectss to the existin
ng SAN Fabricc B through p
ports X1 and X
X2 on Module
e 2 (Bay 2).
IInstalla
ation and
a configura
ation
S
Switch configur
c
ation
App
pendices A an
ds required to
o configure the
app
pendices assum
mes the follow
wing informatiion:
The switch ports

p
connecting to VLAN_TTrunk_x are co
onfigured as V
VLAN TRUNK
K/tagging porrts to
support VLA
ANs 101 and 102 for mana
agement and VMotion. All frames will b
be forwarding
g to
Virtual Conn
nect with VLAN
N tags.
The switch ports

p
connecte
ed to VLAN_M
Many_x are co
onfigured as V
VLAN Trunk/ttagging ports to
support VLA
ANs 2000 thro
ough 3000. All
A frames willl be forwarding to Virtual C
Connect with
VLAN tags. This will form
m the VLAN tu
unnel. When VLAN tunneliing is configured all traffic must
be tagged. By using a VLLAN tunnel in this fashion, a
additional VLA
ANs can be a
added to the
witch ports and
d will be pressented to Virtuual Connect trransparently, n
no changes to
o the
upstream sw
Virtual Conn
nect configura
ation would be
e required to present thesee additional VLANs to the server.

onal uplinks to
o a SUS or VLLAN Tunnel, i f the additional uplinks are
e
t same Flex
xFabric module to the samee switch, in ord
connecting from the
der to ensure all the uplinkks are
ed to be confiigured for LAC
CP within the same Link Ag
ggregation Grroup.
activve, the switch ports will nee
m
with redu
h Fabric. SAN
N switch portss
connecting to the
e FlexFabric module
m
must be

ANs and Shared
phere Ethernet a
and FCoE SAN
N
119
VC CLI commands
Many of the configuration settings within VC can also be accomplished via a CLI command set. In
order to connect to VC via a CLI, open an SSH connection to the IP address of the active VCM. Once
logged in, VC provides a CLI with help menus. Through this scenario the CLI commands to configure
VC for each setting will also be provided.
Physically connect Ports X1 and X2 on the FlexFabric module in Bay 1 to switch port in SAN
Fabric A
Physically connect Ports X1 and X2 on the FlexFabric module in Bay 2 to switch port in SAN
Fabric B

Physically connect Port 1 of Network switch 1 to Port X6 on the VC module in Bay 2, if you have
only one network switch, connect VC port 1 (Bay 2) to an alternate port on the same switch.
This will NOT create a network loop and Spanning Tree is not required

minute.
Configuring the VC Module for Tunnel VLAN Tags and Fast Mac Cache Failover via GUI (Ethernet settings)
Enable Tunnel VLAN Tags within Virtual Connect
Select Tunnel VLAN Tags (click apply)
Select apply
Select Apply
C
Configurin
ng the VC Module for
f Tunnel VLAN Ta
ags and Fa
ast Mac C
Cache Faillo
over via CLI
C (Ethernet settingss)
The following com
mmand(s) can
n be copied and
a pasted intto an SSH bassed CLI sessio
on with Virtual
Con
nnect
nced Ethernet Settings to "TTunnel VLAN TTags" and sett "Force serve
er connectionss" to
# Set Advan
disabled
set enet-vlan
n vlantagcontrrol=tunnel
set mac-cach
he Enabled=TTrue Refresh=5
5
o Max speed
only
s
we will
d Link
e.
or Maximum
M
Con
nnection spee
w for reference
Sele
alue
e
netw
works to 500M
2.5Gb.
# Se
ngs to a Prefe
d a Max Spee
ed of 2500Mb
b
set enet-vlan
e
PrefSpeedType=C
Custom PrefSp
peed=500 Ma
0
Figu
ure 140 - Ethe
ernet Settings Tunnel VLAN Tags
T
(under EEthernet Settings Advanced
d Settings)

ANs and Shared
phere Ethernet a
and FCoE SAN
N
121
Figure 141 - Set Fast MAC Cache (underr Ethernet Setttings Advancced Settings - Other)
D
Defining a new Sha
ared Uplin
nk Set (VLA
AN-Trunk-1)
Conne
ect Port X5 of FlexFabric module
m
1 to Po
ort 1 on switchh 1
Create a SUS name
ed VLAN-Tru
unk-1 and connect it to FleexFabric Port X
X5 on Module
e1
On the Virtua
al Connect Hom
me page, select Define, Shared
d Uplink Set
Add Networkks as follows (to

o add a networrk, right click onn the grey bar under the Asso
ociate Networkks
(VLAN) head
der, the select ADD;
A
Enable Smarttlink on ALL netw

works
Insert Uplink Set

S Name as VLAN-Trunk-1
V
Select Add Po
ort, then add th
he following port;
Enclosure 1, Ba
ay 1, Port X5
Optionally, if one of the VLA

ANs is configured as Default/uuntagged, on t hat VLAN only,, set Native to
Enabled
Click Apply
D
Defining a new Sha
ared Uplin
nk Set (VLA
AN-Trunk-2
2)
Conne
ect Port X5 of FlexFabric module
m
2 to Po
ort 1 on switchh 2
Create a SUS name
ed VLAN-Tru
unk-2 and connect it to FleexFabric Port X
X5 on Module
e2
On the Virtu
ual Connect Home
H
page, se
elect Define, S
Shared Uplinkk Set
Insert Uplinkk Set Name ass VLAN-Trunkk-2
Select Add Port,
P
then add
d the following
g port;
Enclosure 1, Ba
ay 2, Port X5
Add Networks as follows (to add a nettwork, right c lick on the greey bar under the Associatte
VLAN) heade
er, the select ADD;
A
Networks (V


n
Click Apply
V
is conffigured as Deefault/untaggeed, on that VLLAN only, set
Optionally, if one of the VLANs
nabled
Native to En

ANs and Shared
phere Ethernet a
and FCoE SAN
N
122
D
Defining tw
wo new Shared
S
Up
plink Sets via
v CLI
The fo
ollowing script can be used
d to create the
e first Shared U
Uplink Set (VLLAN-Trunk-1)
d CLI session with Virtual
The fo
ollowing comm
mand(s) can be
b copied and
d pasted into an SSH based
Conne
ect
# Create Shared
S
Uplink Set "VLAN-Trrunk-1" and cconfigure uplin
nks
add uplinkkset VLAN-Trun
nk-1
add uplinkkport enc0:1:X
X5 Uplinkset=VLAN-Trunk-1 speed=auto
# Create Networks
N
VLAN101-1 throu
ugh VLAN102
2-1 for Shared
d Uplink Set "VLAN-Trunk-1
1"
add netwo
ork VLAN101--1 uplinkset=V
VLAN-Trunk-1 VLanID=101
Set Network VLAN101-1 SmartLink=E
Enabled
add netwo
Enabled
The fo
ollowing script can be used
d to create the
e Second Sharred Uplink Seet (VLAN-Trunkk-2)
# Create Shared
S
Uplink Set "VLAN-Trrunk-2" and cconfigure uplin
nks
add uplinkkset VLAN-Trun
nk-2
add uplinkkport enc0:2:X
X5 Uplinkset=VLAN-Trunk-2
2 speed=auto
# Create Networks
N
VLAN101-2 throu
ugh VLAN102
2-2 for Shared
d Uplink Set "VLAN-Trunk-2
2"
add netwo
Enabled
add netwo
Enabled
Figu
ure 142 - Define a Shared Uplink Set (VLAN
N-Trunk-1) and add VLANs
Notte: The Conn

AC address an
nd port number of
the switch
s
available in all sw
ected to.

ANs and Shared
phere Ethernet a
and FCoE SAN
N
123
Figu
ure 143 - Actiive / Active Shared Uplink Se
et
Figu
ure 144 - Asso
ociated Networks tab
Sum
nario we have
each
orig
F
Mo
odules, by do
oing so we pro
separate and red
dundant conne
e create the se
erver
nnect to VLAN
Ns accessed th
odule,
which provides th
plink scenario
have
crea
wou
plink scenario
o, example beelow.
Figu
ure 145 - Example of an Acttive/Standby Shard Uplink Seet

ANs and Shared
phere Ethernet a
and FCoE SAN
N
124
Defining two new vNets via GUI

Create a vNet and name it VLAN-Many-1
Enter the Network Name of VLAN-Many-1
Select Smart Link

Select Enable VLAN Tunneling

Select Apply
Create a vNet and name it VLAN-Many-2
Enter the Network Name of VLAN-Many-2
Select Smart Link

Select Enable VLAN Tunneling

Select Apply
Note: By creating TWO vNets we have provided a redundant path to the network. As each uplink
originates from a different VC module within each vNet, both uplinks will be active. This
configuration provides the ability to lose an uplink cable, network switch or even a VC module.
Note: Smart Link In this configuration Smartlink SHOULD be enabled. Smartlink is used to turn off
downlink ports within Virtual Connect if ALL available uplinks to a vNet or SUS are down. In this
scenario if an upstream switch or all cables to a vNet were to fail, VC would turn off the downlink
ports connect to that vNet, which would then force the NIC Teaming software to fail-over to the
alternate NIC.
Defining two new vNets via CLI

Connect
# Create the vNet "VLAN-Many" and configure uplinks as discussed above
add Network VLAN-Many-1
add uplinkport enc0:1:X6 Network=VLAN-Many-1 speed=auto
set network VLAN-Many-1 SmartLink=Enabled VLanTunnel=Enabled
add Network VLAN-Many-2
add uplinkport enc0:2:X6 Network=VLAN-Many-2 speed=auto
set network VLAN-Many-2 SmartLink=Enabled VLanTunnel=Enabled
Figu
ure 146 - Deffine a vNet Tun
nnel (VLAN-Man
ny-1), VLANs w
will be tunneled
D
a GUI
Cre
eate a Fabricc and name iti FCoE_A
On the Virtu
ual Connect Manager
M
scree
he first Fabric
Ether the Ne
etwork Name of FCoE_A
Select Add Port,
P
then add
d the following
g ports;
Enclosure 1, Ba
ay 1, Port X1
Enclosure 1, Ba
ay 1, Port X2
Select Automattic Login Re-Distribution
Select Apply
y
Crea
ame it FCoE_
_B
On the Virtu
ual Connect Manager
M
scree
he second Fab
bric
Ether the Ne
etwork Name of FCoE_B
Select Add Port,
P
then add
d the following
g ports;
Enclosure 1, Ba
ay 2, Port X1
Enclosure 1, Ba
ay 2, Port X2
Select Automattic Login Re-Distribution
Select Apply
y
D
Defining SAN
S
Fabriics via CLI
The following com
mmand(s) can
on with Virtuall
Con
nnect
#Crreate the SAN
N Fabrics FCoE
Add
d fabric FCoE_
Add
d fabric FCoE_

ANs and Shared
phere Ethernet a
and FCoE SAN
N
126
Figu
ure 147 - SA
AN Login Disttribution Settin
ng
Figu
ure 148 FC
CoE SAN fab
brics configure
ed with a two 4Gb uplinks per fabric. N
Note the bay a
and
Figu
ure 149 FC
CoE SAN Serrver Connectio
ons (After servver profiles ha
ave been crea
ated, assigned
d and

ANs and Shared
phere Ethernet a
and FCoE SAN
N
127

We will create a server profile with two server NICs.
Each server NIC will connect to a specific network.
On the main menu, select Define, then Server Profile

Create a server profile called ESX-1
In the Network Port 1 drop down box, select VLAN101-1
Set the port speed to Custom at 1Gb
Set the port speed to Custom at 1Gb
Left click on either of Port 1 or Port 2 in the Ethernet Connections box, then right click and select
ADD network (add four additional network connections)
Leave the network speed as Auto
In the Network Port 6 drop down box, select VLAN-Many-2
Set the port speed to Custom at 1.5Gb
In the Network Port 5 drop down box, select VLAN-Many-1
Set the port speed to Custom at 1.5Gb
Leave the network speed as Auto
Expand the FCoE Connections box, for Bay 1, select FCoE_A for Bay 2, select FCoE_B
Do not configure FC SAN or iSCSI Connection
In the Assign Profile to Server Bay box, locate the Select Location drop down and select Bay 2,
they apply
Prior to applying the profile, ensure that the server in Bay 1 is currently OFF
Note: You should now have a server profile assigned to Bay 2, with 6 Server NIC connections.
NICs 1&2 should be connected to networks VLAN101-x, NICs 3&4 should be connected to
VLAN102-x (which will be used for VMotion) NICs 5&6 should be connected to networks VLANMany-x and FCoE SAN fabrics FCoE_A and FCoE_B.

Connect
# Create Server Profile ESX-1 with 6 NICs and 2 FCoE SAN connections
add profile ESX-1
set enet-connection ESX-1 1 pxe=Enabled Network=VLAN101-1 SpeedType=Custom Speed=1000
set enet-connection ESX-1 2 pxe=Disabled Network=VLAN101-2 SpeedType=Custom Speed=1000
add enet-connection ESX-1 pxe=Disabled Network=VLAN102-1 SpeedType=Custom Speed=1500
add enet-connection ESX-1 pxe=Disabled Network=VLAN102-2 SpeedType=Custom Speed=1500
add enet-connection ESX-1 pxe=Disabled Network=VLAN-Many-1 SpeedType=Auto
add enet-connection ESX-1 pxe=Disabled Network=VLAN-Many-2 SpeedType=Auto
set fcoe-connection ESX-1:1 Fabric=FCoE_A SpeedType=4Gb
set fcoe-connection ESX-1:2 Fabric=FCoE_B SpeedType=4Gb
poweroff server 1
assign profile ESX-1 enc0:2
Figu
ofile (ESX-1)
nd SAN conn
nections, as w
well as the MA
AC and WWN
N. Also, note that

gned to the tw
LOM
M:2-b. If 4Gb
b of FC bandw
width is not re
equired, the FC
CoE connectio
on speed can
n be reduced, that
extra
a bandwidth can then be applied
a
to one
e of, or distrib
buted across, the NIC conn
nections.
Figu
ure 151 - Con
nfigure Custom NIC Speed - move
m
the slider left/right to co
onfigure NIC sp
peed.

ANs and Shared
phere Ethernet a
and FCoE SAN
N
129
Figu
ure 152 - Servver Profile View
w Bay 2
R
Review
w
In th
his scenario we
w have create
ed Two Share
ed Uplink Setss (SUS), providing support for 2 VLANs.
Uplinks originatin
ng from the ea
ach FlexFabric Module connnect to each SUS, by doin
ng so we provvide
redu
undant conne
ections out of the
t Virtual Co
onnect domainn. In this scena
ario, all uplinks will be active.
We
e also create tw
wo FCoE SAN
N Fabrics. In addition, we created TWO
O vNets (redu
undant) Tunne
els to
support to connection of many
y VLANs to Fle
exFabric.
We
e created a server profile, with
w two NICss connected to
o the same VLA
AN, VLAN10
01-1 and
VLA
AN101-2, whiich provides the ability to sustain a link o
or module failure and not lose connectio
on to
the network. VLA
AN101-1 and VLAN101-2 are
a configureed to support V
VLAN 101, frrames will be
pressented to the NIC(s) withou
ut VLAN tags (untagged), thhese two NIC
Cs are connectted to the sam
me
VLA
AN, but taking
g a different path
p
out of the
e enclosure. TThis is also true for NICs 3 and 4, which
h
connect to VLAN 102 (VMotio
on) through VLLAN102-1 annd VLAN102-2
2.
Nettwork Ports 5 and 6 were added,
a
these NICs
N
will be cconnected to VLAN-Manyy-1 and VLAN-Many-2. Netwo
orks VLAN-Ma
any-x are connected to swittch ports conffigured for a llarge number of
VLA
AN. NICs 5 an
nd 6 will be connected
c
to the
t same vSw
witch, which w
will provide loa
ad balancing
acro
oss both NICss, as well as fail-over capab
bilities. VLAN
N tagged fram
mes for these n
networks will be
forw
warded to the vSwitch and then passed on
o to the app ropriate Virtu al Machine.

ANs and Shared
phere Ethernet a
and FCoE SAN
N
130
Add
ditionally, FCo
oE port 1 is co
onnected to SAN fabric FC
CoE_A and FC
CoE SAN portt is connected
d to
SAN
N Fabric FCoE_B, providing a multi-path
hed connected
d to the SAN.
The FCoE SAN fa
N fabric over a pair of upliinks per modu
ule. SAN logins
are distributed accross the multiple paths.
R
Results vSphere Netw
working Examplles
We have successsfully configured FlexFabricc with a Sharee Uplink Set, ssupporting ma
any VLANs an
nd
redu
undant SAN fabrics. We have
h
created a server profi le to connect to the variouss vNet with SIIX
NIC
Cs and the SAN fabrics usin
ng the FCoE connections
c
crreated within the profile.
Although both Eth
hernet and Fib
apter used the
e in
AN and SAN
of the adapter,
they
ndividual netw
work and SAN
N adapters.
The following gra
aphics show an
a ESX 4.0 se
erver with SIX FlexNICs con
nfigured, two at 1Gb (conssole),
two at 1.5Gb (VM
Motion) and two
t
at 3.5Gb (Guests port groups). If w
we did not req
quire SAN
connectivity on th
his server, the FCoE connecctions could b
be deleted and
d the server w
would then havve 8
NIC
C ports available to the OS. In addition,, if we did no t want FCoE cconnectivity a
and instead wanted
to le
everage iSCSI, we could de
elete the FCoE
E connected a
and re-create those conneccts as iSCSI
connections, with
h offload and optionally iSC
CSI boot.
685c G7 utiliz
60c G7, BL620c
G6 and BL680c G7
G utilizes an
n NC553i chiipset. This sceenario leverag
ges a BL465cc G7 server.
Figu
ure 153 ESX
X 4.0 Network Connections

ANs and Shared
phere Ethernet a
and FCoE SAN
N
131
Figu
ure 154 - ESX
X 4.0 networkin
ng - three vSwittches configureed. (Note the N
NIC speeds)
Notte: add port grroups for the ap
ppropriate VLANs

ANs and Shared
phere Ethernet a
and FCoE SAN
N
132
Figu
ure 155 - Con
nfiguring the vS
Switch for multiple port groupss / VLANs
Figu
ure 156 - VM
M1 configured fo
or VLAN 2793

ANs and Shared
phere Ethernet and FCoE SAN
N
133
Figu
ure 157 VM
M1 on VLAN 27
793
R
Results vSphere SAN Connectivity
Figu
ure 158 - Durring server boo
ot, you can see that Port 1 of thhe FlexHBA is cconnected to an MSA1000 SA
AN
LUN
N

ANs and Shared
phere Ethernet a
and FCoE SAN
N
134
Figu
ure 159 ESX
X storage config
guration, the Sh
hared Storage LUN is provideed through the FFCoE connectio
ons to
the SAN.
S
S
Summa
ary
V
Conne
SUS
S is connected
d with ONE acctive uplink; both
b
SUS cann actively passs traffic. In ad
ddition, we
crea
ated two vNett Tunnels, which will pass ta
agged framess. We also in
ncluded a dua
al path SAN ffabric
for storage
s
conne
ectivity.
When VC profile ESX-1 is app
plied to the serrver in bay2 a
and is powereed up, it has o
one NIC conn
nected
through FlexFabriic module 1 (cconnected to VLAN_101- 1), the secon
nd NIC is con
nnected throug
gh
Flex
xFabric module 2 (connecte
ed to VLAN_101-2). Each N
NIC is configuured at 1Gb. These NICs are
connected to the console vSwittch for manag
gement. The ssecond pair o
of NICs are co
onnected to th
he
seco
ond vSwitch, which
w
is confiigured for VM
Motion and is cconnected to VLAN102-x tthrough NICs 3
and 4 which are configured att 1.5Gb. The
e last pair of N
NICs 5 and 6, are connectted to the third
d
vSw
witch, which iss configured to
o support man
ny tagged VLA
ANs. As add
ditional VLANss are configurred
on the network sw
witch ports, Virtual connectt will simply p ass those VLA
ANs to the switch. A port g
group
wou
uld then be cre
eated to supp
port connectiviity to each VLA
LAN. This hosst is also configured for FCoE
base
ed SAN acce
ess and connects to a SAN LUN to store the Guest VM
Ms. Each FCo
oE port is
conffigured for 4G
Gb of SAN ba
andwidth.
Add
ditional NICs could
c
ng the server o
off and addin
ng up
to a total of 6 NIC
NIC.
If ad
dditional or le
ess SAN band
adju
usted. If the FCoE
F
SAN connections are not required,, these could be deleted, in
n which two
add
ditional NIC ports would the
en be made available
a
to thhe host.
As additional
a
servers are adde
ed to the enclo
e the NICs for LAN and SA
AN fabrics as required and
d apply them tto the approp
priate
servver bays and power
p
the serrver on.

ANs and Shared
phere Ethernet a
and FCoE SAN
N
135
Appendix A Scenario-based CISCO

Command Line Reference
All of the following commands in this appendix assume an unaltered factory default configuration
before execution of the switch commands.
Scenario 1 Cisco IOS command line configuration

(Simple vNet with Active/Standby Uplinks)
Connect to the Cisco switch servicing the VC-Enet uplink ports and enter the following IOS commands.
NOTE: If two switches are being used, issue the same commands on the second
switch.
Table 1 Cisco IOS command line configuration
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#config t
Configure via terminal
#interface gigabitethernet0/1
#int gi1/0/1
Focus on Port 1
#switchport mode access
#sw mo ac
Set port 1 for Single VLAN mode
#switchport access vlan 1
#sw ac vl 1
Allow Port 1 access to VLAN 1
#spanning-tree portfast trunk
#sp portf tr
Enable portfast on Port 1
#exit
#exit
Remove focus from Port 1
#show vlan brief
#sh vl br
Display all VLANs
#show interface gigabitethernet0/1 #sh int gi0/1 status

status
Display the status of Port 1
#copy running-config startup-config
Save the running configuration to

NVRAM. Otherwise, the changes
will be lost on the next reboot.
(For permanent changes only)
#cop ru st
Appendix A Scenario-based CISCO Command Line Reference 136
Scenarios 2 through 5 - Cisco IOS command line

configuration (Shared Uplink Set with VLAN tagging
and 802.3ad (LACP))
switch.
Table 2 Cisco IOS command line configuration (802.1Q, 802.3ad)
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#config t
#int gi0/1
Focus on Port 1
#switchport trunk allowed vlan 101- #sw tr ac vl 101-105

105
Configure port for VLANs 101

through 105
#switchport mode trunk
#sw mo tr
Enable trunking on Port 1
#channel-protocol lacp
#channel-p l
Configure port 20 for 802.3ad

LACP
#channel-group 10 mode active
#channel-g 10 mo ac
Enable channel group 10
#sp portf tr
#exit
#ex
#int gi0/2
Focus on Port 2

105
Configure Port 2 for VLANs 101

through 105
#sw mo tr
#channel-protocol lacp
#channel-p l
Configure Port 21 for 802.3ad

LACP
#channel-group 10 mode active
#channel-g 10 mo ac
Enable channel group 10
#sp portf tr
#exit
#ex
#show lacp 10 internal
#sh la 10 i
Show the LACP group 10

configuration
#show etherchannel summary
#sh eth sum
Show the etherchannel configuration
#show interface port-channel10

trunk
#sh int port-channel 10 tr
Show the port channel 10 trunk

configuration
#cop ru st

Scenarios 6 - Cisco IOS command line configuration

(Shared Uplink Set with VLAN tagging)
switch.
Table 3 Cisco IOS command line configuration (802.1Q)
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#config t
#int gi0/1
Focus on Port 1

102
Configure port for VLANs 101

through 102
#sw mo tr
#sp portf tr
#exit
#ex
#int gi0/2
Focus on Port 2
#switchport trunk allowed vlan

2102,2113,2507,2793
#sw tr ac vl
2102,2113,2507,2973
Configure port for VLANs

2102,2113,2507,2793
#sw mo tr
#sp portf tr
#exit
#ex
#cop ru st

Appendix B Scenario-based ProCurve

Command Line Reference
Scenario ProCurve command line configuration
(Simple vNet with Active/Standby Uplinks)
Connect to the ProCurve switch servicing the VC-Enet uplink ports and enter the following commands.
switch.
Table 4 ProCurve command line configuration (simple network)
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#conf
Configure in global mode
#span
#span
Enables spanning-tree (MSTP mode

by default) Not required for Virtuakl
#vlan 1 untagged [Ethernet] 1
#vlan 1 untag 1
Allow VLAN 1 on Port 1 , and set

Port 1 to untagged mode
#spanning-tree 1 admin-edge-port
#span 1 admin-edge
Set Port 1 to be an edge port (non

bridging port). Note: port is set by
default in auto-edge mode which
automatically sets port to Edge if no
BPDU are received after 3 sec.
#show interface brief 1
#sh int br 1
Display the status of Port 1
#show vlan ports 1 detail
# show vlan ports 1 detail
Displays the VLAN detail for Port 1
#show vlan 1
#sh vlan 1
Display VLAN 1 port information
#write memory
#write mem

Appendix B Scenario-based ProCurve Command Line Reference
139
Scenarios 2 through 5 ProCurve command line

configuration (Shared Uplink Set with VLAN tagging
and 802.3ad (LACP)
switch.
Table 5 ProCurve command line configuration (tagged VLANs)
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#conf
#span
#span

by default)
#trunk 1-2 trk1 lacp
#trunk 1-2 trk1 lacp
Configure LACP port-trunk 1 to

include Ports 1 & 2
#vlan 101 tagged trk1
#vlan 101 tag trk1
Allow VLAN 101 on Ports 1 and 2.

Trk1 (tagged)
#vlan 102 tag trk1

Trk1 (tagged)
#vlan 103 tag trk1

Trk1 (tagged)
#vlan 104 tag trk1

Trk1 (tagged)
#vlan 105 tag trk1

Trk1 (tagged)
#spanning-tree ethernet trk1 adminedge-port
#span e trk1 admin-edge
Set port 1 to be an edge port (non

bridging port). Note: port is set by
default in auto-edge mode which
automatically sets port to Edge if no
BPDU are received after 3 sec.
#show vlan 101
#sh vlan 101
Display VLAN 101
#show vlan 102
#sh vlan 102
Display VLAN 102
#show vlan 103
#sh vlan 103
Display VLAN 103
#show vlan 104
#sh vlan 104
Display VLAN 104
#show vlan ports trk1 detail
# show vlan ports trk1 detail
Displays the VLAN detail for Trunk 1
#show interface brief1-2
#sh int br1-2
Show Port 1-2 status
#write memory
#write mem

140
Scenario 6 ProCurve command line configuration

(Tunneled vNet and Shared Uplink Set with VLAN
tagging)
switch.
Table 6 ProCurve command line configuration (802.1Q)
Command
Shortcut
Description
>enable
>en
Privilege mode
#configure terminal
#conf
#span
#span

by default)
#vlan 101 tagged 1
#vlan 101 tag 1
Allow VLAN 101 on Port 1 and set

to tagged mode
#vlan 102 tagged 1
#vlan 102 tag 1
Add VLAN 102 on Port 1 and set to

tagged mode
#vlan 103 tagged 2
#vlan 103 tag 2
Allow VLAN 103 on Port 2 and set

to tagged mode
#vlan 104 tagged 2
#vlan 104 tag 2

tagged mode
#vlan 105 tagged 2
#vlan 105 tag 2

tagged mode
#vlan 1
#vlan 1
Set focus to VLAN 1
#no untagged 1
#no untagged 1
Disables VLAN 1 on Ports 1
#vlan 2
#vlan 2
Set focus to VLAN 2
#no untagged 2
#no untagged 2
Disables VLAN 1 on Ports 2
#exit
#exit
Exit VLAN 2
#write memory
#write mem

141
Appendix C: Acronyms and abbreviations

Term
Definition
Auto Port Speed**
Let VC automatically determine best Flex NIC speed
CLP String
Flex-10 NIC settings written to the server hardware by VC/OA when the server is
power off. Read by the server hardware upon power in.
Custom Port Speed**
Manually set Flex NIC speed (up to Maximum value defined)
DCC**
Dynamic Control Channel. Future method for VC to change Flex-10 NIC port settings
on the fly (without power no/off)
EtherChannel*
A Cisco proprietary technology that combines multiple NIC or switch ports for greater
bandwidth, load balancing, and redundancy. The technology allows for bi-directional
aggregated network traffic flow.
Flex NIC**
One of four virtual NIC partitions available per Flex-10 Nic port. Each capable of
being tuned from 100Mb to 10Gb
Flex-10 Nic Port**
A physical 10Gb port that is capable of being partitioned into 4 Flex NICs
Flex HBA***
Physical function 2 or a FlexFabric CNA can act as eitheran Ethernet NIC, FCoE
connection or iSCSI NIC with boot and iSCSI offload capabilities.
IEEE 802.1Q
An industry standard protocol that enables multiple virtual networks to run on a single
link/port in a secure fashion through the use of VLAN tagging.
IEEE 802.3ad
An industry standard protocol that allows multiple links/ports to run in parallel,

providing a virtual single link/port. The protocol provides greater bandwidth, load
balancing, and redundancy.
LACP
Link Aggregation Control Protocol (see IEEE802.3ad)
LOM
LAN-on-Motherboard. Embedded network adapter on the system board
Maximum Link Connection

Speed**
Maximum Flex NIC speed value assigned to vNet by the network administrator. Can
NOT be manually overridden on the server profile.
Multiple Networks Link Speed

Settings**
Global Preferred and Maximum Flex NIC speed values that override defined vNet
values when multiple vNets are assigned to the same Flex NIC
MZ1 or MEZZ1; LOM
Mezzanine Slot 1; (LOM) Lan Motherbard/systemboard NIC
Network Teaming Software
A software that runs on a host, allowing multiple network interface ports to be

combined to act as a single virtual port. The software provides greater bandwidth, load
balancing, and redundancy.
pNIC**
Physical NIC port. A Flex NIC is seen by VMware as a pNIC
Port Aggregation
Combining ports to provide one or more of the following benefits: greater bandwidth,
load balancing, and redundancy.
Port Aggregation Protocol (PAgP)*
A Cisco proprietary protocol aids in the automatic creation of Fast EtherChannel links.
PAgP packets are sent between Fast EtherChannel-capable ports to negotiate the
forming of a channel.
Port Bonding
A term typically used in the Unix/Linux world that is synonymous to NIC teaming in the
Windows world.
Preferred Link Connection Speed**
Preferred Flex NIC speed value assigned to a vNet by the network administrator.
Appendix C: Acronyms and abbreviations 142
Share Uplink Set (SUS)
A set of Ethernet uplinks that are used together to provide improved throughput and
availability to a group of associated Virtual Connect networks. Each associated Virtual
Connect network is mapped to a specific VLAN on the external connection and
appropriate VLAN tags are removed or added as Ethernet packets enter or leave the
Virtual Connect domain.
Smart Link
A feature that, when enabled, configures a Virtual Connect network so that if all
external uplinks lose link to external switches, Virtual Connect will drop the Ethernet link
on all local server blade Ethernet ports connected to that network.
Trunking (Cisco)
802.1Q VLAN tagging
Trunking (Industry)
Trunking (Industry)
VLAN
A virtual network within a physical network.
VLAN Tagging
Tagging/marking an Ethernet frame with an identity number representing a virtual

network.
VLAN Trunking Protocol (VTP)*
A Cisco proprietary protocol used for configuring and administering VLANs on Cisco
network devices.
vNIC
Virtual NIC port. A software-based NIC used by Virtualization Managers
vNet
Virtual Connect Network used to connect server NICs to the external Network
*The feature is not supported by Virtual Connect

**This feature was added for Virtual Connect Flex-10
***This feature was added for Virtual Connect FlexFabric
Appendix C: Acronyms and abbreviations 143
Appendix D: Useful VC CLI Command sets

The following are a collection of useful VC CLI commands. These CLI commands and many more are
documented in detail in Virtual Connect Manager Command Line Interface Version 1.31 (or later) User
Guide. The following CLI commands can be copied and pasted into an SSH session with the VCM and
will apply immediately upon paste.
VC Domain Configuration
#Enclosure Setup
#Import Enclosure and Set Domain Name
#Ensure password matches the OA password
import enclosure username=Administrator password=Administrator
set domain name=VC_Domain_1
#Importing additional or multiple Enclosures to an existing VC Domain
# Importing an Enclosure into an existing VC Domain (Note: As of this writing (VC firmware 2.30) the
following commands must be executed individually and cannot be part of a larger script).
#The IP address, login and password information used in this command are from the OA of the enclosure
being imported.
Import enclosure 10.0.0.60 UserName=Administrator Password=password
#Configure MAC and WWN to VC Defined and select pool #1
set domain mactype=vc-defined macpool=1
set domain wwntype=vc-defined wwnpool=1
set serverid type=vc-defined poolid=1
#Change Administrator
set user Administrator password=Administrator
# Set Advanced Ethernet Settings to "Map VLAN Tags" and set "Force server connections" to disabled
# Set Advanced Ethernet Settings to "Tunnel VLAN Tags"
set enet-vlan vlantagcontrol=tunnel
# Set Advanced Ethernet Settings to a Preferred speed of 500Mb and a Max Speed of 2500Mb
set enet-vlan PrefSpeedType=Custom PrefSpeed=500 MaxSpeedType=Custom MaxSpeed=2500
#Add additional User to VCM, creates User jimbo
add user jimbo password=password privileges=domain,network,server,storage
Creating Shared Uplink Sets
#Create Shared Uplink Set "Prod-Net" and configure one uplink VC module 1, port X1
add uplinkport enc0:1:X1 Uplinkset=Prod-Net speed=auto
#Create Shared Uplink Set "Prod-Net" and configure multiple uplinks on VC Module 1, Ports X1 X3
add uplinkset Prod-Net
Appendix D: Useful VC CLI Command sets 144

# Create Networks PROD-A through PROD-D, supporting VLANs 101 through 104 on Shared Uplink Set
"Prod-Net"
add network VLAN_101 uplinkset=Prod-Net VLanID=101
# (optionally) Set network VLAN_104 as a Private Network
set network VLAN_104 Private=Enabled
Creating vNets
#Create vNet "Prod-Net" and configure uplinks
add Network Prod-Net
add uplinkport enc0:1:X3 Network=Prod-Net speed=auto
#Optionally enable the vNet as a Private Network
set network Prod-Net Private=Enabled
Server Profiles
#Create Server Profile App-1, apply this profile to Server Slot 1 and configure NIC 1 to Multiple
Networks VLAN_101 and VLAN_102
add profile App-1 -nodefaultenetconn
add enet-connection App-1 pxe=Enabled
add enet-connection App-1 pxe=Disabled
add server-port-map App-1:1 VLAN_101 VLanId=101
add server-port-map App-1:1 VLAN_102 VLanId=102
# As an alternative when connection to Multiple Networks, if you want ALL networks
# configured on a specific Shared Uplink Set to be presented to a server NIC, ensure that
# the Force VLAN mappings as Shared Uplink Set check box is enabled.
# Shared Uplink Set, use the following commands to do so
# This will set the Force same VLAN mappings as Shared Uplink Sets check box to enabled
# Result is that only VLANs from this shared uplink will be available to this NIC
add server-port-map App-1:1 VLAN_101 Uplinkset=Prod-Net
add server-port-map App-1:1 VLAN_102 Uplinkset=Prod-Net
# Create Server Profile App-1 Both NICs are configured on network VLAN_102
add profile App-1 -nodefaultenetconn
add enet-connection App-1 pxe=Enabled
add enet-connection App-1 pxe=Disabled
set enet-connection App-1 1 Network=VLAN_102
set enet-connection App-1 2 Network=VLAN_102
# Create Server Profile ESX-1 Both NICs are configured on both networks VLAN_102 and VLAN_102
add profile ESX-1 -nodefaultenetconn
add enet-connection ESX-1 pxe=Enabled
add enet-connection ESX-1 pxe=Disabled
add server-port-map ESX-1:1 VLAN_101 VLanId=101

# Create Server Profile Server-1 with 8 Flex-10 NICs configured for specific speeds
add profile Server-1 -nodefaultenetconn
add enet-connection Server-1 pxe=Enabled Network=Console-101-1 SpeedType=Custom Speed=500
add enet-connection Server-1 pxe=Disabled Network=Console-101-2 SpeedType=Custom Speed=500
add enet-connection Server-1 pxe=Disabled Network=VMotion-102-1 SpeedType=Custom Speed=2500
add enet-connection Server-1 pxe=Disabled Network=VMotion-102-2 SpeedType=Custom Speed=2500
add enet-connection Server-1 pxe=Disabled Network=Prod-103-1 SpeedType=Custom Speed=2000
add enet-connection Server-1 pxe=Disabled Network=Prod-103-2 SpeedType=Custom Speed=2000
add enet-connection Server-1 pxe=Disabled
add server-port-map Server-1:7 Prod-104-1 VLanId=104
add enet-connection Server-1 pxe=Disabled
Assign profile Server-1 enc0:1
# Add TWO fc connections to Profile ESX-1 with a specific WWN
add fc-connection ESX-1 Fabric=SAN_3 AddressType=User-Defined
PortWWN=50:06:0B:00:00:C2:ff:00 NodeWWN=50:06:0B:00:00:c2:ff:01
add fc-connection ESX-1 Fabric=SAN_4 AddressType=User-Defined
PortWWN=50:06:0B:00:00:C2:ff:02 NodeWWN=50:06:0B:00:00:C2:ff:03
# Add TWO NIC connections to Profile ESX-1 with a specific MAC and iSCSI MAC address
add enet-connection ESX-1 AddressType=User-Defined EthernetMac=00-17-00-00-AA-AA IScsiMac=0017-00-00-BB-BB pxe=Enabled
add enet-connection ESX-1 AddressType=User-Defined EthernetMac=00-17-00-00-AA-CC IScsiMac=0017-00-00-BB-CC pxe=Disabled
FlexFabric additions
#Create the FCoE SAN Fabrics FCoE_A and FCoE_B and configure uplinks as discussed above
Add fabric FCoE_A Bay=1 Ports=1,2 LinkDist=Auto
Add fabric FCoE_B Bay=2 Ports=1,2 LinkDist=Auto
# Create Server Profile ESX-1 with 6 NICs and 2 FCoE SAN connections
add profile ESX-1
set enet-connection ESX-1 1 pxe=Enabled Network=VLAN101-1 SpeedType=Custom Speed-1000
set enet-connection ESX-1 2 pxe=Disabled Network=VLAN101-2 SpeedType=Custom Speed-1000
add enet-connection ESX-1 pxe=Disabled Network=VMotion SpeedType=Custom Speed-1500
add enet-connection ESX-1 pxe=Disabled Network=VMotion SpeedType=Custom Speed-1500
add server-port-map ESX-1:5 VLAN_103-1 VLanId=103
set fcoe-connection ESX-1:1 Fabric=FCoE_A SpeedType=4Gb
set fcoe-connection ESX-1:2 Fabric=FCoE_B SpeedType=4Gb
poweroff server 1

Cookbook

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Cookbook

Загружено:

Авторское право:

Доступные форматы

V

Virtual Connect FlexFab

PPart number c02

Appendix A Scenario-based CISCO Command Line Reference ............................................................. 136

Introduction to Virtual Connect FlexFabric

Additional Virtual Connect Reference Material

Introduction to Virtual Connect FlexFabric 6

Virtual Connect Ethernet Cookbook

Virtual Connect Fibre Channel Cookbook

Virtual Connect 3.15 Firmware Release

Link Init/Fill Word

If ARBF/ARBF fails use IDLE/ARBF

Introduction to Virtual Connect FlexFabric 7

Virtual connect FlexFabric Uplink Port Mappings

Ports X5 X8: Can be configured as 1Gb or 10Gb Ethernet

Introduction to Virtual Connect FlexFabric 8

Figure 1 - FlexFabric Module

Note: Even thou

Tunneled VLAN and Mapped VLANS

Tunnel vs. Map VLAN tags setting

Tunneled VLAN and Mapped VLANS

Shared Uplink Set (SUS)

Select a single network which would be mapped to a specific VLAN.

Select multiple networks which would provide connectivity to several VLANs.

Tunneled VLAN and Mapped VLANS

Virtual Connect VLAN Support SUS - Restrictions and limitations

Tunneled VLAN and Mapped VLANS

Single Domain/Enclosure Scenario

Scenario 1 Simple vNet with Active/Standby

Scenario 1 Simple vNet with

Figure 5 - Logical View; Shows

The switch ports

This scenario assumes the switch port is con

Configuring the VC module

Physically connect Port 1 of Network switch 1 to Port X5 on the VC module in Bay 1.

Configuring Fast MAC Cache Failover

Scenario 1 Simple vNet with

Defining a new vNet via GUI

Select Add Port, then add the following ports;

Enclosure 1, Bay 1, Port X5

Enclosure 1, Bay 2, Port X5

Leave Connection Mode as Auto

Defining a new vNet via CLI

Enclosure 1, Bay 1, Port X1

Enclosure 1, Bay 2, Port X1

Scenario 1 Simple vNet with

Create a serrver profile ca

Scenario 1 Simple vNet with

Defining a Server Profile with

the FCoE connecttions are assig

Scenario 1 Simple vNet with

The following gra

Notte: The NICs that

Note: In windowss 2003 the NIC

Scenario 1 Simple vNet with

ure 17 - Windows 2008 R2, Device Manager,

The following gra

Scenario 1 Simple vNet with

Scenario 1 Simple vNet with

Scenario 1 Simple vNet with

Scenario 1 Simple vNet with

Scenario 2 Shared Uplink Sets with

The switch ports

Note: when adding additionall uplinks to the