Demartek February 2013

IBM Emulex 16Gb Fibre Channel HBA

Evaluation
Evaluation report prepared under contract with Emulex

Executive Summary
The computing industry is experiencing an increasing demand for storage performance and
bandwidth due to increases in virtual machine density, increasing demands for application
performance and continual data growth. Fibre Channel storage area networks (SANs) carry the
bulk of storage traffic in the enterprise data center and are beginning to feel the stresses of these
increased demands.

In many cases, enterprises are currently constrained by the available bandwidth between the servers
and storage, or foresee a constraint as they observe their growing data consumption patterns. The
IBM Emulex 16Gb Fibre Channel (16GFC) host bus adapter (HBA) addresses these increasing
demands on storage performance by providing double the bandwidth of previous generation Fibre
Channel HBAs.

Demartek deployed an IBM System x3650 M4 server with the IBM Emulex 16GFC HBA
(81Y1662) and connected this server to an all-flash storage array with four 8GFC host ports in the
Demartek lab in Colorado. We ran a read-intensive data warehousing workload to determine if this
type of workload could take advantage of the increased bandwidth and performance that 16GFC
provides. We repeated the database workload test with previous generation IBM Emulex 8GFC
HBA and compared the results.
Key Findings
We found that for this database workload, the 16GFC HBA exceeded the performance of the
8GFC HBA and provided the additional bandwidth needed by the database workload allowing the
job to be completed in less time.

The 16GFC HBA:

Completed the real database workload 33% faster than the 8GFC HBA
Allows for instantaneous doubling of throughput when needed, providing peak workload
headroom
Incorporates a simple, plug-n-play performance upgrade for the replacement of older 4GFC
and 8GFC HBAs

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 2 of 8

The Need for More Bandwidth

Todays data centers face a variety of challenges brought by seemingly insatiable demands on server
and storage infrastructure. At the same time, new technologies are being introduced that offer both
challenges and possible solutions to meet these growing challenges.

VM Density
When Demartek presents to users about next-generation storage networking technologies at various
industry events, we usually ask the audience of primarily technical users and first-line managers a
few questions about their environments. Among the responses are that virtual machine (VM)
density has been increasing over the last few years, with higher numbers of guest operating systems
running on one physical server than in the past. We expect this trend to continue.

8GFC Saturation
During the year 2012, when we asked the end-users in our audiences about saturation of Fibre
Channel links, we consistently heard from a few users who indicated that they had saturated their
8GFC links and needed something faster. The applications consistently identified as needing this
higher bandwidth are database applications, regardless of the brand of database. These include
single database instances running on physical hardware, multiple database instances running on
physical hardware and multiple database instances running in VMs. These users are generally
looking for something compatible with their existing infrastructure but that provides higher
bandwidth to meet their growing demands.

SSD
Solid State Disk (SSD) technology is another driver of bandwidth growth. Although relatively early
in the deployment cycles, we have found that those who deploy any form of SSD technology in the
enterprise experienced significant storage performance improvements. Many of these SSD
deployments are in SAN environments, which drive up storage networking bandwidth
consumption. Based on comments from users and many of the tests we have performed in our own
lab, we concluded that SSD technology and faster storage networking technology such as 16GFC
are well suited for each other.

New IBM Servers

In March 2012, IBM introduced its M4 servers that support the newest Intel Xeon E5-2600
processors (Romley) and PCI Express (PCIe) 3.0. In May 2012, IBM extended this server family
with the introduction of servers with the Intel Xeon E5-4600 processors. These servers provide
not only higher numbers of cores and performance improvements in processor power but also
provide significant increases in I/O throughput. PCIe 3.0 doubles the maximum possible I/O rates
and processors that support PCIe 3.0 support approximately double the number of PCIe lanes
available to each processor. As a result, the total I/O bandwidth available in one of these new
servers is approximately quadruple that of the previous generation of servers.

Windows Server 2012 and Hyper-V

Windows Server 2012 with Hyper-V was recently released, and it also addresses growing demands
on computing infrastructure. Windows Server 2012 Hyper-V supports 320 logical processors and 4

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 3 of 8

TB of physical memory. It supports 64 virtual processors, along with 1 TB of memory per VM. This
enables virtualization environments not previously possible. When coupled with todays newer
server hardware environments and new technologies such as 16GFC, much heavier workloads can
be supported.

A new feature for Windows Server 2012 Hyper-V is the support for virtual Fibre Channel, also
known as Synthetic FC. This allows guest VMs to connect directly to Fibre Channel storage LUNs,
allowing guests to take advantage of existing Fibre Channel infrastructure. This includes the ability
for guest operating systems to be clustered over Fibre Channel. In order to take advantage of this
feature, newer Fibre Channel HBAs that support virtual Fibre Channel are required. The IBM
Emulex 16GFC HBA supports this feature and provides up to four virtual Fibre Channel ports per
VM. Also required for virtual Fibre Channel is NPIV in the switch and HBA, which the IBM
Emulex Fibre Channel HBA supports. Hyper-V in Windows Server 2012 supports the use of multi-
path I/O (MPIO) and virtual SANs, both of which are also supported by the IBM Emulex 16GFC
HBA.

Bandwidth Growth Summary

When using storage intensive applications like backup/restore, database transactions, virtualization
and rich media, there is clearly a need for higher storage networking bandwidth and performance.
The improved I/O performance of 16GFC enables faster storage and retrieval of data. For those
enterprises that dont believe that they need this higher performance yet, now is the time to start
planning for these eventualities.

When we discuss storage networks with enterprise users, we find that Fibre Channel is still the
dominant storage interface in large-scale data centers, and is expected to remain dominant as a
SAN interface for the foreseeable future.

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 4 of 8

IBM Emulex Series

Technology advances including the
IBM Emulex series of 16GFC HBAs
can help address these growing
constraints on current server and
storage infrastructure.

Whether the environment includes

mission critical stand-alone database
workloads or increasing numbers of
VMs running on a single server,
IBM Emulex 16GFC HBAs enable
higher workloads and more
applications and VMs to run on a
single server and port, resulting in reduced cabling and higher return on IT investment.

The IBM Emulex series adapters provide several features designed for supporting enterprise I/O
workloads:
Twice the performance of 8GFC adapters
Backward compatible with 4GFC and 8GFC infrastructure
Support for Windows Server 2008 and 2012 with and without Hyper-V, VMware ESX and
ESXi, Red Hat Linux, SUSE Linux Enterprise Server (SLES)
In-box drivers for Windows Server 2012 and VMware vSphere 5.1
N_Port ID Virtualization (NPIV) support standard
An IBM-branded solution which has undergone extensive IBM interoperability testing for
connecting System x servers into storage and networking environments

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 5 of 8

Test Description and Environment

Demartek ran a read-intensive data warehouse workload in the Demartek lab in Colorado with the
configuration shown below. This test was run with the 16GFC HBA running at 16Gb/sec. The test
was repeated with the same HBA running at 8 Gb/sec and then repeated with the IBM Emulex
8GFC HBA.

Server IBM x3650 M4

2x Intel Xeon E5-2690, 2.9GHz, 16 total cores, 32 logical processors
32GB RAM
Host firmware 1.10, 6/20/2012
Boot drive: IBM 500GB 10K RPM SAS HDD
Microsoft Windows Server 2012

Fibre Channel HBA

IBM Emulex 16GFC dual-port HBA (81Y1662)
IBM Emulex 8GFC dual-port HBA (42D0494)

Fibre Channel Switch

IBM SAN48B-5 16GFC Switch

Storage Array
Nimbus Data S-Class, 4x 8GFC host ports
24x 100GB 6Gb SAS SSD, configured as RAID0

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 6 of 8

Data Warehouse Workload and Performance Results

Database Workload
This read-intensive database workload consists of a suite of business oriented ad-hoc queries and
concurrent data modifications. The queries and the data populating the database have been chosen
to have broad industry-wide relevance. This benchmark illustrates decision support systems that
examine large volumes of data, execute queries with a high degree of complexity, and give answers
to critical business questions. This particular workload consists of 22 different queries, each
exercising a different area of the database. The workload runs the same queries in the same order
for each run, so a lower elapsed time for each query indicates a faster system.

Real vs. Synthetic Workload

This workload uses a real database (Microsoft SQL Server) with database tables, indexes, etc., and
performs actual database transactions. As a result, the I/O rate varies as the workload progresses
because the database performs operations that consume varying amounts of CPU and memory
resources, in addition to I/O resources. These results more closely resemble a real customer
environment.

This is unlike a synthetic benchmark that performs the same I/O operations repeatedly resulting in
relatively steady I/O rates which although potentially faster, do not resemble real customer
environments.

Hardware and Software Specifications

The specifications for the software used for this test are listed below. The RAM allocated to the
database application was limited to 4GB in order to force the I/O through the Fibre Channel HBA
and out to the storage system, with minimal host memory caching.

Database
Windows Server 2012
Microsoft SQL Server 2008 R2
RAM allocated to SQL Server: 4GB
Database size: 30GB
Total size of all database files: 54GB

The server was rebooted between runs to clear any host memory caching.

Bandwidth Results
For this set of tests, we used a single host connection to the SAN. This allowed us to make a simple
comparison of the 16GFC adapter with the previous generation 8GFC adapter.

This data warehouse workload was able to achieve more than 8 Gb/sec of bandwidth for some of
the queries, and in some cases, nearly line-rate with the 16GFC adapter. For those queries that
show a flat top on the graph below using the 8GFC adapter, this indicates that more
performance is available, but the 8GFC adapter is throttling the performance. Also note that when
running this test with the 16GFC adapter, the time to complete the run was 67% of the time
required by the 8GFC adapter, or 33% faster.

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 7 of 8

MB/s per Server and Adapter 16 Gb Workload

Completed 33%
1800 faster than 8 Gb
1600
1400 Flat tops indicate
1200 throttling by 8GFC
adapter
MB/s

1000
800
600
400
200
0

438
115
134
153
172
191
210
229
248
267
286
305
324
343
362
381
400
419

457
476
495
514
533
552
571
1

39
20

58
77
96

Workload Duration (seconds)

IBM Emulex 16Gb IBM Emulex 16Gb @ 8Gb IBM Emulex 8Gb

Time to Completion All Queries

The following chart shows the aggregate time to complete each of the 22 queries, with each of the
adapters. Tests were run with the IBM Emulex 16GFC HBA and then repeated with the IBM
Emulex 8GFC HBA. For this workload, on average, the 16GFC adapter completed its workload in
67% of the time required by the 8GFC adapters, or 33% faster.

Full Query Sets Time to Execute

(Single FC HBA port: 16 Gb vs. 8 Gb)
700
16GFC adapter
600 completed real database
workload 33% faster than
500 the 8GFC adapter
Seconds

400

300 575
538
200 385

100

0
IBM Emulex 16Gb IBM Emulex 16Gb @ 8Gb IBM Emulex 8Gb

2013 Demartek www.demartek.com Email: info@demartek.com

 IBM Emulex 16GFC HBA Evaluation

Demartek February 2013

Page 8 of 8

Summary and Conclusion

With the availability of IBM Emulex 16GFC PCIe 3.0 Fibre Channel HBA, environments with
growing performance requirements such as virtualization servers, database applications, SSDs, flash
caching and more have an excellent choice in Fibre Channel HBA.

These database workload test results using the IBM Emulex 16GFC HBA show approximately 33%
better performance than the same server and storage using 8GFC HBAs. Testing revealed that
8GFC HBAs throttle the performance and cause the application to run longer than necessary. IBM
Emulex 16GFC HBAs enable the doubling of throughput when needed, alleviating bottlenecks
under peak workload scenarios.

16GFC provides the performance horsepower for both new environments and existing
environments that demand higher performance than are available today with older technologies.
For existing environments with 4GFC or 8GFC HBAs, installing IBM Emulex HBAs provides a
simple plug-and-play performance upgrade.

Emulex is a registered trademark of Emulex Corporation.

IBM and System X are trademarks of IBM.
Intel and Xeon are registered trademarks of Intel Corporation.
Microsoft, Windows Server and Hyper-V are registered trademarks of Microsoft.
VMware, ESX and ESXi are registered trademarks of VMware.
Demartek is a trademark of Demartek, LLC.
All other trademarks are the property of their respective owners.

2013 Demartek www.demartek.com Email: info@demartek.com