h10568 Vmax 40k Fast VP Vmware Cloud WP

White Paper
EMC INFRASTRUCTURE FOR VMWARE CLOUD ENVIRONMENTS

EMC Symmetrix VMAX 40K, EMC Symmetrix FAST VP, EMC SRDF, NEC Express5800/A1080a-E, VMware vSphere 5.0
Simplified storage management with FAST VP Remote replication with assured performance Simplified storage provisioning with EMC Unisphere for VMAX
EMC Solutions Group

Abstract
This white paper describes an automated storage tiering solution for multiple mission-critical applications virtualized with VMware vSphere on the EMC Symmetrix VMAX 40K storage platform. EMC SRDF coordination with EMC FAST VP provides site-to-site replication for disaster recovery and assured performance by automatically monitoring and tuning storage at the sub-LUN level at both sites. September 2012
Copyright 2012 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware, ESX, vMotion, VMware vCenter, and VMware vSphere are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions All trademarks used herein are the property of their respective owners. Part Number H10568.1
EMC Infrastructure for VMware Cloud Environments

Contents
Executive summary ............................................................................................................................. 7 Business case .................................................................................................................................. 7 Solution overview ............................................................................................................................ 7 Key results ....................................................................................................................................... 8 Introduction ....................................................................................................................................... 9 Purpose ........................................................................................................................................... 9 Scope .............................................................................................................................................. 9 Audience.......................................................................................................................................... 9 Solution overview ............................................................................................................................. 10 Overview ........................................................................................................................................ 10 Key components ............................................................................................................................ 10 Physical architecture ...................................................................................................................... 11 Hardware resources ....................................................................................................................... 12 Software resources ........................................................................................................................ 13 Storage environment ........................................................................................................................ 14 EMC Symmetrix VMAX 40K ............................................................................................................. 14 EMC Virtual Provisioning ................................................................................................................ 14 EMC FAST VP .................................................................................................................................. 14 EMC Symmetrix Remote Data Facility (SRDF) .................................................................................. 15 EMC Unisphere for VMAX ............................................................................................................... 15 EMC Symmetrix FAST VP ................................................................................................................... 17 FAST VP overview ........................................................................................................................... 17 FAST VP components...................................................................................................................... 17 FAST VP performance measurement and data movement ............................................................... 18 FAST VP allocate by policy .............................................................................................................. 18 FAST VP with OLTP workload .......................................................................................................... 19 FAST VP with DSS workload ............................................................................................................ 19 FAST VP with SRDF ......................................................................................................................... 19 Server Environment .......................................................................................................................... 21 NEC Express5800/ A1080a-E overview .......................................................................................... 21 VMware vSphere 5.0 ......................................................................................................................... 22 VMware vSphere 5.0 overview ....................................................................................................... 22 VMware vSphere configuration....................................................................................................... 22 VMware virtual machine configuration ........................................................................................... 23

Overview of mission-critical applications deployed in this solution ................................................... 26 Overview ........................................................................................................................................ 26 Microsoft SQL Server ...................................................................................................................... 26 Oracle Database 11g Release 2 ...................................................................................................... 26 SAP ERP and NetWeaver ................................................................................................................. 26 Application profile ......................................................................................................................... 27 Storage design for consolidation of applications on VMAX 40K ......................................................... 28 Overview ........................................................................................................................................ 28 Front-end port usage and zoning .................................................................................................... 28 Thin pool configuration .................................................................................................................. 29 FAST VP configuration .................................................................................................................... 29 Storage design considerations for mission-critical database applications with FAST VP ................. 30 Configuring EMC Symmetrix FAST VP with Unisphere for VMAX ......................................................... 31 Overview of FAST VP configuration ................................................................................................. 31 Step 1: Enable the FAST controller and set the control parameters ................................................. 31 Step 2: Create storage tiers ............................................................................................................ 32 Step 3: Create FAST policies ........................................................................................................... 32 Step 4: Associate storage groups with FAST policies and enable RDF coordination......................... 33 Step 5: Configure FAST VP monitoring and move windows ............................................................. 34 Cascaded storage groups ............................................................................................................... 35 Site protection with SRDF ............................................................................................................... 35 Performance testing and validation results ....................................................................................... 36 Overview ........................................................................................................................................ 36 Validation ...................................................................................................................................... 36 Test scenarios ................................................................................................................................ 36 End-to-end validation with FAST VP under normal conditions ............................................................ 37 Objectives ...................................................................................................................................... 37 Application workloads ................................................................................................................... 37 Storage performance overview ....................................................................................................... 37 FAST VP capacity use by storage group ........................................................................................... 39 SAP test result overview ................................................................................................................. 40 Oracle OLTP test results overview ................................................................................................... 41 SQL OLTP test result overview ........................................................................................................ 42 SQL DSS test result overview.......................................................................................................... 43 Summary of results ........................................................................................................................ 44 FAST VP workload tuning validation .................................................................................................. 45 Objectives ...................................................................................................................................... 45

Configuration ................................................................................................................................. 45 Policy tuning results and analysis .................................................................................................. 46 Summary of results ........................................................................................................................ 49 RDF coordination with continuous production workload .................................................................... 50 Objectives ...................................................................................................................................... 50 RDF coordination overview ............................................................................................................. 50 Test summary................................................................................................................................. 51 Failover with performance continuity ................................................................................................ 52 Objectives ...................................................................................................................................... 52 Failover test results ........................................................................................................................ 52 Test summary................................................................................................................................. 54 Conclusion ....................................................................................................................................... 55 Summary ....................................................................................................................................... 55 Findings ......................................................................................................................................... 55 References ....................................................................................................................................... 57 White papers ................................................................................................................................. 57 Product documentation.................................................................................................................. 57 Other documentation ..................................................................................................................... 57 Appendix A: Detailed application design and LUN layout................................................................... 58 Overview ........................................................................................................................................ 58 SAP overview ................................................................................................................................. 58 SAP ERP 6.0 ................................................................................................................................... 58 SAP IDES ........................................................................................................................................ 58 HP LoadRunner .............................................................................................................................. 58 SUSE Linux Enterprise Server for SAP applications ......................................................................... 59 SAP system architecture................................................................................................................. 59 SAP landscape ............................................................................................................................... 60 LoadRunner landscape .................................................................................................................. 62 SAP LUN Configuration ................................................................................................................... 62 Oracle Database 11g R2 ................................................................................................................. 62 Oracle ASM .................................................................................................................................... 63 Oracle grid infrastructure ............................................................................................................... 63 Oracle database and workload profile ............................................................................................ 63 Oracle workload description .......................................................................................................... 64 Oracle database schema ................................................................................................................ 64 Oracle database services ............................................................................................................... 65 Oracle LUN configuration ............................................................................................................... 65

Microsoft SQL Server ...................................................................................................................... 65 SQL Server 2012 DSS workload ...................................................................................................... 65 SQL Server 2012 DSS LUN configuration ........................................................................................ 66 SQL Server 2012 OLTP workload .................................................................................................... 66 SQL Server 2012 OLTP LUN configuration ....................................................................................... 66 SQL Server 2012 and Windows 2008 R2 settings for DSS and OLTP workload ................................ 67 Appendix B: Configuring Symmetrix remote replication .................................................................... 68 Configuring SRDF for remote replication ......................................................................................... 68

Executive summary
Business case As todays enterprises look to increase workforce productivity and transform their business, they are moving their database and applications to the private cloud. As a result, IT organizations face more demanding business objectives for more efficiency and improved quality of service, including: Maximizing the use of storage assets Maintaining performance levels at both production sites and business continuity sites Reducing capital expenditures and ongoing costs
To meet this challenge, IT organizations are evolving to provide more agile service delivery and to design their architecture for the future. At the same time, they must still cost-effectively manage their business requirements and service levels. To achieve this, organizations are beginning to offer IT-as-a-Service (ITaaS) by taking advantage of: Solution overview Resource pooling Virtualization Dynamic and virtual provisioning Commodity computing
EMC Symmetrix VMAX 40K with Enginuity 5876, along with Unisphere for VMAX, EMC Fully Automated Storage Tiering for Virtual Pools (FAST VP), and EMC Symmetrix Remote Data Facility (SRDF), are ideally suited to support the demands of the evolving enterprise infrastructure. By simplifying storage management and improving capacity use, these tools provide an infrastructure foundation that meets real business needs, including: Automated performance tuningWith FAST VP enabled, the storage array continuously tunes the application based on the access patterns, allowing you to monitor performance using Unispheres performance analyzer. Simplified storageThe FAST VP allocate-by-policy feature simplifies the capacity management of FAST VP environments. It allocates storage based on performance metrics, or from any tier in the FAST policy that has space. Assured performance at the disaster recovery (DR) siteFAST VP coordination with SRDF supports SRDF for tiering on the remote array, enabling optimized performance at both sites. Ease of managementUnisphere for VMAX provides an intuitive taskorientated interface for configuring and monitoring VMAX arrays, enabling simplified provisioning.
NEC Express5800/A1080a series is the base server platform of this solution. Representing the fifth generation of enterprise server architecture from NEC, this line of servers maintains NECs legacy for developing scalable enterprise servers that offer exceptional configuration flexibility, capacity, reliability and availability features.

Pairing the NEC Express5800/A1080a-E with VMware vSphere 5.0, the platform creates an outstanding solution for enterprise virtualization needs. Key results Our testing shows that this solution, based on EMC Symmetrix VMAX with Enginuity 5876, FAST VP, and SRDF, provides the following performance results: Sustained high performance levels for multiple critical database applications deployed on virtual storage, managed and automatically tuned by EMC FAST VP. FAST VP responds quickly to workload changes. Flexible policies are a powerful tool to further enhance performance when required. In the event of a failover to remote site, FAST VP SRDF coordination ensures performance at the DR replication site, bringing the benefits of FAST to the remote site.

Introduction
Purpose This white paper describes the design, testing, and validation of an enterprise VMware infrastructure using the EMC Symmetrix VMAX 40K storage platform with Enginuity 5876, EMC FAST VP, and EMC SRDF as its foundation. This solution demonstrates the performance, scalability, and application-specific functionality of the solution using multiple, representative application environments including Microsoft SQL, Oracle, and SAP. Specifically, this solution: Validates the performance and scalability of the test environment based on industry-standard online transaction processing (OLTP) and decision support system (DSS) benchmarks. Demonstrates simpler management using FAST VP allocation to a FAST policy. This allows data to be written to any pool defined by a FAST policy, simplifying capacity management. Demonstrates how FAST VP SRDF-coordination enables enterprise applications to seamlessly replicate virtual provisioned devices under FAST VP control at the production site to the replicated site. Demonstrates the responsiveness of FAST VP to changing performance requirements. You can tune FAST VP policies to enable storage administrators to dynamically increase performance for applications from storage on request.
Scope
This white paper discusses multiple EMC products as well as those from other vendors. Some general configuration and operational procedures are outlined. However, for detailed product installation information, refer to the user documentation for provided with those products. This white paper is intended for EMC employees, partners, and customers including IT planners, virtualization architects and administrators, and any other IT professionals involved in evaluating, acquiring, managing, operating, or designing infrastructure that leverages EMC technologies. Throughout this white paper we assume that you have some familiarity with the concepts and operations related to enterprise storage and virtualization technologies and their use in information infrastructures.
Audience

Solution overview
Overview EMC solutions are validated architectures that are designed to reflect real-world deployments. This section describes the key components, resources, and overall architecture that make up the solution and its environment. The key elements used in this solution include: EMC Symmetrix VMAX 40K storage array EMC FAST VP EMC Unisphere for VMAX NEC Express5800/A1080a-E VMware vSphere
Key components
These elements are described in more detail in subsequent sections.

10
Physical architecture
Figure 1 depicts the physical architecture for the solution described in this white paper.
Figure 1.
Physical architecture diagram
This solution is built on EMC Symmetrix VMAX 40K arrays running Enginuity 5876. Both source and target arrays provide a mix of Flash, FC, and SATA/SAS drives. FAST VP continually monitors and tunes performance by relocating data across storage tiers based on access patterns and predefined FAST policies. This continuous tuning occurs on both sites using the FAST VP SRDF coordination feature. We provisioned Microsoft SQL Server 2012 (two OLTP and one DSS), Oracle 11g R2 (OLTP), and a full SAP landscape running on Oracle. These applications ran on virtual machines in a VMware vSphere5 environment on EMC VMAX 40K storage, replicating to a DR site using SRDF. Load generation tools drove each of these applications simultaneously to validate the infrastructure and function of the FAST VP RDF coordination. We replicated the environment to a remote site within synchronous RDF distance over two 8-Gb/s FC links. Failover was performed to verify the performance of applications at the remote site.

11
The production site enterprise server environment was built on NEC Express5800/A1080a-E servers. This enterprise scalable server supports demanding Oracle OLTP, Microsoft SQL OLTP and DSS, and SAP workload on virtual machines. The effects of applying the FAST policy are documented in Performance testing and validation results. Hardware resources Table 1 lists the hardware resources used in the solution environment.
Table 1. Equipment EMC Symmetrix VMAX 40K Enginuity 5876 Hardware resources Quantity 2 Configuration 3-engine, 128-GB cache per engine 33 Flash 200 GB (including 1 HS) 132 600-GB 15k FC drives (including 6 HS) 70 2 TB 7.2k SATA/SAS drives (including 3 HS) Production site servers NEC Express5800/ A1080a-E 2 Production Site (Site A) 8 CPUs (10 C/2.40 GHZ/30 MB cache) 1 TB RAM 4 GbE IP Ports 4 146-GB 2.5-in. 15k SAS Disks 1 internal RAID controller 12 8 PCIe slots/2 16 PCIe slots 2 dual-port 8-Gb/s HBAs (4 FC) 1 quad GbE NIC Disaster recovery site servers 2 2 CPUs (10 C/2.40 GHZ/30 MB cache) 384 GB memory 2 x dual-port 8-Gb/s HBAs (4 FC) 1 x quad GbE NIC SAN 1 8 Gb SAN backbone

12
Software resources Table 2 lists the software resources used in the solution environment.
Table 2. Software EMC Symmetrix VMAX Enginuity code EMC Power Path/VE for VMware EMC Unisphere for VMAX EMC Solutions Enabler VMware vSphere 5 (Enterprise Plus) SAP Oracle ASMlib Oracle Database 11g R2 Microsoft Windows Server 2008 R2 Microsoft SQL Server 2012 Microsoft MSTPC E Toolkit Quest Benchmark Factory SUSE Linux Enterprise Server Red Hat Enterprise Linux Server SwingBench Software resources Version 5876 5.7 1 7.4 5.0.1 6.4 2.0.5 11.2.0.3 SP1 RTM 1.12.0 5.8.1 11 5.7 2.3

13
Storage environment
EMC Symmetrix VMAX 40K EMC Symmetrix VMAX 40K with Enginuity version 5876 provides the tiered storage configuration used in the test environment. The two primary Symmetrix VMAX features employed were FAST VP and SRDF. Built on the strategy of simple, intelligent, modular storage, the solution incorporates a highly scalable Virtual Matrix Architecture that enables Symmetrix VMAX arrays to grow seamlessly and cost-effectively from an entry-level configuration into the worlds largest storage system. Symmetrix VMAX supports Flash drives, FC drives, and SATA drives within a single array, as well as an extensive range of RAID types. The EMC Enginuity operating environment controls all components in the Symmetrix VMAX array. Enginuity 5876 for Symmetrix VMAX offers: More efficiency: New zero-downtime technology for migrations (technology refreshes) and lower costs with automated tiering More scalability: Up to two times more performance, with the ability to manage up to 10 times more capacity per storage administrator More security: Built-in encryption, RSA-integrated key management, increased value for virtual server and mainframe environments, replication enhancements, and a new e-licensing model
EMC Virtual Provisioning
EMC Virtual Provisioning is EMCs implementation of thin provisioning. It is designed to simplify storage management, improve capacity utilization, and enhance performance. Virtual Provisioning provides for the separation of physical storage devices from the storage devices as perceived by host systems. This enables nondisruptive provisioning and more efficient storage use. This solution uses virtually provisioned storage for all deployed applications.
Symmetrix Array Controls CLI v7.4 Product Guide.

EMC FAST VP
For detailed information on virtual provisioning, refer to the EMC Solutions Enabler
EMC FAST VP is a feature of Enginuity version 5875 and higher that provides automatic storage tiering at the sub-LUN level. Virtual pools are Virtual Provisioning thin pools. FAST VP is a key component of the solution described in this white paper. For a detailed overview, see EMC Symmetrix FAST VP.

14
EMC Symmetrix Remote Data Facility (SRDF)
The EMC Symmetrix Remote Data Facility (SRDF) family of software is a suite of remote storage replication solutions for DR and business continuity. The SRDF family offers deployment flexibility and scalability, delivering distance-replication capabilities and helping customers meet mixed service-level requirements with minimal effect on operations. SRDF features include: Massively-parallel high performance that delivers unsurpassed recovery point objectives (RPOs) and recovery time objectives (RTOs), with little effect on servers. Zero data exposure, very long-distance capability, and multi-hop functionality that enable you to optimize resources while meeting mixed service levels. Coordinated processing across multiple sets of data and systems that enhances enterprise-wide application restart. Seamless integration with hundreds of leading enterprise, storage, and backup applications that enables faster deployment and simpler management. Flexible, automated, and easy-to-use management options to ensure continuous protection of your data. Integration with FAST VP for coordinating performance movement at the source and target sites.
EMC Unisphere for VMAX
EMC Unisphere for VMAX is an advanced graphical user interface (GUI) for managing Symmetrix VMAX arrays. Unisphere for VMAX enables you to provision, manage, and monitor any Symmetrix VMAX array from one screen and significantly reduces storage administration time. As shown in Figure 2 (page 16), Unisphere for VMAX uses the same GUI framework as the unified EMC VNX platforms. For customers who use Symmetrix VMAX and VNX in the same data center, Unisphere provides a consistent look and feel that simplifies management operations. Unisphere provides a web browser interface that enables the following operations: Access management Configuration management Replication management Monitoring and alerts, performance analysis, and reporting

15
Figure 2 shows the Unisphere for VMAX user interface.
Figure 2.
Unisphere for VMAX interface

16
EMC Symmetrix FAST VP

FAST VP overview FAST VP provides support for sub-LUN data movement in thinly provisioned environments. It combines the advantages of Virtual Provisioning with automatic storage tiering at the sub-LUN level to optimize performance and cost, while radically simplifying storage management and increasing storage efficiency. FAST VP data movement between tiers is based on performance measurement and user-defined policies, and is executed automatically and nondisruptively by FAST VP. This section provides an overview of FAST VP features and functionality. Configuring EMC Symmetrix FAST VP outlines the main steps for configuring FAST VP on Symmetrix VMAX and the settings defined for the solution. FAST VP components As shown in Figure 3, configuring FAST VP involves three types of components storage groups, FAST policies, and storage tiers: A storage group is a logical grouping of storage devices used for common management. A storage group is associated with a FAST policy that determines how the storage groups devices are allocated across tiers. A FAST policy is a set of tier usage rules that is applied to associated storage groups. A FAST policy can specify up to three tiers and assigns an upper usage limit for each tier. These limits determine how much data from a storage group can reside on each tier included in the policy. Administrators can set high-performance policies that use more Flash drive capacity for critical applications, and cost-optimized policies that use more SATA drive capacity for less-critical applications. A storage tier is made up of one or more virtual pools. To be a member of a tier, a virtual pool must contain only data devices that match the technology type, drive speed, and RAID protection type of the tier.
Figure 3.
FAST VP components

17
In summary, by simply pooling storage resources, defining a policy, and applying it to the application, FAST VP automatically and dynamically moves application data to the tier that best suits the level of service required. FAST VP performance measurement and data movement FAST VP works at the sub-LUN level, introducing finer granularities of both performance measurement and data movement, and can spread the data from a single thin device across multiple tiers. The sub-LUN metrics collected for thin devices under FAST VP control contain measurements that enable FAST VP to make separate data movement requests for every 7,680 KB unit of storage that makes up the thin device. This unit of storage consists of 10 contiguous thin device extents and is known as an extent group. FAST VP algorithms perform two types of moves: Compliance movement: Initially, FAST VP distributes data across the different tiers to enforce compliance with the datas associated FAST policy. Performance movement: When compliance with the policy is achieved, FAST VP continues moving data between tiers to optimize performance, while maintaining compliance with the policy.
FAST VP automatic analysis identifies the busiest extent groups and moves them to the highest-performing Flash tier. It also identifies inactive extent groups and moves them to the SATA tier. This results in the thin devices data being distributed across multiple thin pools. Because the most active data is residing on the highest-performing storage devices, application response times are unaffected. FAST VP continuously tunes the storage resources to ensure that the right data is placed on the right tier at the right time with automatic analysis and data re-tiering happening at all times Configuring EMC Symmetrix FAST VP outlines the main steps for configuring FAST VP on Symmetrix VMAX and the settings defined for the solution. FAST VP allocate by policy To further simplify the management and capacity planning of FAST VP environments, Enginuity 5876 and Solutions Enabler 7.4 provides FAST VP allocation by policy. This system-wide setting ensures that new allocations for thin devices associated with FAST VP policies no longer only come from the pool to which a thin device is bound but from any one of the tiers associated with the FAST policy. FAST VP attempts to allocate new writes in the most appropriate tier first, based on available performance metrics. If no performance metrics are available, the allocation is attempted in the pool the device is bound to. If the bound tier cannot service a new allocation because it is full, the tracks are allocated from one of the remaining tiers. EMC recommends that you enable the VP allocation by FAST policy. Note For more information on the decision-making process of the VP allocation by FAST policy feature, see the Advanced FAST VP features section of
Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC Symmetrix VMAX Series ArraysTechnical Notes.

18
FAST VP with OLTP workload
FAST VP is an enabling technology for workloads with small, random I/O and relatively small working sets that fit into the higher-performing tiers of a FAST policy. OLTP databases tend to be highly random in nature, with small working sets compared to the total database size. Additionally, OLTP databases have inherent locality of reference with varied I/O patterns, for the following reasons: The relative importance of data changes from object to object. Some tables tend to be accessed more than others. The number of IOPS per object size in gigabytes, also known as object intensity, changes quite significantly. A good example is a database index compared with a database table. The relative IOPS received by a database block occupied by an index object can be very high compared to the IOPS received by a database block consumed by a table object. Database redo logs have a very predictable sequential write workload, and this type of activity does not benefit significantly from tiering up to Flash. EMC recommends that you either exclude these logs from any FAST policy or pin them to their existing tier so that FAST VP will not include them in its analysis.
Note
FAST VP with DSS workload
FAST VP is also an enabling technology for DSS workloads. Data warehouses often grow into very large database environments due to the growth of application data and increased regulatory requirements. The value of business data stored in the data warehouse can change over time, and availability as well as performance change accordingly. Data warehouse applications tend to issue scan-intensive operations that access large data portions of the data at a time and also commonly perform bulk loading operations. These operations result in larger I/O sizes than OLTP workloads do and they require a storage subsystem that can provide the necessary throughput. This makes throughput, or megabytes per second (MB/s), the critical metric. Although Flash disk storage can provide more than 100 MB/s of throughput, generally it is best suited to serving a small portion of the databases hot data. Therefore, in this solution, we used a two-tier policy consisting of FC and SATA storage to provide a cost-efficient mix of storage to satisfy the needs of a DSS workload.
FAST VP with SRDF
A core feature of Enginuity 5876 is the SRDF enhancement to consider remote devices (for example, R2) in a FAST VP policy. Previously, FAST VP operated independently on each side of the SRDF link. Before Enginuity 5876, FAST VP promotion and demotion decisions were based on the workload seen by each individual device. While a source device (R1) would typically undergo a read and write workload mix, the corresponding target device would only see a write workload. Reads against the R1 were not reflected across the link. As a result, the R2 device data might not be located on the same tier as the corresponding data on the R1 device. Enginuity 5876 introduces SRDF awareness for FAST VP. The performance metrics collected for R1 devices are periodically transmitted across the link to the corresponding R2 devices. On the R2 device, the R1 performance metrics are merged EMC Infrastructure for VMware Cloud Environments
19
with the actual R2 metrics. FAST VP takes into account the workload on the R1 device and then makes promotion and demotion decisions for the R2 device data. In an SRDF swap operation, which reverses the direction of replication, FAST VP statistics are automatically transferred to the new target site. SRDF coordination must be enabled at both sites. FAST VP SRDF coordination can operate in synchronous, asynchronous, and adaptive copy modes. It also supports concurrent SRDF configurations.
Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC Symmetrix VMAX Series ArraysTechnical Notes.
Figure 4 illustrates FAST VP coordinated movement with SRDF.
For detailed information on this feature, see the Advanced Features section of
Figure 4.
FAST VP SRDF coordination

20
Server Environment
NEC Express5800/ The NEC Express5800/A1080a is NECs flagship, highly scalable HA enterprise server. A1080a-E overview Boasting a maximum memory configuration of 2 TB and eight CPU sockets, the NEC Express5800/A1080a is both highly scalable and highly flexible. The NEC Express5800/A1080a has many key design features that are ideal for mixworkload large-scale virtualization. Below is a few of the technological advances that make consolidating entire database, application, and Web infrastructures onto a single NEC Express5800/A1080a the preferred solution to growing IT needs. Featuring the Intel Xeon E7 series of processor and eight processor sockets, the NEC Express5800/A1080a can expand to as many as 80 cores (160 threads) of CPU power. This means that by using vSphere 5.0, which has an achievable maximum of 25 vCPUs per core depending on workload, NEC Express5800/A1080a has the potential to run 2,000 vCPUs. With this increased processing capacity, overall virtual machine capacity is expanded and vCPUs are more optimally utilized. With these large core and thread capacities, ancillary but necessary functions, such as live migration and backup processes, need not interfere with application processing. The NEC Express5800/A1080a and Intel Xeon E7 series processors provide ample resources to ensure virtual machines are never starved for compute power, even during maintenance activities. The NEC Express5800/A1080a is ideal not only for Web server and application server virtualization, but also for database server virtualization and other workloads that may require high I/O capabilities. The NEC Express5800/A1080a, designed with 12 x 8 and 2 x 16 PCI Express 2.0 slots, can handle large numbers of multiple-port NICs and HBAs for many I/O connections to external networks or storage devices as well as provide sufficient connection redundancy. NEC Express5800/A1080a has full support for Direct I/O. This feature, called VMDirectPath for vSphere, allows virtual machines direct access to physical NIC and HBA ports without using para-virtualized adapters. With the large number of I/O slots, Direct I/O is a feature you might want to use for specialized virtual machines and workloads. If you opt to use this feature, use special caution and consult the latest VMware documentation.

21
VMware vSphere 5.0

VMware vSphere 5.0 overview For the solution, the Microsoft SQL, Oracle, and SAP application servers are fully virtualized using VMware vSphere 5. This section describes the virtualization infrastructure, which uses the following components and options: VMware vSphere 5.0.1 VMware vCenter Server VMware vSphere vMotion EMC PowerPath/VE for VMware vSphere Version 5.7
VMware vSphere 5.0 VMware vSphere 5.0 is a complete, scalable, and powerful virtualization platform, with infrastructure services that transform IT hardware into a high-performance shared computing platform, and application services that help IT organizations deliver the highest levels of availability, security, and scalability. VMware vCenter Server VMware vCenter is the centralized management platform for vSphere environments, enabling control and visibility at every level of the virtual infrastructure. VMware vSphere vMotion VMware vSphere vMotion supports the live migration of virtual machines across servers with no disruption to users or any loss of service. Storage vMotion is VMware technology that enables live migration of a virtual machines storage without any interruption in the availability of the virtual machine. This allows the relocation of live virtual machines to new data stores. EMC PowerPath/VE EMC PowerPath/VE for VMware vSphere delivers PowerPath multipathing features to optimize VMware vSphere virtual environments. PowerPath/VE installs as a kernel module on the VMware ESXi host and works as a multipathing plug-in (MPP) that provides enhanced path management capabilities to ESXi hosts. VMware vSphere configuration VMware vCenter Server provides a scalable and extensible platform to centrally manage VMware vSphere environments, providing control and visibility at every level of the virtual infrastructure. In this solutions virtual environment, we configured four VMware vSphere 5.0 servers to host virtual machines at the production and DR sites. A fifth ESXi sever hosted the management virtual machines and vCenter servers. At Site A, two ESXi5 hosts connect to the VMAX 40K array. Host A runs virtual machines for Oracle and SAP applications, and Host B runs the Microsoft SQL OLTP and DSS virtual machines serving as the production environment (R1). At Site B, separate ESXi 5 hosts connect to the DR environment (R2) and are used to mount the virtual machines in the event of a failover. Figure 5 shows an excerpt from

22
vCenter with the virtual machines running at the production site and the standby hosts at Site B.
Figure 5.
Production (Site A) and DR (Site B) shown in vCenter
VMware virtual machine configuration
All virtual machines in this configuration use virtual machine disks (VMDK) from VMware Virtual Machine File System (VMFS) data store volumes. Each VMFS data store hosts a single VMDK disk, ensuring high performance and zero contention. This practice also ensures you have the ability to restore at an application level with EMC TimeFinder Clone/Snap on the VMAX 40K array. Table 3 shows the virtual machines CPU and memory allocation for each application virtual machine.
Table 3. Virtual machine CPU and memory allocation Virtual machine name ORACLEDB SAPDI1 SAPDI2 SAPASCS SAPCI SAPDB Microsoft SQL DSS SQLTPCH01 CPU count 12 8 8 2 8 16 32 Memory size 54,272 MB 16,384 MB 16,384 MB 4,096 MB 16,384 MB 32,768 MB 131,072 MB
Application Oracle SAP

23
Application Microsoft SQL OLTP
Virtual machine name SQLTPCE01 SQLTPCE02 Domain controller
CPU count 16 16 4
Memory size 32,768 MB 32,768 MB 4,096 MB
Because the virtual machines are replicated to Site B, the virtual machine CPU and memory configuration is identical at both sites. Full details of the LUNs provisioned for each virtual machine are found in Appendix A: Detailed application design and LUN layout. Selecting the SCSI driver type for data LUNs VMware Paravirtual SCSI (PVSCSI) adapters are high-performance storage drivers that can improve throughput and reduce CPU use. PVSCSI adapters are best suited for SAN environments, where hardware or applications drive high I/O throughput. As show in Figure 6, the SCSI controllers type changed to paravirtual to improve the driver efficiency. The default SCSI controller driver is LSI Logic SAS. LUNs are spread across all available SCSI drivers.
Figure 6.
SCSI controllers
EMC Virtual Storage Integrator EMC Virtual Storage Integrator (VSI) provides enhanced visibility into Symmetrix VMAX 40K directly from the vCenter GUI. Figure 7 shows the data store and storage pool information, which provides information about virtual pool usage for the Oracle_SOE1_1 data store.

24
Figure 7.
Data store and storage pool information viewed from VSI
VMAX 40K volumes host the VMFS data stores for this solution. Figure 7 shows the ESXi server and the Symmetrix VMAX 40K storage mapping with details about VMFS data stores and the LUNs. The Storage Viewer identifies details about VMFS data stores such as the VMAX storage volumes hosting the data store, the paths to the physical storage, pool usage information, and data store performance statistics. Figure 8 shows the LUN view from VSI. From here, administrators can identify the Symmetrix device ID for LUNs and data stores, if user-defined labels are set on VMAX LUNs. Administrators can export these listings to CSV files for manipulation with VMware PowerCLI scripts for rapid provisioning of data stores to ESXi hosts.
Figure 8.
EMC Virtual Storage integrator LUN view

25
Overview of mission-critical applications deployed in this solution

Overview This section gives a brief overview of the applications deployed in the test environment, including: Microsoft SQL Server Oracle Database SAP ERP
Detailed configuration information is included in Appendix A: Detailed application design and LUN layout. Microsoft SQL Server Microsoft SQL Server 2012 is the latest version of the Microsoft database management and analysis system for e-commerce, line-of-business, and data warehousing solutions. In the test environment, we engaged two applications, each with different workload patterns running on the Microsoft SQL Server 2012 enterprise class platform. The applications are a TPC-H-like application (acting as a typical DSS application), and a TPC-E-like application (acting as a typical OLTP application). Oracle Database 11g Release 2 Oracle Database 11g Release 2 Enterprise Edition delivers industry-leading performance, scalability, security, and reliability on a choice of clustered or single servers running Windows, Linux, or UNIX. It provides comprehensive features for transaction processing, business intelligence, and content management applications. This solution deploys a single OLTP database instance using Oracle Automatic Storage Management. SAP ERP and NetWeaver SAP ERP 6.0, powered by the SAP NetWeaver technology platform, is a fully-integrated enterprise resource planning (ERP) application that fulfills the core business needs of midsize companies and large enterprises across all industries and market sectors. SAP ERP 6.0 delivers a comprehensive set of integrated, cross-functional business processes and can serve as a solid business process platform that supports continued growth, innovation, and operational excellence. SAP IDES (Internet Demonstration and Evaluation System) systems support demos, testing, and functional evaluation based on preconfigured data and clients. IDES contains sample application data for various business scenarios, with business processes that are designed to reflect real-life business requirements and have access to many realistic characteristics. This solution uses IDES to represent a model company for testing purposes, running an Oracle 11g Release 2 database fully virtualized on VMware ESXi5. We provisioned and optimized the architecture according to EMC and SAP recommended practices. For more information, see Appendix A: Detailed application design and LUN layout.

26
Application profile
Table 4 summarizes the profile for each of the four applications deployed in this solution.
Table 4. Application profiles VM configuration 3 SAP ERP 6 IDES EHP 4 instances, 16 vCPUs with 32 GB RAM DB instance, 12 vCPUs with 53 GB RAM 2 SQL instances, 16 vCPUs with 32 GB RAM 1 SQL instance, 32 vCPUs with 128 GB RAM DB configuration 1 Oracle DB instance at 845 GB capacity 1 Oracle DB per virtual machine, 2 TB capacity 1 DB per virtual machine, 1 TB capacity 1 DB per virtual machine, 2 TB capacity Workload configuration 1,000 LoadRunner Update users + local client copy simulation, 80:20 R/W ratio SwingBench order entry workload with 400 heavy access users, 60:40 R/W Ratio Mixed workloads to simulate hot, warm applications, 85:15 R/W ratio 2 concurrent loads, 100% Read
Application SAP
Oracle OLTP
SQL OLTP
SQL DSS
We provisioned and optimized virtual machine resources for load-testing purposes, according to the recommended practices specific to each application.

27
Storage design for consolidation of applications on VMAX 40K

Overview This section describes the storage configuration and provisioning for this solution and is structured as follows: Front-end port usage and zoning Front-end port usage and zoning Thin pool configuration Application LUN layout FAST VP policy design
The application workloads were logically separated using masking views within the VMAX 40K and HBAs. Figure 9 shows the front-end port use for each application. Although physically running on the same server as Oracle, SAP is segregated to use different front-end ports and HBAs, using zoning and masking. Both MS SQL OLTP workloads running similar workloads use the same ports and are separated from DSS workloads.
Figure 9.
Logical grouping of ports to applications

28
Thin pool configuration
EMC Virtual Provisioning greatly simplifies the storage design. Because this configuration involves remote replication with SRDF, both source and target arrays are configured in exactly the same manner. We created four thin pools on each array, based on the drive types available. Table 5 shows the thin pool definitions.
Table 5. Thin pool configuration RAID protection RAID5 3+1 RAID1 RAID1 RAID6 6+2 No. of drives 32 126 64 72 TDAT size 68.8 GB 66 GB 49.2 GB 240 GB No. of TDAT 64 504 256 256 Pool capacity 4.2 TB 32 TB 12.2 TB 60 TB
Thin pool name FLASH_3RAID5 FC10K_RAID1 FC15K_RAID1 SATA_6RAID6
Drive size/ technology/RPM 200 GB Flash 600 GB FC 10k 450 GB FC 15k 2 TB SATA 7.2k
For this solution, the SAP, Oracle, and MS OLTP applications are bound to the FC10K_RAID1 pool. The MS DSS application is bound to the FC15K_RAID1 pool which is backed by a smaller number of drives. FAST VP configuration VMAX administrators can set high-performance policies that use more Flash drive capacity for critical applications, and cost-optimized policies that use more SATA drive capacity for less-critical applications. The ideal FAST VP policy would be to specify 100 percent for each of the included tiers. Such a policy would provide the greatest amount of flexibility to an associated storage group, as it would allow 100 percent of the storage groups capacity to be promoted or demoted to any tier within the policy. In this implementation, we designed the FAST policies to prevent any single application from consuming high portions of Flash storage to ensure a cost-effective mix of storage with consistently high performance. Conversely, we set the SAP policy to limit SATA usage to ensure minimum levels of performance during periods of low activity. EMC offers an analysis service for customers to estimate the performance and cost of mixing types of drive technologies (Flash, FC, and SATA) within Symmetrix VMAX storage arrays. Table 6 shows the FAST VP policies used for the application workloads in this solution for Oracle, SAP, SQL OLTP, and SQL DSS.

29
Table 6.
FAST VP policy for Oracle, SAP, SQL OLTP, and SQL DSS FAST policy name MSSQL_OLTP MSSQL_OLTP MSSQL_DSS Oracle SAP Flash 5% 5% 0% 15% 10% FC 40% 40% 100% 35% 80% SATA 100% 100% 100% 50% 10%
Storage group MSSQL1_OLTP MSSQL2_OLTP MSSQL_DSS Oracle SAP
The policies set for this solution (as shown in Table 6) are a result of the workload analysis and dynamic tuning to ensure that application workload performed within acceptable performance boundaries. Further tuning for enhanced performance is discussed in FAST VP workload tuning. Storage design considerations for mission-critical database applications with FAST VP The design incorporates the following recommended practices: Use separate storage volumes for data files and log files Use separate file groups for large databases For ASM, EMC recommends separate ASM disk groups for DATA, REDO, FRA, and TEMP when replicating with SRDF Bind all thin devices to the FC tier. Log devices and temp files should be pinned to the FC tier
Figure 10 shows an overview of how each critical application is configured for FAST VP. In this implementation, only data LUNs are managed by FAST VP. LUNs for OS and LOG are pinned to the FC tier, excluding them from FAST decisions and movement. For full details of LUN layout and sizing for each application refer to Appendix A: Detailed application design and LUN layout.
Figure 10.
General view of FAST VP configuration for mission-critical applications

30
Configuring EMC Symmetrix FAST VP with Unisphere for VMAX

Overview of FAST VP configuration This section outlines the steps for configuring FAST VP on a Symmetrix VMAX array. 1. 2. 3. 4. 5. Enable the FAST controller and set the control parameters. Create the storage tiers. Create the FAST policies. Associate each storage group with the relevant FAST policy and enable FAST VP RDF coordination. Configure performance and move time windows.
To complete these steps, you can use the Unisphere FAST Configuration Wizard, the Unisphere menu commands, or the SYMCLI. Step 1: Enable the You can access FAST VP settings through the Storage tab in Unisphere under FAST. Figure 11 shows the settings used in the test environment. FAST controller and set the control parameters
Figure 11.
FAST VP settings
Note that Allocate by FAST Policy is checked. This ensures that FAST VP will use all tiers for new allocations based on performance and capacity restrictions. For further information on the control parameters used by FAST VP, see the EMC Solutions Enabler Symmetrix Array Controls CLI Product Guide.

31
Step 2: Create storage tiers
When creating a storage tier, specify the following attributes: The tier name, which uniquely identifies the storage tier The disk technology on which the tier will reside The RAID protection type for the tier The names of the virtual pools that belong to the tier
The solution defines four storage tiers as shown in Figure 12. Unisphere guides administrators through the process of creating the necessary VP tiers. In this solution, four virtual pools reside on the array. Tiers can be deleted and renamed as required.
Figure 12.
Defined storage tiers
Note: When possible, EMC recommends that you configure each FAST VP tier with a single virtual pool, to ensure that all pools within a tier have the same overall performance capabilities. For example, if a tier contains one pool spread over 128 drives and another pool spread over 16 drives, the number of spindles making up the underlying storage will be different for each pool. This could result in unbalanced use of each pools performance capabilities. Step 3: Create FAST policies When creating a FAST policy, specify the following attributes: The policy name, which uniquely identifies the policy The name of each tier to be added to the policy The upper limit (percent) of each tier that an associated storage group can occupy under the specified policy
Figure 13 shows how to create the FAST policy for this solution, using the FAST Configuration Wizard.

32
Figure 13.
Creating a FAST policy
Step 4: Associate storage groups with FAST policies and enable RDF coordination
To apply a FAST policy to a schema, associate the schemas storage group with the FAST policy. Figure 14 shows how to associate the Oracle policy with the Oracle storage group.
Figure 14.
Associating a FAST policy with a storage group
Under the Advanced options check Enable FAST VP RDF Coordination. This enables the feature allowing for R1 and R2 to send and receive performance statistics for FAST VP movement for the associated storage group.

33
Step 5: Configure FAST VP monitoring and move windows
Time windows are used by FAST VP to specify when data can be collected for performance analysis and when data movements can be executed. FAST VP shares monitoring windows with FAST and Symmetrix Optimizer. However, FAST VP requires a separate data movement window. Performance time windows define the days and times when FAST VP performs analysis. Data movement windows define the days and times when FAST VP moves data between tiers. EMC recommends that you configure both the monitoring and data movement windows to be always open so that FAST VP can use the most recent analysis and metrics to optimize data placement. Figure 15 shows the definition of a monitoring window for this solution.
Figure 15.
Creating/managing time windows
Note that both source and target arrays were configured with the same FAST VP policies and storage groups. Table 6 (page 30) shows the FAST policies assigned to the each application.

34
Cascaded storage groups
Enginuity 5876 supports cascaded storage groups, which is the ability to nest storage groups within storage groups. A parent storage groups is associated with a masking view, and contains a number of nested child groups. Cascaded storage groups make it easier to manage ESXi clusters with FAST VP enabled storage. Figure 16 shows the cascaded storage group MSSQL_OLTP_Apps, with three child storage groups, two of which are associated with FAST policies.
Figure 16.
Cascaded storage groups
Site protection with SRDF
Both the source and target Symmetrix VMAX 40K arrays in this solution have the same number of disks and storage tiers. We created devices and storage groups using the processes described in the previous sections. For continuity, we used the same names for the storage groups and FAST VP policies on the arrays at both sites, and provisioned the DR mount servers that connected to the target array. All application environments are protected with SRDF. SRDF is configured using the Solutions Enabler command line interface (CLI). The steps used to configure SRDF protection for this configuration are detailed in Appendix B: Configuring Symmetrix remote replication.

35
Performance testing and validation results

Overview This section describes how we tested the applications in our solution environment. Each test is described in more detail in later sections. Notes: Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, this workload should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated. All performance data contained in this report was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly. EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute. Validation To validate the environment, we deployed all applications and populated them with test data. Each of the four applications (SAP, Oracle, SQL OLTP, and SQL DSS) was deployed at the production location, Site A, and workloads were driven against each application running simultaneously on the VMAX 40K storage array. Each application is associated with a FAST policy, and replicates to the remote location, Site B, using SRDF. We used Unispheres Performance Analyzer module to monitor and gather storage performance data in addition to application performance monitoring tools. Test scenarios The test contains following scenarios: End-to-end validation with FAST VP under normal conditions FAST VP workload tuning RDF coordination with continuous production workload Failover with performance continuity

36
End-to-end validation with FAST VP under normal conditions

Objectives The objective of this test was to validate the solution build under normal operating conditions for a normal work day, with FAST VP storage tiering enabled. Each application performs within the defined boundaries, which are the basis for an acceptable service-level agreement. We evaluated all aspects of the solution, including the VMware vSphere server and virtual machine performance, SAP, Oracle, SQL OLTP, and SQL DSS server and client experience, with FAST VP policies applied to each application. Application workloads For each application, we used load generation tools to simulate real world user interactions. The details are as follows: We used an MSTPCE toolkit on the client virtual machines to generate TPC-Elike loads simultaneously for SQL Server OLTP databases. This emulated warm and hot workloads. We used Quest Benchmark Factory to generate a TPC-H-like load for the SQL Server DSS database. We generated a SwingBench TPC-C-like order entry workload with 400 users and ran it against the Oracle database. For SAP, using four LoadRunner generators, we programmed 1,000 virtual interactive/dialog users to log on to SAP and perform real-life update transactions simultaneously. Additionally, a test client copy was performed to simulate generation of reports (such as month-end closing). LoadRunner produces a constant database write workload by having virtual users perform update transactions at a fixed pace. Conversely, the client copy simulation reads several tables of varying sizes simultaneously. Therefore the IOPS workload varies just as much, depending on the set of tables being read in parallel at any given time.
Storage performance overview
Each of the workloads for the four applications (SAP, Oracle, SQL OLTP, and SQL DSS) were run together and stabilized within three hours. We measured each applications performance to ensure it was within predefined KPIs and that all workloads coexisted without a negative impact on each other. Figure 17 shows the running workload on the source array for the Oracle, SAP, and Microsoft OLTP applications.

37
Figure 17.
Host IOPS as shown in Unisphere for VMAX
The MSSQL DSS workload issued typical online analytical processing queries, one after another. Each query generated a large quantity of table/index scans with an average bandwidth of just over 800 MB/s as shown in Figure 18.
Figure 18.
MSSQL DSS bandwidth during baseline test (MB/s)

38
FAST VP capacity use by storage group
Figure 19 shows a capacity breakdown of each applications storage by tier. This output from Unisphere FAST demonstrates how each application storage group is spread out on the underlying FAST tiers and whether the storage group is compliant with the policy set.
Figure 19.
Capacity breakdown of storage groups under FAST control
Figure 19 shows the usage of each storage group. MSSQL_DSS is spread across the FC and SATA tiers. Oracle, SAP, and MSSQL_OLTP are using capacity from three storage tiers. You can use Tier Usage reports from Unisphere to monitor tier usage and FAST VP demands to ensure that sufficient capacity exists on the array for more applications. Figure 20 shows the Tier Demand report for the FLASH_3RAID5 tier. The purple triangle shows the maximum demand placed on the tier by FAST VP storage groups, the blue area shows the capacity currently used, and the green area indicates the remaining capacity for future use.
Figure 20.
Tier Demand Report for the FLASH_3RAID5 tier
You can configure alerts based on usage thresholds through Unisphere to email or send SNMP traps to administrators if usage reaches the defined thresholds. You can also set up audit level accounts for users who need to monitor storage usage without giving the ability to make changes.

39
SAP test result overview
Figure 21 shows the number of users who logged on from SAP. LoadRunner generated a user load of 1,000 active users. Users tend to spawn multiple remote (RFC) users as needed, resulting in a higher user count across the servers. Figure 21 shows 1,352 users.
Figure 21.
SAP logged-on user sessions
Table 7 shows the different KPIs that were checked to verify SAP stability.
Table 7. Metric Dialog response time KPIs for SAP and observed values Ideal value 1,100 ms or less < 50% - 60% < 20% >20% 0% - 10% Recorded value 929.93
Description An SAP KPI that measures the total time from when SAP receives a request until a result or output screen is presented to the user. Actual front-end usage. System/OS usage. Unused resources. Time when SAP work processes are placed on hold, waiting for an I/O response; usually an indicator of a hardware issue if neither client copy nor an upgrade is running. Time spent waiting for database response. The final level of memory that a dialog work process can consume before it is terminated. Its use indicates a memory bottleneck. Measure of a buffers efficiency by counting how much of the data requested is already loaded on the buffer versus what must still be read from the disk. The higher the value, the more efficient it is.
User utilization System use Idle time I/O wait
21% 1% 78% 0%
Wait time Heap memory usage
< 10% of response time 0%
0.04% 0%
Program/PXA buffer hit ratio
95%
99.96%

40
Oracle OLTP test results overview
We generated a SwingBench order entry workload with 400 users and ran it against SOE1 schema with FAST VP enabled. The test procedure was carried out with the four application (SAP, Oracle, SQL OLTP, and SQL DSS) workloads running together. The performance metrics for 400 users were: The Oracle I/O pattern is 8 KB read and 8 KB write, with a read/write ratio of 60/40 percent, respectively. 7,656 IOPS, with an average disk read latency of 5.5 ms and a disk write latency of 7 ms.
Table 8, Table 9, Table 10, and Table 11 list the Oracle performance for the test.
Table 8. Oracle OLTP performance Recorded value 97,017 36 ms
Oracle OLTP performance TPM SwingBench response time Table 9. Oracle storage performance
Storage performance IOPS Disk read response time Disk write response time Table 10. Oracle foreground events
Ideal value < 10ms < 10ms
Recorded value 7,656 5.5 ms 7 ms
Oracle top 5 timed foreground events DB file sequential read Log file sync DB CPU Library cache: mutex X Read by other session Table 11. Oracle background wait events
Average wait (ms) 9 6 1 9
% DB time 80.05 11.80 5.66 0.09 0.07
Oracle top timed background events Log file parallel write
Average wait (ms) 3
% Background time 46.66

41
The test results included the following Oracle events: DB file sequential read: The session waits while a sequential read from the database is performed. This event is also used for rebuilding the control file, dumping the data file headers, and getting the database file headers. Log file parallel write: The writing of redo records to the redo log files from the log buffer.
The acceptable value for average database I/O latency (the Oracle measurement DB file sequential read) is less than or equal to 20 ms. Log file parallel write should be no more than 15 ms. For the test we implemented, with FAST policy in place, both DB file sequential read and log file parallel write (9 ms and 3 ms) exceeded the acceptable values. SQL OLTP test result overview We performed the baseline SQL OLTP performance test with the four application (SAP, ORACLE, SQL OLTP, and SQL DSS) workloads running together with FAST VP enabled. A performance baseline was defined to represent the OLTP environment before applying the FAST VP policies as follows: This configuration represented the database performance characteristics after FAST VP was enabled and the workload had stabilized. Running the simulated user load with this configuration showed that the ESX server had no CPU or memory constraints, and the client application emulated the varying workload. The SQL OLTP application I/O pattern is typically 8 KB read/write, with a read/write ratio of 85:15 percent, respectively. The SQL Server processed 176,940 transactions per minute and the client processed 47,760 transactions per minute, including TempDB transactions. The drives supported a workload of 6,802 IOPS in total. The average disk latency is less than 20 ms. The average CPU use of SQL VM was less than 75 percent.
Table 12 lists the performance results for the SQL OLTP load test.
Table 12. Performance data with SQL OLTP load Ideal value SQL01 SQL02 Client transactions per minute SQL01 SQL02 SQL Server transactions per second SQL01 SQL02 < 75% < 75% Value 3.5% 15.3% 9,240 38,520 37,920 139,020
OLTP performance Average CPU use (%)

42
OLTP performance Average IOPS SQL01 SQL02 OLTP data LUN average latency (milliseconds) (read/write/transfer) OLTP log LUN average latency (milliseconds) (read/write/transfer) TempDB data LUN average latency (milliseconds) (read/write/transfer) TempDB log LUN average latency (milliseconds) (read/write/transfer) SQL01 SQL02 SQL01 SQL02 SQL01 SQL02 SQL01 SQL02
Ideal value < 20ms < 20ms < 5ms < 5ms < 20ms < 20ms < 5ms < 5ms
Value 1,646 5,156 9/4/10 11/7/11 0/2/2 0/3/3 0/0/0 3/0/3 0/3/3 0/3/3
SQL DSS test result overview
We performed the baseline SQL DSS performance test with the four applications (SAP, ORACLE, SQL OLTP, and SQL DSS) workload running together with FAST VP enabled on the Flash, FC, and SATA tiers. We defined a performance baseline to represent the DSS environment after applying the FAST VP policies, which stabilized the workload as follows: The SQL DSS I/O pattern is a typical 64k read/write, with a read/write ratio of 100:0 percent for the data LUN. The SQL Server TempDB read/write ratio is 1:1.5, with a total of 1,141 IOPS. The average CPU use of the SQL virtual machine was less than 75 percent. With a balanced workload, SQL DSS recorded 12,018 IOPS, and the average bandwidth is at 808 Mb/s with a peak bandwidth of more than 1.1 Gb/s.
Table 13 shows the performance data with the SQL DSS workload.
Table 13. Performance data with SQL DSS load Ideal value < 75% Value 61.0% 808 1,141 10,896 1,122
DSS performance Average CPU use (%) Average disk bytes/sec (Mb/s) Maximum disk bytes/sec (Mb/s) DSS data LUN average IOPS TempDB data LUN average IOPS

43
Summary of results
The test results indicated the following: All four mission-critical applications are able to coexist and perform within service-level agreements. Although each application only used a small percentage of Flash storage, we saw high performance levels that FAST VP continued to fine tune to meet the applications requirements. FAST VP data movements are nondisruptive and transparent to all applications.

44
FAST VP workload tuning validation

Objectives It is important to configure FAST VP policies to adapt to changing application workloads. For example, a service provider may have customers who want higher performance levels with minimal impact on other applications. The purpose of this test is to tune FAST VP policies to meet the changing workloads of SQL OLTP applications and their on-the-fly storage group changes. This test validates how FAST VP rebalances the storage, making dynamic performance improvements as a result of this rebalance. Configuration The initial FAST VP policy for the SQL OLTP storage group is 5 percent Flash, 40 percent FC, and 100 percent SATA. The SQL OLTP storage group contains two child groupsMSSQL1_OLTP and MSSQL2_OLTP. All four applications (SAP, Oracle, SQL OLTP, and SQL DSS) ran at the same time. At 4 a.m., we increased the Flash percentage for the MSSQL policy to 30 percent. The load generators were left running on all applications and the environment was monitored. We also changed the FAST policy for the MSSQL_OLTP on the remote array to ensure the same performance impact was seen at the remote site. We used Unisphere to make the policy change. Figure 22 shows the policy changes in the FAST Policies management GUI.
Figure 22.
SQL FAST policy changes

45
Policy tuning results and analysis
After increasing the Flash percentage in the FAST VP policy for SQL OLTP, the other applications (SAP, Oracle and SQL DSS) were unaffected and continued to process their given workloads without interruption. Figure 23 shows each workload running during the policy change. The dotted line at 7:40 a.m. shows when the SQL OLTP FAST VP policy was changed. 30 minutes later, at 8:10 a.m., FAST VP responded by promoting more hot data to the Flash tier. As a result, the combined host IOPS for both MSSQL1 and MSSQL2 increased from around 7,000 to 12,000, with 71 percent more host I/O at the storage layer.
Figure 23.
Effects of tuning FAST VP policy on running workload
Figure 24 shows the SQL performance for the running workloads during FAST VP policy tuning.
Figure 24.
SQL performance changes during tuning FAST VP policy on running workloads

46
Figure 25 shows the combined effects of adjusting the FAST VP policy on the SQL OLTP applications. Both SQL OLTP applications showed improvement on all KPIs.
Figure 25.
Performance improvement of MS-SQL environment with FAST policy changes
After the policy change and balance, SQL1 processed 176 percent more transactions per minute, almost a 3x improvement, while SQL2 showed an improvement of over 50 percent. Both applications show 60 percent improvement in LUN latency. After increasing the Flash percentage in the FAST VP policy for SQL OLTP, the other applications (SAP, Oracle, and SQL DSS) remained unaffected and continued to process their given workloads without interruption, as shown in Table 14.
Table 14. Application summary Stable and KPIs met? (Before policy change) Yes Yes Yes During policy rebalance No impact No impact No impact Stable and KPIs met? (After policy change) Yes Yes Yes
Application Oracle OLTP SQL DSS SAP

47
Figure 26, Figure 27, and Figure 28 show the response times of the OLTP applications at maximum load during the test.
Figure 26.
SAP Average Response Time during testing
Figure 26 has three sections: before the FAST VP policy change (red region), during the change (yellow region), and after the change (blue region). The average and normal ranges were based on the red region and plotted as a baseline reference.
Figure 27.
Oracle performance statistics

48
Figure 27 shows the Oracle transaction rate. Tuning the SQL OLTP FAST policy on the fly did not have any noticeable impact on the database.
Figure 28.
SQL DSS latency (no impact on performance)
Summary of results
The tests shown here indicate the responsiveness of FAST VP in a multi-application environment. FAST VP is flexible and enables storage administrators to respond to the demand for increasing service levels by adjusting the mix of storage for an application. FAST VP is responsive: Testing showed that within 30 minutes of adjusting the FAST policy, performance increases were realized. Note that we made no changes at the application level during the test. We found no negative impact on other running applications as a result of the policy tuning.

49
RDF coordination with continuous production workload

Objectives The RDF coordination test demonstrates the operation of FAST VP with SRDF coordination. To validate SRDF coordination with FAST VP, we moved all applications to the FC tier on both arrays and completed a full SRDF establish operation to ensure a neutral starting point. To accomplish this, we used VLUN migration, which is a nondisruptive migration of data between storage pools. We monitored the tier usage of each application storage group and associated LUNs. Figure 29 shows the use of one of the Oracle devices at both R1 and R2 during the rebalance process. Figure 29 shows the RDF coordination at the LUN level.
RDF coordination overview
Figure 29.
FAST VP policy balance of LUN at production and target site
The solid lines represent the tier usage on R1 and the dotted lines represent the storage tiers on R2. At the start, all data resided on FC and FAST policies were associated with the applications. Figure 29 shows the allocation usage of tiers on R1 and R2 over the rebalance period. This test shows that capacity is being balanced across this device in a similar manner on both R1 and R2. The placement of R1 and R2 data is made by the FAST VP controller at both sites, based on the same performance data transferred across the SRDF links. The similar use of all three tiers at both the DR and production sites shows how effective FAST VP is at balancing both sites.

50
Figure 30 and Figure 31 show a detailed listing using the symcfg list command set.
Production array
Figure 30.
Detailed list of Oracle LUN 426 after FAST VP balance at production site
Figure 30 shows the LUN use of each tier in the associated policy on the production site. Figure 31 shows the LUN use at the DR site for the paired device. After the RDF synchronization and with the policy balanced, tier usage on both sites is very similar. The FAST VP engine on R2 makes its decisions based on the same information as the R1 array. The total written tracks on R1 and R2 have slight differences due to zero block detection on the RDF replication transfer. Zero block detection refers to the elimination of tracks containing all zeros due to file system or database formatting methods. These extents are not replicated between arrays, so conserving storage and bandwidth resources.
Disaster recovery
Figure 31.
Detailed list of Oracle LUN 426 after FAST VP balance at DR site
Test summary
When compared to similar usage tracked over time, all R1 and R2 devices show similar usage on a per-device basis after the FAST policy rebalanced the storage. Testing shows that SRDF coordination with FAST VP ensures that similar movement takes place at the production and DR sites.

51
Failover with performance continuity

Objectives To fully validate whether or not performance at the DR site will match that of the production site, we conducted a failover test. We failed over both Oracle and SAP application to the DR site. Note: In this representative failover test, only Oracle and SAP applications were running and failed over. The test methodology is as follows: SAP running 1,000 users and Oracle running 400 users at production site. Application load run period at the R1 site and FAST VP balanced storage groups on both arrays. SRDF synchronized state applications tier usage is balanced at both sites. When balance was achieved at both sites, we restarted the Oracle and SAP applications at the production site and measured the ramp-up time. RDF links slit to simulate failover. Failover simulated to the DR array. Applications started at DR site.
Failover test results
Figure 32 shows the ramp up of the Oracle/SAP workload at the production site. Ramp up for Oracle took approximately five minutes to generate 6,500 IOPS, and SAP took 35 minutes to load 1,000 users and reach 5,000 IOPS.
Figure 32.
Oracle and SAP running at the production site
At 3:20 a.m., we shut down the systems on the R1 site, initiating a failover to the remote site. The applications were mounted on the ESX servers at the remote site and the ramp-up times were noted. Figure 33 shows the workloads running at the remote site.

52
Within five minutes, Oracle ramped up to the same user load running at the production site prior to failover, generating more I/O than was seen prior to the failover. SAP observed the same ramp-up time of 35 minutes and also reached higher performance levels. The improved I/O throughput is due to the suspended replication; writes are local to the DR array and can therefore be serviced faster.
Figure 33.
Oracle and SAP running at the DR site
Figure 34 shows the number of transactions per minute and response time of the Oracle application at the source site, while synchronizing SRDF traffic to the R2 devices at the target site. It also shows the number of transactions per minute processed at the target site following the failover to the remote site. The chart shows that the transactions per minute and the response times are similar before and after failover.
Figure 34.
Oracle transactions per minute at the R1 and R2 sites

53
Figure 35 shows the performance of the SAP application. We observed slight improvements in the number of transactions per minute and response times.
Figure 35.
SAP transactions per minute and response times before and after failover
Test summary
This test shows that FAST VP with SRDF coordination ensures good performance in the event of a failover. After the storage and virtual machines failed over to the remote site, we observed that similar performance occurs on that site. The ramp-up times for applications were satisfactory at both the production and DR sites. We also observed that application KPIs were very close, and that they slightly exceeded production when failed over.

54
Conclusion
Summary This EMC solution has shown the implementation of multiple critical applications in a VMware private cloud environment hosted by VMAX 40K storage. Each application had different workload characteristics and placed varying demands on the underlying storage. The entire virtualized environment was protected with EMC SRDF to a secondary VMAX array at a remote site for disaster recovery. With SRDF coordination for FAST VP enabled, both sites were tuned to the recent performance statistics. This ensures that, in the event of a disaster, the remote site performance will match the levels seen at the production site. Our testing demonstrated the flexibility and responsiveness of FAST VP, which responded to policy changes and increasing performance within 30 minutes. If the workload (or its importance) changes, storage administrators can make adjustments that take effect in minutes without affecting other running applications. The new Unisphere for VMAX management and monitoring interface reduces provisioning complexity for managing and monitoring VMAX storage. New Enginuity 5876 features, such as cascaded storage groups and FAST VP allocate-bypolicy, along with the tight integration of EMC Virtual Storage Integrator for VMware vCenter, make provisioning and managing storage for virtual environments much simpler. These features enable storage and VMware administrators to work more effectively. Findings The key findings of the testing performed for this solution show that: VMAX 40K provides an ideal platform for critical virtualized database applications, running varied workloads simultaneously and managed by FAST VP. Cascaded storage groups enable easier management of multi-application ESXi environments for FAST and for masking. FAST VP allocate-by-policy reduces the complexity of capacity management. In FAST VP environments, data is placed in the most appropriate tier, based on performance requirements and available capacity in the storage tiers. FAST VP is easy to configure. It takes only a few simple steps to complete setup for automated performance tuning of the environment. FAST VP is tunable and responsive. By adjusting FAST policies, performance can be tuned to meet changing business performance needs. Increased performance can be seen within 30 minutes with no negative effect on other running workloads. SRDF coordination with FAST VP ensures production-level performance at the remote site in a FAST VP environment in the event of a failover, delivering all the benefits of FAST VP at both the production and DR sites. End-to-end visibility with EMC Virtual Storage Integrator for VMware vCenter enables simplified management and identification of EMC devices directly from VMware vCenter.

55
The NEC Express5800/A1080a is ideal not only for Web server and application server virtualization, but also for database server virtualization and other workloads that may require high I/O capabilities. NEC Express5800/A1080a has full support for Direct I/O. This feature, called VMDirectPath for vSphere, allows virtual machines direct access to physical NIC and HBA ports without using para-virtualized adapters.

56
References
White papers For additional information, see the white papers listed below.
Implementing Virtual Provisioning on EMC Symmetrix VMAX with Oracle Database 10g and 11gApplied Technology EMC Mission Critical Infrastructure for Microsoft SQL Server 2012 Provisioning EMC Symmetrix VMAXe Storage for VMware vSphere Environments Maximize Operational Efficiency for Oracle RAC with EMC Symmetrix FAST VP (Automated Tiering) and VMware vSphereAn Architectural Overview Best Practices for Fast, Simple Capacity Allocation with EMC Symmetrix Virtual ProvisioningTechnical Notes EMC Symmetrix Virtual ProvisioningApplied Technology FAST VP Theory and Practices for Planning and PerformanceTechnical Notes Best Practices for Fast, Simple Capacity Allocation with EMC Symmetrix Virtual ProvisioningTechnical Notes Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC Symmetrix VMAX Series Arrays EMC Storage Optimization and High Availability for Microsoft SQL Server 2008 R2
Product documentation Other documentation
For additional information, see EMC Solutions Enabler Symmetrix Array Controls CLI Version 7.4 Product Guide. For additional information, see the documents listed below.
NEC Express5800/A1080a Users Guide and NEC Express5800/A1080a-E Server VMware vSphere 5 Best Practices SQL Server Best Practices Article Oracle Grid Infrastructure Installation Guide 11g Release 2 (11.2) for Linux Oracle Real Application Clusters Installation Guide 11g Release 2 (11.2) for Linux Oracle Database Installation Guide 11g Release 2 (11.2) for Linux Oracle Database Storage Administrator's Guide 11g Release 2 (11.2)

57
Appendix A: Detailed application design and LUN layout

Overview This section describes the implementation details of the application environments and storage LUN configuration used in this solution. Table 15 lists the software layers of the SAP system architecture used in the landscape for the two data centers.
Table 15. Layer Application layer Database layer Operating system layer Simulator layer SAP landscape software layers Software SAP Enhancement Package 4 (EHP4) for SAP ERP 6.0 IDES SAP NetWeaver Application Server for ABAP release 7.01 Oracle 11g SUSE Linux Enterprise Server (SLES) for SAP Applications 11 SP1 HP Load Controller HP Load Runner Generator HP Virtual User Controller
SAP overview
All SAP and database instances are installed on VMware vSphere virtual machines. SAP ERP 6.0 SAP ERP 6.0, powered by the SAP NetWeaver technology platform, is a world-class, fully-integrated enterprise resource planning (ERP) application that fulfills the core business needs of midsize companies and large enterprises across all industries and market sectors. SAP ERP 6.0 delivers a comprehensive set of integrated, crossfunctional business processes and can serve as a solid business process platform that supports continued growth, innovation, and operational excellence. SAP IDES (Internet Demonstration and Evaluation System) systems support demos, testing, and functional evaluation based on preconfigured data and clients. IDES contains sample application data for various business scenarios, with business processes that are designed to reflect real-life business requirements and have access to many realistic characteristics. This solution uses IDES to represent a model company for testing purposes. HP LoadRunner software is the industry standard for performance validation. It helps prevent performance problems by detecting bottlenecks before a system deployment or upgrade. The Load Generation system provisioned for this test consists of the following tools:
SAP IDES
HP LoadRunner
Virtual User Generator, an application used to record and script the steps
performed by a user to simulate a virtual user processing an SAP transaction. The Controller, a server used to manage virtual users and monitor resources during the execution of the script prepared using the Virtual User Generator.

58
Load Generators, servers used to spawn virtual users and carry out the scripts against the SAP system to simulate multiple user transactions, thereby generating load.
SUSE Linux Enterprise Server for SAP applications
SUSE Linux Enterprise Server (SLES) is a highly reliable, scalable, and secure server operating system that is built to power physical, virtual, and cloud applications. SLES is a preferred Linux platform for SAP. SUSE Linux Enterprise Server for SAP Applications, based on the newest SLES technology, is optimized for all mission-critical SAP NetWeaver software solutions and appliances. SAP and SUSE validate and certify SLES for SAP Applications jointly to eliminate potential software incompatibilities. This partnership tightly integrates the application workload with the operating system and eliminates the possibility of incompatibilities when patches are applied to either the applications or the operating system. The solution implements a high-availability SAP system architecture, as shown in Figure 36.
SAP system architecture
Figure 36.
SAP system architecture

59
SAP landscape
The SAP system deployed for the solution implements these key design features: The SAP landscape was provisioned as a distributed system, where the following components were installed on their own virtual machines: ABAP central services instance Three application instances Database server
SAP patches, parameters, basic kernel and load balancing are all installed and configured according to the SAP installation guide and the SAP notes listed below. The three application servers were assigned to a logon group. This centralized the logon, and balanced multiple users logging on against a servers existing load and available resources. The application and database are optimized to achieve standard performance KPIs of up to 1,000 normal to heavy users, while maintaining a stable disk I/O rate of 3,000 to 4,000 IOPS. SAP performance tools were configured to exclude unnecessary front-end response times, caused by external front-end application loading and LoadRunner generator resource limits, to ensure accurate measurement of processing and database time. Work process allocation was adjusted to adapt to the workload simulation. Sample data and configurations pre-packaged in SAP IDES were used for transactional load simulation Oracle archive logging was enabled to simulate disk I/O in a production environment. To prevent the archive disk from filling up during testing, a periodic chron job was created to purge these files. VMware best practices for SAP are adopted in this solution. SAP update processes (UPD/UP2) are configured on the additional application server instances. The storage for the entire SAP environment is encapsulated and virtualized for this solution.
Table 16 lists the Oracle configuration for the SAP landscape.

Table 16. SAP landscape Oracle configuration Values set 160 1,004 Remarks PARALLEL_MAX_SERVERS= #DB-CPU-Cores * 10. fyi, CPU_COUNT=16 PROCESSES = #ABAP work processes * 2 + #J2EE server processes * <max-connections> + PARALLEL_MAX_SERVERS + 40 SESSIONS = 2 * PROCESSES
Oracle parameters PARALLEL_MAX_SERVERS PROCESSES
SESSIONS
2,008

60
Oracle parameters db_cache_size shared_pool_size ARCHIVELOG sga_max_size sga_target
Values set 4 GB 6 GB ON 30 GB 30 GB
Remarks 1% of database size
A chron cleanup script was run every 5 minutes to prevent disks from getting full Max size is amount of physical RAM Best practice to set equal to sga_max_size
Table 17 lists the SAP landscape work process configuration.

Table 17. SAP work process configuration Number 70 50 10 0 3 1 Remarks Same in all servers Same in all servers Same in all servers Same in all servers Same in all servers Same in all servers
Work process DIA UPD UP2 ENQ BGD SPO
Table 18 lists the SAP notes that were accounted for in the SAP landscape.
Table 18. References Note 923610 Note 1171650 Note 1431798 Note 617416 Note 789011 Note 618868 Note 1073521 SAP notes Title Memory parameter recommendations for banking systems Automated Oracle DB parameter check Oracle 11.2.0: Database Parameter Settings Oracle9i: Dynamic SGA FAQ: Oracle memory areas FAQ: Oracle performance Response time without GUI time II

61
LoadRunner landscape
For this simulation, we used the following specifications: LoadRunner will generate a total of 1,000 concurrent users in total, distributed among four generators located on the site opposite the SAP servers, running 250 users each 40 users will logon simultaneously to the same SAP message server every minute The simulation will last for 12 hours. Each virtual user will create service delivery orders and other documents continuously for 100 iterations or until the simulation time is reached.
SAP LUN Configuration
Table 19 shows the LUN configuration for the SAP application.

Table 19. Purpose SAP DB OS Oracle Bin SAP mirror log SAP orig log SAP Archive Log SAP trace SAP Bin SAP data1 SAP data2 SAP data3 SAP data4 SAP data5 SAP data6 SAP LUN configuration LUN size 128 32 32 32 64 64 32 64 64 64 64 64 64 Quantity required 1 1 1 1 1 1 1 3 3 3 3 3 3 Total (TB) Capacity (GB) 128 32 32 32 64 64 32 192 192 192 192 192 192 1.5
Oracle Database 11g R2
Oracle Database 11g Release 2 Enterprise Edition delivers industry-leading performance, scalability, security, and reliability on a choice of clustered or single servers running Windows, Linux, or UNIX. It provides comprehensive features for transaction processing, business intelligence, and content management applications.

62
Oracle ASM
ASM is a volume manager and a file system for Oracle database files that supports single-instance Oracle Database and Oracle Real Application Clusters (Oracle RAC) configurations. ASM is Oracle's recommended storage management solution that provides an alternative to conventional volume managers, file systems, and raw devices. ASM uses disk groups to store data files. An ASM disk group is a collection of disks that ASM manages as a unit. Within a disk group, ASM exposes a file system interface for Oracle database files. The content of files that are stored in a disk group are evenly distributed, or striped, to eliminate hot spots and to provide uniform performance across the disks. The performance is comparable to the performance of raw devices.
Oracle grid infrastructure
For this solution, Oracle Grid Infrastructure was installed with the Standalone Server option. The Oracle Grid Infrastructure for a standalone server is the Oracle software that provides system support for an Oracle database including volume management, file system, and automatic restart capabilities. If you plan to use Oracle Restart or Oracle Automatic Storage Management (Oracle ASM), then you must install Oracle Grid Infrastructure before you install and create the database. Oracle Grid Infrastructure for a standalone server is the software that includes Oracle Restart and Oracle ASM. Oracle combines the two infrastructure products into a single set of binaries that is installed as the Oracle Grid Infrastructure home.
Oracle database and workload profile
Table 20 details the Oracle database and workload profile for the solution.
Table 20. Oracle database and workload profile Details 2 TB Oracle database 11g R2 single instance Oracle Automatic Storage Management (ASM) 40 GB OLTP SwingBench Order Entry (TPC-C-like) workload Transactions per minute (TPM) 60/40 400
Profile characteristic Database size Database version Storage type Oracle system global area (SGA) Workload type Workload profile Database metric Workload read/write ratio SwingBench sessions

63
Oracle workload description
The SwingBench Order Entry - PL/SQL (SOE) schema was used to deliver the OLTP workloads required by the solution. SwingBench consists of a load generator, a coordinator and a cluster overview. The software enables a load to be generated and the transactions/response times to be charted. Order Entry schema introduces heavy contention on a small number of tables and is designed to stress interconnects and memory. It is installed using the oewizard located in the bin directory. Both a pure jdbc and a pl/sql (lower network overhead) variant exist for the benchmark. To establish an optimal configuration, testing was carried out with 400 users accessing order entry schema. Think time is used to simulate an amount of time that a user would think about the results of a previous transaction. This could be time spent performing analysis on the results of a database query. SwingBench allows selecting a variable think time (delay) between transactions. Figure 37 shows the Load panel on the SwingBench interface, with these parameters set.
Figure 37.
SwingBench Load panel
Oracle database schema
Two identical schemas were used to deliver the OLTP workloads required by the solution: schemas SOE1 and SOE2. A SwingBench Order Entry workload was generated and run against schema SOE1. Since the I/O distribution across the database was completely even and random, this reduce sub-LUN skewing (since the whole database was highly active), therefore the second schema SOE2 was left idle to simulate a more normal environment where some objects are not highly accessed. Table 21 lists the tables and indexes for the SOE schema used in this solution.
Table 21. SOE schema Index CUSTOMERS_PK (UNIQUE), CUST_ACCOUNT_MANAGER_IX, CUST_EMAIL_IX, CUST_LNAME_IX, CUST_UPPER_NAME_IX INVENTORY_PK (UNIQUE), INV_PRODUCT_IX, INV_WAREHOUSE_IX ORDER_PK (UNIQUE), ORD_CUSTOMER_IX, ORD_ORDER_DATE_IX, ORD_SALES_REP_IX, ORD_STATUS_IX ORDER_ITEMS_PK (UNIQUE), ITEM_ORDER_IX, ITEM_PRODUCT_IX PRD_DESC_PK (UNIQUE), PROD_NAME_IX PRODUCT_INFORMATION_PK (UNIQUE), PROD_SUPPLIER_IX WAREHOUSES_PK (UNIQUE)
Table name CUSTOMERS INVENTORIES ORDERS ORDER_ITEMS PRODUCT_DESCRIPTIONS PRODUCT_INFORMATION WAREHOUSES

64
Table name LOGON
Index
Oracle database services
Database services are entry points to an Oracle database that enable the management of workloads across the cluster. They are a logical grouping of similar user sessions that can be dynamically assigned to and between individual or multiple Oracle RAC nodes and instances. For this solution, each of the test schemas has a corresponding database service mapped to it as shown in Table 22. This enables monitoring of the Oracle I/O to be mapped to each individual service and hence schema.
Table 22. Schema SOE1 SOE2 Oracle database services Service SOE1.oracledb.ie SOE2.oracledb.ie Instance orafast orafast SwingBench sessions 400 0
Oracle LUN configuration
Table 23 lists the LUN configuration for the Oracle application.

Table 23. Oracle LUN configuration TDev hyper size 64 GB 64 GB 64 GB 64 GB 64 GB 64 GB Number of TDevs 2 15 15 2 1 1 Capacity (GB) 128 GB 960 GB 960 GB 128 GB 64 GB 64 GB
ASM disk group +DATA +SOE1 +SOE2 +FRA +TEMP +REDO
Microsoft SQL Server
Microsoft SQL Server 2012 is the latest version of Microsoft database management and analysis system for e-commerce, line-of-business, and data warehousing solutions. In the test environment, the following two applications generate the different workload patterns running on the Microsoft SQL Server 2012 enterprise class platform: A TPC-H -like application, acting as a typical DSS application A TPC-E -like application, acting as a typical OLTP application
SQL Server 2012 DSS workload
SQL Server 2012 DSS workload emulates real-world data warehouse queries. Data warehouses need to support a broad range of queries from a wide-ranging audience (for example: finance, marketing, operations, and research teams). Data warehousing can involve much larger data requests and benefit greatly from the increased total throughput potential of sequential disk scans.

65
The test environment for the SQL Server 2012 DSS application is based on a TPCH -like workload. The TPC-H -like application models the analysis end of the business environment where trends are computed and refined data are produced to support the making of sound business decisions. In TPC-H -like application, periodic refresh functions are performed against a DSS database whose content is queried on behalf of or by various decision makers. SQL Server 2012 DSS LUN configuration Table 24 shows the LUN utilization for MS SQL DSS LUNs, and how they were used. VM1 and VM2 were hosting different size databases so smaller LUNs were used for root, file groups and log devices.
Table 24. Purpose SQL DSS VM data store filegroup1-8 Log TempDB Log TempDB Table LUN use for MS SQL DSS Quantity of LUNs 1 8 1 1 8 Meta size 8 8 1 1 8 Total (TB) LUN capacity (GB) 128 480 64 32 480 2.912
SQL Server 2012 OLTP workload
The test environment for SQL Server 2012 OLTP application is based on a TPC-E-like workload. It is composed of a set of transactional operations simulating the real workload online transaction processing. The OLTP application are primarily heavily indexed to support low latency retrieval of small numbers of rows from data sets that often have little historical data volume. These types of database operations induce significant disk head movement and generate classic random I/O scan patterns. Table 25 shows the LUN use for MSSQL LUNs and how they were used. VM1 and VM2 were hosting different size databases so smaller LUNs were used for root, file groups, and log devices.
Table 25. Purpose SQL OLTP VM data store tpce_root filegroup1-8 (Fixed, Growing, and Scaling) Log TempDB Log TempDB MS SQL LUN use Quantity of LUNs 1 1 8 1 1 4 Total (TB) VM1 capacity (GB) 128 64 128 256 64 256 1.75 VM2 capacity 128 32 64 128 64 256 1.12
SQL Server 2012 OLTP LUN configuration

66
SQL Server 2012 OS & SQL instance settings and Windows 2008 For the SQL Server 2012 test OSWindows 2008 R2: R2 settings for Use Large Page for SQL instance by enabling -834 startup parameters. DSS and OLTP workload Use Lock page in memory for SQL instance. Use 64K format unit for all data and log LUNs.
DB settings For user databases: Use multiple data filesUse 16 data files on 16 TDevs for DSS user database. Use multiple data filesUse eight data files on eight TDevs for OLTP user database. Disable the autogrow option for data files, and manually grow all data files when the current size limit is being approached.
For TempDB: Use six data file on six TDevs for the TempDB of the DSS instance Use two data files on two TDevs for the TempDB of the OLTP instances. Pre-allocate space, and add a single data file per LUN. Be sure to make all files the same size. Assign temp log files onto one of the LUNs dedicated to LOG files. Enable autogrowIn general the use of a large growth increment is appropriate for DW workloads. A value equivalent to 10 percent of the initial file size is a reasonable starting point. Follow standard SQL Server best practices for database and tempdb sizing considerations. Greater space allocation may be required during the migration phase or during the initial data load of the warehouse. For more information, see Capacity Planning for tempdb in SQL Server Books Online.
For the transaction log, do the following: Create a single transaction log file per database on one of the LUNs assigned to the transaction log space. Spread log files for different databases across available LUNs or use multiple log files for log growth as required. Enable the autogrow option for log files.

67
Appendix B: Configuring Symmetrix remote replication

Configuring SRDF for remote replication Both the source and target Symmetrix arrays in this configuration are configured with the same number of disks and storage tiers. Devices and storage groups were created using same names on both sites, and provisioned to the DR servers that were zoned to the target array. After creating the devices on both sites, we created the SRDF groups and paired the devices. A SRDF group defines the logical relationship between SRDF devices and SRDF directors on both sides of the SRDF links. It comprises a range of SRDF devices and SRDF directors that reside on a given Symmetrix system. The storage administrator must configure these RDF groups before creating RDF replica pairs. Separate SRDF groups were configured for each application in this configuration. The use of separate SRDF groups is not required but does increase flexibility when failing over and back and adjusting SRDF mode of operation at an application level. Figure 38 shows the command line syntax for creating an SRDF group.
Figure 38.
SRDF group creation using the CLI
In this instance, directors 8H and 10H were used for all RDF traffic on both arrays. Once the RDF group has been created, a pairs file can be set up to configure the source target pairs for the replication. Since the devices we used were all labeled with device identifiers, it was a simple matter to obtain a list of devices from both arrays and pairing them in a text file. Figure 39 shows the syntax used to pair the Oracle devices.
Figure 39.
Pair Oracle devices using the CLI
SRDF device groups were created on the management host to manage each applications SRDF relationship. Figure 40 shows the commands used to create the SRDF device group for the SAP device group.
Figure 40.
Create SRDF device group for SAP using the CLI

68

h10568 Vmax 40k Fast VP Vmware Cloud WP

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

h10568 Vmax 40k Fast VP Vmware Cloud WP

Загружено:

Авторское право:

Доступные форматы

White Paper

EMC INFRASTRUCTURE FOR VMWARE CLOUD ENVIRONMENTS

EMC Solutions Group

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

These elements are described in more detail in subsequent sections.

EMC Infrastructure for VMware Cloud Environments

Physical architecture diagram

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

EMC Virtual Provisioning

Symmetrix Array Controls CLI v7.4 Product Guide.

EMC Infrastructure for VMware Cloud Environments

EMC Symmetrix Remote Data Facility (SRDF)

EMC Unisphere for VMAX

EMC Infrastructure for VMware Cloud Environments

Figure 2 shows the Unisphere for VMAX user interface.

Unisphere for VMAX interface

EMC Infrastructure for VMware Cloud Environments

EMC Symmetrix FAST VP

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

FAST VP with OLTP workload

FAST VP with DSS workload

FAST VP with SRDF

FAST VP SRDF coordination

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

VMware vSphere 5.0

EMC Infrastructure for VMware Cloud Environments

Production (Site A) and DR (Site B) shown in vCenter

VMware virtual machine configuration

Application Oracle SAP

EMC Infrastructure for VMware Cloud Environments

Application Microsoft SQL OLTP

Virtual machine name SQLTPCE01 SQLTPCE02 Domain controller

Memory size 32,768 MB 32,768 MB 4,096 MB

EMC Infrastructure for VMware Cloud Environments

Data store and storage pool information viewed from VSI

EMC Virtual Storage integrator LUN view

EMC Infrastructure for VMware Cloud Environments

Overview of mission-critical applications deployed in this solution

EMC Infrastructure for VMware Cloud Environments

EMC Infrastructure for VMware Cloud Environments

Storage design for consolidation of applications on VMAX 40K

Logical grouping of ports to applications

EMC Infrastructure for VMware Cloud Environments

Thin pool configuration

Thin pool name FLASH_3RAID5 FC10K_RAID1 FC15K_RAID1 SATA_6RAID6

EMC Infrastructure for VMware Cloud Environments

Storage group MSSQL1_OLTP MSSQL2_OLTP MSSQL_DSS Oracle SAP

General view of FAST VP configuration for mission-critical applications

EMC Infrastructure for VMware Cloud Environments

Configuring EMC Symmetrix FAST VP with Unisphere for VMAX

EMC Infrastructure for VMware Cloud Environments