SAP Note

1122388 - Linux: VMware vSphere configuration guidelines

Version 32 Validity: 15.04.2014 - active

Language English (Master)

Header Data
Released On
Release Status
Component

15.04.2014 11:45:46
Released for Customer
BC-OP-LNX-ESX Linux on VmWare ESX

Other Components BC-OP-LNX Linux

Priority
Category

Recommendations / Additional Info

Installation information

Symptom
You are setting up your server with VMware vSphere and need some configuration recommendations.

Other Terms
Virtualization, VMware, Linux, SLES, RHEL, ESX, ESXi, vSphere

Reason and Prerequisites

SAP supports VMware ESX 3.5, vSphere 4.x and vSphere 5.x
VMware provides the following recommendations for these supported platforms if you intend to run SAP applications on it.

Solution
For best performance, we recommend the following guidelines:
1. We highly recommend to use the latest processor generations, due to their enhanced support for virtualization. Especially the feature EPT
(Intel Nehalem microarchitecture based XEON and successors) and RVI (AMD 10h microarchitecture based Opteron and successors)
enhance greatly the performance of Virtual Machines.
2. For optimal results we recommend to size the VMs within the NUMA node boundaries of the specific hardware. If a VM doesn't fit into the
NUMA node, we recommend to use vSphere 5.x Virtual NUMA configuration which reflects the underlying hardware.
3. We encourage to enable hyper-threading technology. If an SAP workload which relies on high single computing unit performance cannot
achieve satisfying performance, set the "Hyperthreaded Core Sharing" option to "none" for that specific VM.
4. To balance and limit available CPU resources across virtual machines use resource pools. Avoid to set limits separately for each VM as
this may result in increasing response times.
5. Follow SAP's rules for setting up file systems for your database installation. Distribute database files over different disks. Separate log
and db files. Use at least two separate VMFS (VMware Filesystem) volumes to create the virtual disks.
6. We highly recommend to use multiple virtual SCSI Controllers for the Database VM and VMs with high I/O load. The use of multiple virtual
SCSI Controllers allows the execution of several parallel I/O operations inside the Guest OS. We also highly recommend to separate the Log
I/O traffic from the Datafile I/O traffic through separate virtual SCSI Controllers. As a Best Practice, you can use one Controller for OS and
Swap, one Controller for DB Log, and one or more Controllers for DB Datafiles (depending on the amount and size of DB Datafiles). Please
refer to the VMware Administration Guide on how to add additional virtual SCSI Controllers.
7. Virtual disks are created in "zeroedthick" format by default. This can lead to write performance degradation. Therefore, the format
"eagerzeroedthick" should be chosen. See VMware KB 1022242 for details: http://kb.vmware.com/kb/1022242. To determine which format
a virtual disk currently has, consult VMware KB 1011170: http://kb.vmware.com/kb/1011170
8. For optimal throughput, use the paravirtual SCSI Controller (PVSCSI). Please make sure that you install the VMware Tools first. See
VMware KB 1010398 for details: http://kb.vmware.com/kb/1010398
9. For best network performance, use vmxnet3 network card. See VMware KB 1001805 for details: http://kb.vmware.com/kb/1001805.
Additionally, consult the documents "Network I/O Latency in vSphere 5": http://www.vmware.com/resources/techresources/10256 and "Best
Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs":
http://www.vmware.com/resources/techresources/10220
10. Edit the MAC address of your network adapter to a manually set MAC address. The VMware Universally Unique Identifier (UUID)
generates MAC addresses that are checked for conflicts. The generated MAC addresses are created by using three parts: the VMware OUI,
the SMBIOS UUID for the physical ESXi machine and a hash based on the name of the entity that the MAC adress is being generated for.
After the MAC address has been generated, it does not change unless the virtual machine is moved to a different location. The MAC
address in the configuration file of the virtual machine is saved. All MAC addresses that have been assigned to network adapters of running
and suspended virtual machines on a given physical machine are tracked. The Mac address of a powered off virtual machine is not checked
against those of running or suspended virtual machines. It is possible that when a virtual machine is powered on again, it can acquire a

different Mac address. This acquisition is caused by a conflict with a virtual machine that was powered on when this virtual machine was
powered off. This can lead to an invalid SAP license. Please refer to the VMware KB 507 for details: http://vmware.com/kb/507
11. Don't do memory over commit! With VMware it is possible to assign more virtual memory to the VMs than there is physical memory
availableinthehost.ThisisnotrecommendedatallwithSAP.SAPallocatesmemorypermanentlyanddoesnotreleaseitagain.To
enforce this policy, we recommend setting the "Memory Reservation" to the amount of memory configured for the VM. This will ensure that the
VM with the SAP instance will always have the full amount of memory available. The same can be done for the "CPU Reservation", but we
recommend enabling this only in case of performance problems, since the reserved resources need to be available on other hosts when
using DRS or VMware HA.
12. VMware ESX allows an over-commitment of available CPUs. SAP successfully ran performance tests in virtual machines (utilizing all
available virtual CPUs to 100%) which over committed the host system up to 150% (200% in vSphere 4). The performance degradation
inside the virtual machines was linear reciprocal to the over commitment. You may exceed the over-commitment, but keep in mind that the
performance of Virtual Machines in such a scenario is not guaranteed. In case of performance problems, SAP can demand you to shutdown
or pause other running Virtual Machines to check if the over-commitment caused problems.
13. In order to avoid time syncronisaton errors we highly recommend to check the instructions mentioned in SAP note 989963. We also
HIGHLY recommend to use at least Novell SLES 9 and Red Hat RHEL 5.1 since they allow reducing the amount of timer interrupts.
14. Set kernel parameter "elevator=noop" for Virtual Machines. This disables I/O scheduling. The VMware hypervisor has its own I/O
scheduling mechanisms, therefore scheduling I/O inside the Guest Operating System can cause unnecessary overhead. For details, see Red
Hat KB http://kbase.redhat.com/faq/docs/DOC-5428 and SUSE KB http://www.novell.com/support/kb/doc.php?id=7009616 (applies also to
other SLES versions than the mentioned).
15. Installing VMware Tools is mandatory. The VMware Operating System Specific Packages (OSPs) can be used
(http://packages.vmware.com). If you use the standard proprietary VMware Tools which are part of the ESX server, you load modules which
taint your Linux Kernel. Although SAP Note 784391 prohibits using such modules, the use of the standard VMware Tools is tolerated and will
be supported by SAP.
16. SAP requires the implementation of the new virtualization aware monitoring. Please see SAP Note 1102124.
17. Choose the optimum SAP Memory Model For latest processor generations (see point 1.) we recommend to use the "std" memory model
(es/implementation = std) which is the default from NetWeaver 7.10 onwards (SAP Note 941735). For older processors we highly
recommend to use the "map" memory model (es/implementation = map). This memory model gives much better memory throughput on older
processors than the "std" model (SAP Note 386605).
18. When using VMware snapshots, consult VMware KB 1025279 (http://kb.vmware.com/kb/1025279). Make sure that the Virtual Machine
has no active snapshot before reporting performance problems.
In regards to all other aspects, you can treat the VM as any other host. Follow the installation and tuning guidelines from SAP for best results.
Technical Resources at the SAP Community Network:
http://scn.sap.com/docs/DOC-27384

Validity
This document is not restricted to a software component or software component version

References
This document refers to:
SAP Notes
1158363 VMware Performance Snapshot
1122387 Linux: SAP Support in virtualized environments
1070760 Running a Virtual Machine (VM) and MDM
989963 Linux: VMware timing problem
171356 SAP software on Linux: General information

This document is referenced by:

SAP Notes (5)
171356 SAP software on Linux: General information
1070760 Running a Virtual Machine (VM) and MDM
989963 Linux: VMware timing problem
1158363 VMware Performance Snapshot
1122387 Linux: SAP Support in virtualized environments