SQL Server on VMware

Jonathan Kehayias (MCTS, MCITP) SQL Database Administrator Tampa, FL

Agenda
SQL Initiative Overview Performance Metrics
I/O Metrics Processor Performance Counters

High Availability Configuration

Manageability Gains

SQL Initiative
Primary Objective
Disaster Recovery

Success Criteria
Immediate failover to remote data center with minimal data loss Performance

Performance VMware ESX Server, 3.0.2, 63195

GENERAL Manufacturer: Model: Processors: Processor Type: Hyperthreading: Total Memory: Number of NICs: Virtual Machines: VMotion Enabled: HP ProLiant BL460c G1 4 CPU x 2.666 GHz Intel(R) Xeon(R) CPU 5150 @ 2.66GHz Inactive 24.00 GB 4 7 Yes

Performance SQL Virtual Machine Configuration

GENERAL Guest OS: CPU: Memory: Memory Overhead: Disk Configuration: C/D F G Microsoft Windows Server 2003 R2 Standard x64 Edition 2 vCPU 4096 MB 263.96 MB RAID 6+2 50GB XP10K 350GB XP10K 350GB XP10K

Performance SQLIO Benchmarks SQLIO is a tool provided by Microsoft which can also be used to determine the I/O capacity of a given configuration. SQL Server stores data 8K pages allocated in blocks of 8 as 64K extents. Typical SQL I/O operations involve Random Reads of extenteRs from disk. SQLIO benchmarks on this SQL Server for 64K Random Read I/O with 2 threads simulating the recommended setting of one file per processor core for SQL Server were equivalent or better than common physical hardware.

Performance (VMWare) Random Read I/O By The Numbers

Drive Format Test READ Default Read 8KB random IOs/sec MBs/sec

2286.33

17.86

Default Read 64KB random

Default Read 128KB random Default Read 256KB random Default Read 8KB sequential Default Read 64KB sequential Default Read 128KB sequential Default Read 256KB sequential WRITE Default Write 8KB Random Default Write 64KB Random Default Write 128KB Random Default Write 256KB Random Default Write 8KB Sequential Default Write 64KB Sequential Default Write 128KB sequential Default Write 256KB sequential

1940.86
1059.25 500.54 2549.82 1397.24 766.72 466.00 1475.78 609.97 366.03 210.17 1560.25 634.69 222.10 165.52

121.30
132.40 125.13 19.92 87.32 95.84 116.50 11.52 38.12 45.75 52.54 12.18 39.66 27.76 41.38

Dell PowerEdge 6650 2x 2.7 GHz 2GB RAM 4x 36GB RAID 10 array

Drive Format Test READ Default Read 8KB random

IOs/sec

MBs/sec

656

5.1

Default Read 64KB random

Default Read 128KB random Default Read 256KB random Default Read 8KB sequential Default Read 64KB sequential Default Read 128KB sequential Default Read 256KB sequential WRITE Default Write 8KB Random Default Write 64KB Random Default Write 128KB Random Default Write 256KB Random Default Write 8KB Sequential Default Write 64KB Sequential Default Write 128KB sequential Default Write 256KB sequential

258
149 82 3246 378 188 94 1567 809 486 333 7112 1889 977 494

16.1
18.6 20.4 25.4 23.6 23.5 23.5 12.2 50.5 60.7 83.2 55.6 118.1 122.1 123.5

Physical SQLIO outputs were obtained from the SQL Server Performance website Forums: http://sql-server-performance.com/Community/forums/t/15370.aspx

Dual 3.0Ghz Xeon Dual 2GB FC HBA IBM DS4300 SAN RAID 10 (10 x 74GB 15K)

Drive Format Test SQLIO 8k sector Read - Random Read - Sequential Write - Random Write - Sequential SQLIO 32k sector Read - Random Read - Sequential Write - Random Write - Sequential SQLIO 64k sector Read - Random Read - Sequential Write - Random Write - Sequential

IOs/sec 2096.15 3651.36 1985.08 1914.68 1653.85 2736.05 1125.63 1337.12 1280.91 1889.55 724.24 889.01

MBs/sec 16.38 28.53 15.51 14.96 51.68 85.50 35.18 41.79 80.06 118.10 45.27 55.56

Physical SQLIO outputs were obtained from the SQL Server Performance website Forums: http://sql-server-performance.com/Community/forums/t/15370.aspx

DELL 6850 Quad Xeon 3.4ghz w/4GB RAM EMC Clariion CX-700 RAID 10

Drive Format Test 8k random write: 64k random write: 128k random write: 256k random write: 8k seq. write: 64k seq. write: 128k seq. write: 256k seq. write: 8k random read: 64k random read: 128k random read: 256k random read: 8k seq. read: 64k seq. read: 128k seq. read: 256k seq. read:

IOs/sec 14884.96 1573.03 1110.79 722.48 16545.36 3142.00 1573.62 787.21 5776.05 2857.16 1567.23 787.22 9200.54 2805.23 1506.95 783.07

MBs/sec 116.28 98.31 138.84 180.62 129.26 196.37 196.70 196.80 45.12 178.57 195.90 196.80 71.87 175.32 188.36 195.76

Physical SQLIO outputs were obtained from the SQL Server Performance website Forums: http://sql-server-performance.com/Community/forums/t/15370.aspx

Performance Performance Counter Monitoring

To understand how well a SQL Server is performing the SQL Server as well as Windows Subsystem Performance Counters need to be Monitored. Key Counters to monitor include: Processor/%Processor Time should remain below 80%. Processor/%Privileged Time should remain below 20% SQL Server General/User Connections Batches/Sec Page Life Expectancy Pages/Sec Memory Grants Pending Lazy Writes/sec For further information, take a look at the following Screencast series by Kevin Kline (SQL Server MVP and Professional Association for SQL Server (PASS) President): http://searchsqlserver.techtarget.com/generic/0,295582,sid87_gci1302600,00.html

Performance Performance Counters (%Processor Time)

< 80% average

Performance
Performance Counters (Processor\%Privileged Time)

< 20% Average

Performance Performance Counters (User Connections)

This is just a reference counter to be used in tandem with other counters to view system activity.

Performance Performance Counters (Batches/sec)

This is a reference to the amount of activity the Server is performing. It is used along with other counters like Page Splits/sec to determine if there are problems.

Performance Performance Counters (Buffer Cache)

Should remain as close to 100% as possible. Consistent drops below 95-90% signals Memory Pressure

Performance Performance Counters (Page Life Expectance)

Page Life in the cache should ideally remain over 300 seconds. Consistent drops below this should be investigated and can signal Memory Pressure on the server.

Performance Performance Counters (Memory pages/sec)

The rate at which pages are read from or written to disk. If > 100 on a slow disk subsystem or > 600 on a fast disk subsystem it should be investigated.

Performance Conclusions
1: Compared to various disk configurations of physical implementations with local storage, we experience 10x performance for disk subsystem I/O 2: Critical performance counters SQL maintains industry acceptable performance 3: Ability to consolidate multiple VMs along with SQL server, ~5 to 7:1 to save costs on physical hardware, rack space, power, cooling and integrate into DR plan.

Consolidation Architecture
Single SQL server per ESX host Multiple SQL Servers never reside on the same ESX host Logical placement of VMs to eliminate contention of resources Web and App server never communicate with the SQL server on the same host

App Server

Web Server

App Server

Web Server

App Server

Web Server

App Server

Web Server

Average 5-7 to one consolidation ratio

High Availability
SAN Storage Mirrored for Disaster Recovery to Chicago Datacenter. Quarterly Failover Tests of key SQL Servers on Chicago Network with zero data loss at failover.

High Availability
VMotion
Eliminates need for Mirroring Solutions for Hardware redundancy. In the event of a host hardware failure the Virtual Machines can be hot migrated to another host

Allows live migration of Servers during high load operation to shift load to more powerful hosts as needed.

Manageability
Snapshot Technology
Provides a point in time recovery point for risky operations such as upgrading Server OS and or SQL Server.

Hot Add Disk Arrays

Allows Zero Down Time additions of new Storage LUNS as database grow in size

HBA Load Balancing

Allows Disk I/O Load balancing across redundant paths to the SAN storage.

Manageability
Rapid Scale Up
Adding additional vCPUs and Memory is only a reboot away, provided the host has available resources. Easily upsize a server for end of month processing when it requires the most power while minimizing its footprint while under minimal load.

Manageability
VMWare Infrastructure Client
Immediate shared console level access to the SQL Server provides remote administration and rapid response for critical server outages.
No risk of a BSOD requiring physical access to the server or of a stuck ILO interface on a server. Integrated performance monitoring of processor, memory, disk and networking counters.

Manageability
VMWare Infrastructure Client (contd)
Running history of Events occurring through the client including Server Resets, Migrations, and Reconfigurations. Integrated Alarms are configurable for out of tolerance counter statuses.

Lessons Learned
1: Dont lock pages in memory if you plan to do VMotion. 2: Be cautious when implementing Microsoft Recommended Best Practices that affect system configurations and test in Development before deploying in Production. Some may not be compatible with VMotion. 3: Dont accept a vendors statement that VMware is the problem look deeper, and you can generally disprove this statement.

Conclusions
Performance Metrics
I/O Metrics 10x performance, Processor and Performance Counters

High Availability Configuration

VMotion to load balance during end of month reconciliations keeps ahead of the business Reduced downtime with hot add disk

Manageability Gains
SOX compliance for Disaster Recovery Virtual Center Console for team collaboration

Cost Avoidance
Reduces need for Mirroring hardware and software Consolidation of hardware both primary and DR