<Insert Picture Here>

InnoDB: Status, Architecture and New Features

MySQL Conference and Expo, 2010
3:05 pm, April 13, 2010

Calvin Sun, Principal Software Engineer, Oracle/InnoDB

Rob Young, Sr. Product Manager, Oracle/MySQL
 Agenda

• What’s New Since Last UC?

• InnoDB Architecture

2
 Make MySQL a better MySQL
• Where were you on 2005-Oct-7 at 5:14pm EDT?

• 81% of MySQL shops use InnoDB

• Oracle’s commitment
• Be the technology leader for InnoDB/MySQL
• Community involvement
• Strategic, integrated product direction
• Align Engineering/Product teams
• The best technical support. Period.

• Goal = “Make MySQL a better MySQL”

• First fruits in MySQL 5.1, 5.5

3
 From UC 2009
InnoDB 1.0 Announced!
• Barracuda FILE FORMAT (more animals to come…)
• Default for executing threads = unlimited
• Faster Index Creation
• Efficient Data Compression
• Off-page storage options for Large Objects, Variable
Length Columns
• New INFORMATION_SCHEMA tables
• More…

4
 Making MySQL 5.1 a better MySQL GA!

InnoDB 1.0 is now option in MySQL 5.1.46

Performance/Availability improvements (since 1.0.3):
• Better control of Spin Lock polling
• Dynamic control of insert buffering, adaptive hash indexing
• Group commit is back!
• Improved read ahead algorithms
• Enabled multiple background threads
• Making buffer cache scan resistant
• Improved crash recovery performance
• More…
Better Instrumentation/Diagnostics
• Optimized SHOW ENGINE INNODB MUTEX

5
 MySQL 5.1 Pluggable Architecture GA!

MySQL Server
Enterprise
Management Tools and
Utilities

MySQL Enterprise Monitor

MySQL Query Analyzer
MySQL WorkBench
Backup & Recovery
Security
Replication
Cluster
Partitioning
INFORMATION_SCHEMA
PERFORMANCE_SCHEMA

CSV

1.0
6
 Options for Installing/Enabling in 5.1 GA!
My.cnf
ignore_builtin_innodb
plugin-load=innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so;
...

Command Line
mysqld --ignore-builtin-innodb
--plugin-load=innodb=ha_innodb_plugin.so;
...

INSTALL PLUGIN
INSTALL PLUGIN INNODB SONAME 'ha_innodb_plugin.so';
...

7
 Making MySQL 5.5 a better MySQL
Beta
InnoDB 1.1 is a better InnoDB 1.0!
Performance improvements (relative to 1.0):
• Multiple Buffer Pool Instances
• Multiple Rollback Segments
• Extended Change Buffering with delete buffering and purge
buffering
Better isolation of threads and mutex contention
• Improved Purge Scheduling
• Improved Log Sys mutex
• Separate Flush List mutex
Better Instrumentation/Diagnostics
• InnoDB stats in 5.5 PERFORMANCE_SCHEMA

“InnoDB Plugin: Performance Features and Benchmarks”,

InnoDB Team, Thurs, 10:50 am, Ballroom C

8
 MySQL 5.5/InnoDB 1.1 SysBench
Benchmarks

MySQL 5.5.4
(InnoDB 1.1)

MySQL 5.1.40
(InnoDB 1.0)

MySQL 5.1.40
(InnoDB built in)

At 1024 Connections: Intel Xeon X7460 x86_64

4 CPU x 6 Cores/CPU
• 188% performance gain for MySQL 5.5 over 5.1.40 (InnoDB 1.0)
2.66 GHz, 32GB RAM
• 200% performance gain for MySQL 5.5 over 5.1.40 (InnoDB built in) Fedora 10

9
 MySQL 5.5/InnoDB 1.1 SysBench
Benchmarks

MySQL 5.5.4
(InnoDB 1.1)

MySQL 5.1.40
(InnoDB 1.0)

MySQL 5.1.40
(InnoDB built in)

At 1024 Connections: Intel Xeon X7460 x86_64

4 CPU x 6 Cores/CPU
• 164% performance gain for MySQL 5.5 over 5.1.40 (InnoDB 1.0)
2.66 GHz, 32GB RAM
• 364% performance gain for MySQL 5.5 over 5.1.40 (InnoDB built in) Fedora 10

10
 MySQL 5.5 / InnoDB 1.1 Scalability
dbStress

Read-only
TPS

Read-write

cores
MySQL 5.5.4 is reaching higher TPS levels than others
MySQL 5.5.4 is better prepared now to scale up to 32 cores
http://dimitrik.free.fr/blog/archives/2010/04/mysql-performance-554-dbstress.html

11
InnoDB Architecture

12
InnoDB Architecture: Component Model
Server

Handler API

Access Methods Transaction Memory / Buffer

Manager Pool Manager

Concurrency Logging and Monitoring and

Control / Locking Recovery Diagnostics

Storage and IO Manager

13
InnoDB Architecture: Runtime Model
Memory Threads:
-master
Log
Misc -read io
buffer buffer pool
buffer Files -write io
-ibuf io
-log io
-lock
timeout
-monitor
Background threads

Buffer Pool
-data
-index
-undo
InnoDB Code -adaptive
hash index

14
Just Like Traffic System

Synchronization
Threads

Resources Scheduling

15
How to Solve Traffic Problem? Threads

 Have more lanes

• multiple background I/O threads
 Build another road
• Split purge thread
 Car pool / batch
• group commit
• delete buffering and purge buffering

16
 Synchronization
How to Solve Traffic Problem?

 Better / faster signals

• faster locking for improved scalability
• using a portable PAUSE to InnoDB spin loop
• control over Spin lock polling
 Change signal location
• Improved log sys mutex
• Separate flush list mutex

17
How to Solve Traffic Problem?

Resources
 More resource providers
• multiple buffer pool instances
• multiple rollback segments
 Better resource management
• using operating system memory allocators
• native asynchronous I/O on Linux

18
How to Solve Traffic Problem?
Scheduling

 Better scheduling / flow control

• Changes in read ahead algorithm
• Making buffer cache scan resistant
• Controlling master thread I/O rate
• Controlling flushing rate of dirty pages
 Faster accident handling
• Improvements to crash recovery performance
 Monitoring
• Performance schema for InnoDB

19
InnoDB Architecture: Data Storage

Tablespace
Segment
Extent Extent
Leaf node segment
an extent =
Non-leaf node segment Extent Extent
64 pages

Extent
Rollback segment
Page

Row Row Row

Trx id Row Row Row

Roll pointer
Field pointers Row Row

Field 1 Field 2 Field n

20
InnoDB Architecture: Data Storage

 Data compression: Compress data & index

pages from normal 16 KB size to specified
compressed page size
 New row format: New DYNAMIC mode stores
long columns entirely “off-page”.
 Multiple rollback segments: from 1 to 128
 TRUNCATE TABLE re-creates the *.ibd file to
reclaim space

21
 What about InnoDB Hot Backup?

 Now called “MySQL Enterprise Backup”

 Backs up running databases without
locking InnoDB data
 Supports InnoDB and MyISAM
 Features:
• Compressed backup
• Partial backup
• Point-in-time recovery

“Backup Strategies with MySQL Enterprise Backup”, InnoDB

Team, Wed, 3:05 pm, Ballroom D

22
New Features in Backup 3.5
Beta
 Incremental backup
 Support of Barracuda file format
• Backup of compressed tables
 Backup of additional files, such as partition
files
• Backup of in-memory database with
--exec-when-locked option

Email mike.frank@oracle.com to register for beta.

23
Recap: Plugin 1.0 Top Ten Performance
and Scalability Enhancements
1. Faster locking for improved scalability
2. Using operating system memory allocators
3. Multiple background I/O threads
4. Group commit is back
5. Controlling master thread I/O rate
6. Controlling flushing rate of dirty pages
7. Changes in read ahead algorithm
8. Using a portable PAUSE to InnoDB spin loop
9. Making buffer cache scan resistant
10. Improvements to crash recovery performance

24
Recap: Plugin 1.1 Performance and
Scalability Enhancements
 Multiple buffer pool instances
 Multiple rollback segments
 Improved purge scheduling
 Improved Log Sys mutex
 Separate flush list mutex
 Extended change buffering with delete buffering and
purge buffering: speed up random deletes
 Native async I/O support on Linux
 Performance schema for InnoDB

25
 Deep Dive
TALKS
InnoDB Plugin: Performance Features and Benchmarks,
InnoDB Team, Thurs, 10:50 am, Ballroom C
Backup Strategies with MySQL Enterprise Backup, InnoDB
Team, Wed, 3:05 pm, Ballroom D

BLOGS
•http://blogs.innodb.com/innodb-performance-recovery, Inaam Rana
•http://blogs.innodb.com/innodb-multiple-rollback-segments, Sunny Bains
•http://blogs.innodb.com/innodb-performance-schema, Jimmy Yang
•http://mikaelronstrom.blogspot.com/, Mikael Ronstrom
•http://dimitrik.free.fr/blog/archives/2010/04/mysql-performance-554-dbstress.html, Dimitri
Kravtchuk

26
Q&
QUESTIONS

A
ANSWERS

27
 Thanks for attending!

InnoDB: Status, Architecture and New Features

MySQL Conference and Expo, 2010
3:05 pm, April 13, 2010
calvin.sun@oracle.com rob.young@oracle.com

28