Вы находитесь на странице: 1из 334

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

bl
a
r
fe

an
r
t
n

no
a
s
a
h
MySQL Developer
n) ideTechniques
v

om t Gu
c

21 den
s
t
tuGuide
sp Student
S
@
s
i
n
h
a
t
r
t
SQL-4305
Release 1.1
e

s
n
u
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

D62172GC10
Edition 1.0
January 2011
D63882

Copyright 2008, 2009, 2011, Oracle and/or its affiliates. All rights reserved.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Disclaimer

Thi

This document contains proprietary information, is provided under a license agreement containing restrictions on use and
disclosure, and is protected by copyright and other intellectual property laws. You may copy and print this document solely for
your own use in an Oracle training course. The document may not be modified or altered in any way. Except as expressly
permitted in your license agreement or allowed by law, you may not use, share, download, upload, copy, print, display,
perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express
authorization of Oracle.
The information contained in this document is subject to change without notice. If you find any problems in the document,
please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This
document is not warranted to be error-free.
Sun Microsystems, Inc. Disclaimer
This training manual may include references to materials, offerings, or products that were previously offered by Sun
Microsystems, Inc. Certain materials, offerings, services, or products may no longer be offered or provided.Oracle and its
affiliates cannot be held responsible for any such references should they appear in the text provided.

bl
a
r
fe

Restricted Rights Notice


If this documentation is delivered to the U.S. Government or anyone using the documentation on behalf of the U.S.
Government, the following notice is applicable:

an
r
t
n

U.S. GOVERNMENT RIGHTS


The U.S. Governments rights to use, modify, reproduce, release, perform, display, or disclose these training materials are
restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective
owners.
AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro
Devices. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used
under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark
licensed through X/Open Company, Ltd.

uy
g
N

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

This page intentionally left blank.

bl
a
r
fe

an
r
t
n

uy
g
N

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Conventions Used in This Training Guide

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

This training guide uses certain typographical conventions:


TEXT IN THIS STYLE is used for SQL statements; database, table, and column names; program
listings and source code; and environment variables. Example: To reload the grant tables, use the
FLUSH PRIVILEGES statement.

Thi

Text in this style indicates input that you type in examples.


Text in this style indicates the names of executable programs and scripts, examples being mysql (the
MySQL command line client program) and mysqld (the MySQL server executable).
Text in this style is used for variable input for which you should substitute a value of your own
choosing.
Filenames and directory names are written like this: The global my.cnf file is located in the /etc
directory.
Character sequences are written like this: To specify a wildcard, use the '%' character.
Text in this style is used for emphasis.
When commands are shown that are meant to be executed from within a particular program, the prompt shown
preceding the command indicates which command to use. For example, shell> indicates a command that you
execute from your login shell, and mysql> indicates a statement that you execute from the mysql client
program:
shell> type a shell command here
mysql> type a mysql statement here
The shell is your command interpreter. On Linux, this is typically a program such as sh, csh, or bash. On
Windows, the equivalent program is command.com or cmd.exe, typically run in a console window.
When you enter a command or statement shown in an example, do not type the prompt shown in the example.
Database, table, and column names must often be substituted into statements. To indicate that such substitution is
necessary, this manual uses db_name, tbl_name, and col_name. For example, you might see a statement
like this:
mysql> SELECT col_name FROM db_name.tbl_name;
This means that if you were to enter a similar statement, you would supply your own database, table, and column
names, perhaps like this:
mysql> SELECT author_name FROM biblio_db.author_list;
SQL keywords are not case sensitive and may be written in any lettercase. This training guide uses uppercase.
In syntax descriptions, square brackets ('[' and ']') indicate optional words or clauses. For example, in the
following statement, IF EXISTS is optional:
DROP TABLE [IF EXISTS] tbl_name
When a syntax element consists of a number of alternatives, the alternatives are separated by vertical bars ('|'). When
one member from a set of choices may be chosen, the alternatives are listed within square brackets ('[' and ']'):
TRIM([[BOTH | LEADING | TRAILING] [remstr] FROM] str)
When one member from a set of choices must be chosen, the alternatives are listed within braces ('{' and '}'):
{DESCRIBE | DESC} tbl_name [col_name | wild]
An ellipsis (...) indicates the omission of a section of a statement, typically to provide a shorter version of
more complex syntax. For example, INSERT ... SELECT is shorthand for the form of INSERT statement
that is followed by a SELECT statement.
An ellipsis can also indicate that the preceding syntax element of a statement may be repeated. In the following
example, multiple reset_option values may be given, with each of those after the first preceded by commas:
RESET reset_option [,reset_option] ...
Commands for setting shell variables are shown using Bourne shell syntax. For example, the sequence to set the
CC environment variable and run the configure command looks like this in Bourne shell syntax:
shell> CC=gcc ./configure
If you are using csh or tcsh, you must issue commands somewhat differently:
shell> setenv CC gcc and shell> ./configure

an
r
t
n

uy
g
N

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

Supporting Images Used in This Training Guide


The following is a summary of the standard images used in this manual to support the instruction:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

IMAGE

Thi

NAME

DESCRIPTION

Preparation

This image is used to describe the


steps required to be completed prior
to performing a hands-on exercise.

Written Exam

This image is used to identify that


the student is going to be tested
upon the material previously
presented in the instructional
material.

uy
g
N

bl
a
r
Throughout the course the instructorfe
InLine Lab
ns
will conduct labs in line with the
a
r
t
instruction, which are designed
- to
n
o
help you to understand
the
n ofnuts
a
and bolts (inner-workings)
the
s
topic. ha
n) ide
v

image
to convey to
Further Practice
u isthatused
om This
G
c
thetstudent
there is a final
Lab

n to complete prior to the


21 deexercise
s
t
sp s Stu completion of the chapter.
@
i
n
h
a
t
r
t
This image identifies an area on a
Student
Notes
e

s
n
u
o
page designated for students to
o
o
t
m
write notes associated with the
( nse
n
class.
a
r
e
c
T
i
l
et
123

Slide Number
Boxes

This image is used throughout the


course guide to indicate the
existence and number of a
corresponding instruction slide.

COURSE OBJECTIVES
This course is designed for experienced database developers who wish to learn additional skills
required to create complex queries and efficient structures while improving the performance of the
database by learning to create and write queries that are optimized based on the data in the database.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

At the completion of this course, you should be able to:

Thi

Utilize the multiple indexing options available to create better response times of query
executions

Optimize queries that are responsible for searching text fields

Optimize queries that search date fields

Optimize queries that perform calculations and aggregations

Create summary tables that utilize triggers, event scheduler and table merges

Optimize queries that retrieves data from more than one table

Create tables that utilize tree and hierarchical structures

o
n
a
Utilize temporary tables to improve query performance
s
a
h
Explain the affect locking levels have on query performance
n) ide
v

u
ompreciset G
Create report queries that provide the end userscwith
and organized data

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

an
r
t
n

uy
g
N

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Table of Contents
1
1.1
1.2
1.2.1
1.3
1.3.1
1.3.2
1.3.3
1.3.4
1.3.5
1.4
1.4.1
1.4.2
1.4.3
1.4.4
1.5
1.5.1
1.5.2
1.5.3
1.5.4
1.6
1.7
1.8
1.9
1.9.1
1.9.2
1.10
1.11
1.12

Introduction ................................................................................................................................................. 1-1


Learning Objectives .................................................................................................................................... 1-1
The MySQL Overview................................................................................................................................ 1-2
MySQL Partners.......................................................................................................................................... 1-3
MySQL Products......................................................................................................................................... 1-4
MySQL Database Products ......................................................................................................................... 1-4
MySQL GUI Tools...................................................................................................................................... 1-5
Other MySQL Tools.................................................................................................................................... 1-5
MySQL Drivers........................................................................................................................................... 1-6
Solutions for Embedding MySQL............................................................................................................... 1-6
MySQL Services ......................................................................................................................................... 1-7
MySQL Training ......................................................................................................................................... 1-7
MySQL Certification................................................................................................................................... 1-7
MySQL Consulting ..................................................................................................................................... 1-7
MySQL Support .......................................................................................................................................... 1-7
The MySQL Enterprise Subscription .......................................................................................................... 1-8
MySQL Enterprise Server ........................................................................................................................... 1-8
24x7 Enterprise Support.............................................................................................................................. 1-8
MySQL Enterprise Monitor ........................................................................................................................ 1-9
Obtaining a MySQL Enterpise Subscription ..............................................................................................1-12
Supported Operating Systems ....................................................................................................................1-13
MySQL Certification Program ...................................................................................................................1-14
Training Curriculum Paths .........................................................................................................................1-15
MySQL Website.........................................................................................................................................1-19
MySQL Community Web Page..................................................................................................................1-20
MySQL Online Documentation .................................................................................................................1-22
Installing MySQL.......................................................................................................................................1-25
Installing the 'world' database.....................................................................................................................1-26
Chapter Summary.......................................................................................................................................1-27

2.4
2.4.1
2.5
2.6
2.7

When MySQL Uses an Index...................................................................................................................... 2-8


When MySQL Does Not Use an Index ....................................................................................................... 2-9
Retrieving Index Information .....................................................................................................................2-11
Optimize Indexes........................................................................................................................................2-16
Summary ....................................................................................................................................................2-19

3
3.1
3.2
3.3
3.3.1
3.4
3.5
3.6
3.7
3.8
3.8.1
3.9
3.10

Searching Date and Text Fields................................................................................................................... 3-1


Overview ..................................................................................................................................................... 3-1
Strings in Multiple Columns ....................................................................................................................... 3-2
Case Sensitivity ........................................................................................................................................... 3-4
Binary searches ........................................................................................................................................... 3-7
INET_ATON and INET_NTOA................................................................................................................3-11
Searching Dates..........................................................................................................................................3-13
String Dates to SQL Dates .........................................................................................................................3-16
Dates to Integer Values ..............................................................................................................................3-18
Dates to Strings ..........................................................................................................................................3-23
Time values to Strings................................................................................................................................3-24
Nonspecific Date Searches.........................................................................................................................3-28
Summary ....................................................................................................................................................3-30

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nwith
se Indexes ......................................................................................................... 2-1
2
Improving
Performance
n
a
r
e
2.1
Overview
2-1
lic
t T .....................................................................................................................................................
2.2 yeQuery Executions in MySQL ...................................................................................................................... 2-2
2.2.1
gu Query Optimizer.......................................................................................................................................... 2-3
N
hi 2.3 Why Indexes?.............................................................................................................................................. 2-4

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4
4.1
4.2
4.2.1
4.2.2
4.2.3
4.2.4
4.3
4.3.1
4.3.2
4.4
4.5

Improving Inserts ........................................................................................................................................ 4-1


Overview ..................................................................................................................................................... 4-1
INSERT Process.......................................................................................................................................... 4-2
Multiple Row Inserts ................................................................................................................................... 4-3
INSERT ... SELECT ................................................................................................................................... 4-4
LOAD DATA INFILE ................................................................................................................................ 4-4
Transactional Multiple Inserts..................................................................................................................... 4-8
Storage Engine Specifics............................................................................................................................. 4-9
Speeding up Bulk Inserts with MyISAM .................................................................................................... 4-9
Speeding up Bulk Inserts with InnoDB......................................................................................................4-10
MySQL Extensions ....................................................................................................................................4-11
Summary ....................................................................................................................................................4-18

5
5.1
5.2
5.3
5.4
5.5
5.5.1
5.5.2
5.6
5.7
5.8
5.9

Complex Calculations ................................................................................................................................. 5-1


Overview ..................................................................................................................................................... 5-1
Aggregate Multiplication ............................................................................................................................ 5-2
Running Total Query................................................................................................................................... 5-5
Avoiding Divide by Zero ............................................................................................................................ 5-7
Calculate the Median Value ........................................................................................................................ 5-9
Median Value via Implicit Temporary Tables ...........................................................................................5-10
Creating a Median Stored Procedure..........................................................................................................5-13
Simulating RANK ......................................................................................................................................5-16
Divide and Conquer ...................................................................................................................................5-18
Miscellaneous.............................................................................................................................................5-26
Summary ....................................................................................................................................................5-27

6.5.3
6.5.4
6.5.5
6.6

Double Checking EXPLAIN......................................................................................................................6-26


JOIN - Best Practices .................................................................................................................................6-33
Simulating INTERSECT and MINUS .......................................................................................................6-34
Summary ....................................................................................................................................................6-35

7.
7.1.
7.2.
7.3.
7.3.1.
7.3.2.
7.4.
7.4.1.
7.4.2.
7.4.3.
7.4.4.
7.4.5.

Hierarchical Data......................................................................................................................................... 7-1


Overview ..................................................................................................................................................... 7-1
Graphs, Trees and Hierarchies .................................................................................................................... 7-2
Adjacency List Structures ........................................................................................................................... 7-5
Search Tree Depths ..................................................................................................................................... 7-9
Modify Tree Values ...................................................................................................................................7-15
Nested Set Structures .................................................................................................................................7-16
Search Tree Depths ....................................................................................................................................7-18
Display Depth.............................................................................................................................................7-19
Displaying a specific branch ......................................................................................................................7-20
Inserting Nodes ..........................................................................................................................................7-22
Deleting Nodes...........................................................................................................................................7-25

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c
6
Improving Performance of Joins .................................................................................................................
6-1

1
n
2
e
6.1
Overview .....................................................................................................................................................
6-1
s
pt Stud
6.2
Query Optimizer..........................................................................................................................................
6-2
s
@ this
6.2.1
Greedy Optimizer........................................................................................................................................
6-4
n
a
r
6.3
EXPLAIN....................................................................................................................................................
6-6
t
e

s
n
6.4
JOIN's..........................................................................................................................................................
6-8
o to u
o
6.4.1
Nested JOIN's.............................................................................................................................................6-14
m
6.4.2
Self Joins ....................................................................................................................................................6-15
n ( ense
a
r
6.4.3
FULL
ic
lJOIN..................................................................................................................................6-16
t T OUTER
6.5 yeImproving
JOIN performance ....................................................................................................................6-21
u Reducing Rows Touched by Joins .............................................................................................................6-22
6.5.1
g
N
hi 6.5.2 Covering the Index .....................................................................................................................................6-24

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

7.5.
7.5.1.
7.5.2.
7.6.

Path Enumeration .......................................................................................................................................7-28


Search the depths of the path......................................................................................................................7-29
Retrieving hierarchical data from the paths................................................................................................7-31
Summary ....................................................................................................................................................7-32

8.
8.1.
8.2.
8.3.
8.3.1.
8.4.
8.4.1.
8.5.
8.6.

Advanced Index Structures ......................................................................................................................... 8-1


Overview ..................................................................................................................................................... 8-1
MySQL Index Types ................................................................................................................................... 8-2
FULLTEXT Index....................................................................................................................................... 8-5
FULLTEXT Search Functions .................................................................................................................... 8-6
Simulating Function Based Indexes ............................................................................................................ 8-8
Searching for valid words............................................................................................................................ 8-8
Optimizing End of Field Searches..............................................................................................................8-10
Summary ....................................................................................................................................................8-14

9
9.1
9.2
9.3
9.3.1
9.3.2
9.4
9.4.1
9.4.2
9.4.3
9.5
9.6

Locking Levels............................................................................................................................................ 9-1


Overview ..................................................................................................................................................... 9-1
Locks ........................................................................................................................................................... 9-2
Explicit Table Locks ................................................................................................................................... 9-4
LOCK TABLES statement.......................................................................................................................... 9-4
UNLOCK TABLE statements..................................................................................................................... 9-5
Storage Engine Locking Techniques........................................................................................................... 9-6
MyISAM Locking ....................................................................................................................................... 9-6
InnoDB Locking.......................................................................................................................................... 9-9
Locking with Other Storage Engines .........................................................................................................9-10
Locking Issues............................................................................................................................................9-11
Summary ....................................................................................................................................................9-17

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
p Stu
10.
Creating Reports.........................................................................................................................................10-1
s
@
10.1.
Overview ....................................................................................................................................................10-1
is
n
h
a
t
r
10.2.
Calculate Multiple Conditions....................................................................................................................10-2
t use
n
10.3.
Create a Calculated Report.........................................................................................................................10-4
o
o
to
10.4.
Quarterly Reports
.......................................................................................................................................10-9
m
(
e
10.4.1. Cross Tab n
Reports ....................................................................................................................................10-10
ns
aChart..........................................................................................................................................10-11
r
e
10.5.
SQLTBar
c
t Tablesli........................................................................................................................................10-13
10.6. yeDecision
u Materialized Views ..................................................................................................................................10-16
10.7.
g
N
hi 10.8. Producing Sequential or Missing Data .....................................................................................................10-17

10.8.1. Using the integers table to fill in missing data .........................................................................................10-21


10.9.
Summary ..................................................................................................................................................10-23
11
11.1
11.2
11.3
11.4
11.5

Conclusion..................................................................................................................................................11-1
Course Objectives ......................................................................................................................................11-1
Training and Certification Website ...........................................................................................................11-2
Course Evaluation ......................................................................................................................................11-4
THANK YOU!...........................................................................................................................................11-5
Q&A Session..............................................................................................................................................11-6

Appendix A
Appendix B
Appendix C
Appendix D
Appendix E

bl
a
r
fe

Installation Procedures..................................................................................................................A-1
MyISAM Locking.........................................................................................................................B-1
Quiz Solutions...............................................................................................................................C-1
Further Practice Solutions .............................................................................................................D-1
Better Performance, Scalability, and Recovery.............................................................................E-1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

CHAPTER 1
INTRODUCTION

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

1
1.1

Chapter 1: Introduction

INTRODUCTION
Learning Objectives

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

This chapter introduces you to MySQL products and services. In this chapter, you will learn:
8

Thi

Explain the origin and status of the MySQL product

List the available MySQL products and professional services

Describe the MySQL Enterprise subscription

List the currently supported operating systems

Describe the MySQL Community web page

Describe the MySQL Certification program

List all the available MySQL courses

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-1

MySQL Developer Techniques

1.2

The MySQL Overview


MySQL is a Relational Database Management System
(RDBMS) that was originally developed by MySQL
AB. It was (and continues to be) developed and marketed
as a family of high performance, affordable database
servers and tools. Contributing to building the missioncritical, high-volume systems and products worldwide is
what makes MySQL the worlds most popular open
source database, as well as its reliability, excellent
performance and ease of use.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 1: Introduction

MySQL is not only the worlds most popular open source


database. It is also the fastest growing database in the
industry, with over 70,000 downloads per day, ranging
from large corporations to specialized embedded
applications.

MySQL is installed on every ble


continent in the world fera
s
(Yes, even Antarctica!)
tran

MySQL AB was founded in Sweden by two Swedes and


a Finn: David Axmark, Allan Larsson and Michael
"Monty" Widenius who have worked together since the
80's. MySQL AB (Swedish for Inc.) was the sole owner of the MySQL server source code, the MySQL
trademark and the mysql.com domain worldwide.

n
o
n

10

a
s
a
h
)
n
v uide
m
o
Sun Acquisition:
c nt G
1
2
In early 2008, MySQL AB was acquired by Sun
de Inc., giving the MySQL product line the
tsMicrosystems,
u
p
t
s
considerable resources of a major mainstream company.S
Together, Sun and MySQL will now provide global
@
s
i
enterprise-class support 24x7x365. Our
enterprise-class
customers worldwide can now take advantage of our
n
h
t
tra on their
market-leading open source database
choice of hardware, operating system and language with up to a
e

s
n
o to u 90% lower total cost of ownership than many traditional
o
m
database solutions.
( nse
n
a
r
ce
T
i
MySQL has always been, and will continue to be, a strong
l
t
e
supporter of the open source philosophy and the open source
y
gu
community, and strives to work with partners who share the

Thi

same values and mindset. This will be no different within Sun.


Sun is a world-leader in open source, with products and technologies such as Java, OpenSolaris, Open Office,
NetBeans, OpenSparc and many more.
The MySQL mission is still the same...
Make superior database software available and affordable to all!

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-2

MySQL Developer Techniques

1.2.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

11

Chapter 1: Introduction

MySQL Partners
MySQL has had the privilege of forming alliances with excellent partners and attracting some of the most
impressive customers in the industry! When you join our ranks, you are joining a winning team with a wide
variety of MySQL implementations. It never ceases to amaze us, the innovative and powerful ways in which
our tools are being used. To name only a few;

Web / Web 2.0

OEM / ISV's

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

On Demand, SaaS, Hosting


Telecommunications
Enterprise 2.0
21 den
s
t
u
p
t
s sS
@
i
n
h
a
t
r
t
e
Open-source
on to usis powering the World!
o
(m nse
n
a
r
ce
T
i
l
t
"We have
We went from experimental to mission-critical in a couple of
uyeused MySQL far more than anyone expected.months."
g
Jeremy Zawodny--MySQL Database Expert Yahoo! Finance
hi N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-3

MySQL Developer Techniques

1.3

MySQL Products

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

1.3.1

12

Thi

Chapter 1: Introduction

MySQL Database Products

Sun provides MySQL database products to meet the needs of ISV/OEM, Enterprise, and Community Server
users. MySQL database products are recognized for superior ease of use, performance, and reliability.
MySQL Enterprise Server
o

The most reliable, secure and up-to-date version of the world's most popular open source database
for cost-effectively delivering E-commerce, Online Transaction Processing (OLTP), and multiterabyte Data Warehousing applications. (Available only with the MySQL Enterprise subscription).

MySQL Community Server


o

The MySQL database server for open source developers and technology
enthusiasts who want to get started with MySQL. Supported by the large
MySQL open source community. Under the General Public License
(GPL), benefits to the open source community include a commercialgrade framework that is free of charge.

bl
a
r
fe

ns
a
r
t
- a reliable
o The most popular choice for OEMs/ISVs who want to cost-effectively embed orn
bundle
o
and high-performance relational database.
an
s
MySQL Cluster
a
h
)
o A fault tolerant database clustering architecture for deploying
highly
n ideavailable mission-critical
v

database applications.
om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
lice
t Tr

MySQL Embedded Database

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-4

MySQL Developer Techniques

1.3.2
13

Chapter 1: Introduction

MySQL GUI Tools


The MySQL GUI Tools form a comprehensive graphical user interface to your MySQL database. These easy to
use graphical tools enable database Developers and Database Administrators
(DBAs) to be more productive.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Migration Toolkit


o

Using a wizard-driven interface, the MySQL Migration Toolkit


implements a proven methodology and walks you through the
necessary steps to successfully complete a database migration
project.

MySQL Administrator
o A powerful graphical administration console that enables you to easily administer your MySQL
environment and gain significantly better visibility into how your databases are operating.
MySQL Query Browser

Bundled Tools

an
r
t
n

bl
a
r
fe

o An extremely user-friendly graphical tool for creating, executing, and optimizing SQL queries
for your MySQL Database Server.

no
a
s
a
h
n) ide
v
1.3.3 Other MySQL Tools

m Gu
o
c

t server:
14
MySQL has also developed additional tools to assist in
the
MySQL
1 use of the
n
2
e
s
pt Stud
MySQL Workbench
s
@ database
is design tool developed by MySQL. It is
o MySQL Workbench a
is n
a visual
h
t
r
the highly anticipated
of the DBDesigner4 project. MySQL
t successor
seapplication
n
u
o
Workbench
is
currently
available
for
Microsoft
Windows in a GPL licensed open
o
o
t
m
source
edition
as
well
as
a
low-priced
standard
edition which includes additional
( nse
n
time
saving
features
for
professionals.
a
r
ce
T
i
l
t

MySQL
Proxy
uye o MySQL Proxy is a simple program that sits between your
g
N
client and MySQL server(s) that can monitor, analyze or
Thi
transform their communication. Its flexibility allows for

The MySQL graphical user interface (GUI) tools discussed here come bundled together when downloaded from the MySQL
website.

unlimited uses; common ones include: load balancing;


failover; query analysis; query filtering and modification;
and many more.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-5

MySQL Developer Techniques

1.3.4
15

Chapter 1: Introduction

MySQL Drivers
MySQL Connectors are database drivers, providing database client connectivity for a wide range of
programming languages. MySQL provides the following connectors:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL C API
o Our native client library (libmysql), which can be wrapped by other languages.
MySQL Connector/ODBC
o Connect to a MySQL database server using the Open Database Connectivity (ODBC) API on all
Microsoft Windows and most Unix-like platforms. The ODBC driver builds on the client/server
protocol implementation provided by libmysql.
MySQL Connector/J
o A JDBC (Java Database Connectivity) 4.0 driver for Java 1.4 and higher. Provides a java native
implementation of the MySQL client/server protocol.
MySQL Connector/Net

an
r
t
n

o A fully managed ADO.NET provider for the .NET framework (version 1.1 and 2.0). Provides a
.NET implementation of the MySQL client/server protocol.

no
a
sare two MySQL specific PHP
a
o Provides MySQL connectivity for PHP programs. Currently there
h
nand) mysqli
extensions available that use libmysql: the mysql
eextensions. There is also
v

d
i
MySQL support for the generic PHP Databaseo
objects
m (PDO)
uextension. In addition there is the
G
creplace

PHP native driver called mysqlnd which1can


libmysql
in the mysqli extension.
t
n
2
e
s
1.3.5 Solutions for Embedding MySQL
pt Stud
s
@clienttside
isconnectivity, MySQL also provides libraries to embed a
Apart from these Connectors that provide
n
h
a
r
MySQL database server within atprogram. Currently
se the following solutions can be used to embed MySQL:
n
u
o
libmysqldmo
to
16
(
e
n embedded
nsedition of the mysqld server program wrapped in a shared library. Allows the
oaThe
r
e
c
T
MySQL
li Server to be embedded in C programs.
t
e
y
gu MySQL MX/J
N
o A jar wrapper around mysqld binaries. This allows java programs and J2EE environments to
Thi
MySQL Connector/PHP

bl
a
r
fe

instantiate (and install) a MySQL server.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-6

MySQL Developer Techniques

1.4

MySQL Services

1.4.1

MySQL Training
Sun Microsystems offers a comprehensive set of MySQL training courses that give you a competitive edge in
building world-class database solutions.

17

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 1: Introduction

Courses can be chosen individually, as part of a bundle, and/or following our suggested curriculum
path for Developers and Database Administrators (DBAs).

1.4.2

MySQL Certification
MySQL Certification Program is a high quality certification program that provides Developers and DBAs with
the credentials to prove they have the knowledge, experience and skills to use and manage MySQL Server.
With MySQL personal certifications, you get to show that you are among the best-of-breed MySQL users and
lay the foundation for becoming a trusted and valuable resource for your company and customers.

1.4.3

bl
We offer a full range of consulting services. Whether you are starting a new project, needing to optimize e
anra
sf
existing MySQL application, or migrating from a proprietary database to MySQL, we have an affordable
n
a
solution for you.
-tr
n
o
Using industry best practices and proven methodologies, your MySQL certified consultant
nwill help you deliver
a
on-time and on-budget.
s
a
h
MySQL Support
n) ide
v

We also offer a full range of support options. MySQL Technical


Support isu
designed to save you time and to
om
G
c

ensure you achieve the highest levels of performance, reliability,


and
uptime.
t
21 den
s
Community support
t
sp s Stu
o Mailing Lists
@
i
n
h
a
t
r
o Forums (http://forums.mysql.com/)
t
e
n (http://dev.mysql.com/tech-resources/articles)
us
oArticles
o Community
o
o
t
(m
e
o Bugs
Databases
(http://bugs.mysql.com/)
n
n
a
r
e
T o No direct
lic access to support engineers
t
e
o PlanetMySQL Blogs (http://www.planetmysql.org/)
uy

1.4.4

N
Thi

MySQL Consulting

o MySQL Reference Manuals (http://dev.mysql.com/doc)


o MySQLForge (http://forge.mysql.com/)

Purchased support
o Enterprise subscription
o Support for MySQL Cluster
o Support for MySQL Embedded (OEM/ISV)
o Online Knowledge Base

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-7

MySQL Developer Techniques

1.5

The MySQL Enterprise Subscription


A MySQL Enterprise subscription gives you access to the Enterprise Server as well as
other premium products and services. It is a comprehensive set of enterprise-grade
software, support and services to ensure the highest levels of reliability, security and
uptime. As a proactive service that helps you eliminate problems before they occur, it
gives you everything you need in a single, unified offering to successfully develop and
deploy business critical applications using MySQL.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

18

1.5.1
19

Chapter 1: Introduction

MySQL Enterprise Server


The most reliable, secure and up-to-date version of the world's most popular open source database for costeffectively delivering E-commerce, Online Transaction Processing (OLTP), and multi-terabyte Data
Warehousing applications. With an Enterprise subscription you also get the following in addition to the
Enterprise Server:
Updates and Service Packs
o

Emergency Hot Fix Builds

an
r
t
n

no
a
s
a
Drivers
h
details.
n) website
efor
o Many different drivers are supported. See MySQL Enterprise
v

d
i
om t Gu
MySQL Workbench
c

1
n design and maintenance, automates
2simplifies
e
o A graphical user interface (GUI)tthat
database
s
d
p tasks,Sand
tu improves communication among DBA and
time-consuming and error-prone
s
@
s
developer teams.
i
n
h
a
t
r
t
e

Platforms
uares supported. See MySQL Enterprise website for details.
onplatforms
o
o Many different
o
t
m se
(Support
n
1.5.2 24x7 Enterprise
n
a
r
e
c
T
i
l
et Service Level
y
20
u
o MySQL Enterprise gives you the flexibility to choose a service level that matches your
g
N
i
requirements.
Th
o

Can request a special hot fix build that resolves a critical issue that is not addressed by a service
pack.

Basic, Silver, Gold, or Platinum (with incrementally increasing quantity and quality of
support)

Problem Resolution Support


o

Ensures you receive high priority service from the MySQL Support Team for quick resolution
of technical problems as they occur. The services available to you are dependent on your chosen
service level.

Consultative Support
o

bl
a
r
fe

Automatically receive both monthly rapid updates and quarterly service packs with the latest
bug-fixes and security updates.

Expert MySQL Consultants provide you with proactive advice on how to properly design and
tune your MySQL servers, schema, queries and replication to ensure the highest possible
reliability.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-8

MySQL Developer Techniques

Chapter 1: Introduction

Technical Account Manager (TAM)

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

You have the option of adding a Technical Account Manager (Platinum level only) to be your
liaison within MySQL. Your TAM be your single point of contact, providing a custom review
of your systems, regular phone calls and on-site visits, to guarantee that you get the most out of
MySQL Support Services.

Online Knowledge Base


o

For quick self-help knowledge, you will have access to a comprehensive and easily searchable
knowledge base library with hundreds of technical articles regarding difficult problems on
popular database topics such as performance, replication, configuration, and security.
Enterprise support

For more detailed information regarding Enterprise support, please refer to our Enterprise web page located at:
http://www.mysql.com/products/enterprise/support.html .

1.5.3
21

Thi

MySQL Enterprise Monitor

an
r
t
n

The MySQL Enterprise Monitor is a web-based monitoring and advising system. The Enterprise Monitor helps
MySQL DBAs manage more MySQL servers in a scale-out environment, tune their current MySQL servers and
find and fix problems with their MySQL database applications before they can become serious problems or
costly outages. Running completely within the corporate firewall, the Enterprise Monitor pro-actively monitors
enterprise database environments and provides expert advice on how MySQL can tighten security, optimize
performance and reduce downtime of their MySQL powered systems. EM accomplishes all this while reducing
DBA time and effort.
EM provides a rich GUI Enterprise Dashboard that contains pages which allow the DBA to have an
immediate, graphic view of administrative tasks, server and database status, and advisory information.
The principle features of Enterprise Monitor:

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
Enterprise Dashboard
on o usfrom a consolidated console
o Manage o
all MySQLtservers
(m nse management
Server
or Server-group
n
a
ro Autolidetection,
ce grouping and monitoring of replication and scale-out topologies
T
t
e

uy
g
N

bl
a
r
fe

Monitoring page
o "At a glance" global health check of key systems

MySQL provided Advisors and Advisor Rules


o Enforce MySQL Best Practices
Advisor Rule Scheduler
o Schedule unattended operations
Customizable Thresholds and Alerts
o Identify Advisor Rule violations
Custom Advisor Rule
o User defined advisor rules
Events and Alert history
o Lists all Advisor Rules executions
Specialized Scale-Out assistance
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-9

MySQL Developer Techniques

Chapter 1: Introduction

Enterprise Dashboard

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

22

Graphs
(custom)

Page Tabs

ED Control
/Help

bl
a
r
fe

an
r
t
n

an
s
a
h
)
Explorer
n ide
v
Navigation

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
Critical Events
l
t
e
y
gu
N
hi

Heat Chart

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-10

MySQL Developer Techniques

MySQL Enterprise Advisors


The MySQL advisors are divided into the following categories (and specific functions):

23

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 1: Introduction

bl
a
r
fe

an
r
t
n

24

no
a
s
a
Enterprise Monitor Subscription Level
h
n)features
eare only available with the
A Silver or higher level is required to get Enterprise Monitor. Some
v
d
i
u below:
Gold and Platinum levels, respectively. A list of features per
level is shown
om
G
c

t
1 Gold
n
e
Silvers2
Platinum
d
t
u
p
t
s
S
Enterprise Dashboard
@
s
i
n
h
tra se t

Notifications andn
Alerts
oo to u
m
Custom (
Advisor
n ense
a
r
TUpgrade Advisor
lic
t
e
guy Administration Advisor

Thi

Security Advisor
Replication Monitor
Replication Advisor
Query Analysis Advisor
Memory Usage Advisor
Schema Advisor
Performance Advisor
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-11

MySQL Developer Techniques

1.5.4

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

25

Thi

Chapter 1: Introduction

Obtaining a MySQL Enterpise Subscription


For volume discounts or to order by phone, the MySQL Enterprise subscription can be obtained by contacting
Sun Microsystems/MySQL sales personnel:

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

A permanent MySQL Enterprise subscription can be purchased through the MySQL website located at :
https://shop.mysql.com/enterprise/.
A 30-day trial (with limited features) is also available: http://www.mysql.com/trials/ .

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-12

MySQL Developer Techniques

1.6

Supported Operating Systems

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

26

Thi

Chapter 1: Introduction

MySQL runs on more than 20 platforms, giving users the kind of flexibility that
puts them in control. Windows, Linux and Solaris are the most popular operating
systems for running MySQL. Versions of MySQL are currently available for the
following operating systems:
FreeBSD
HP-UX
IBM AIX and i5
Linux (multiple)
Mac OS/X
Microsoft Windows (multiple)
Novell netware

Open BSD
QNX
SGI Irix

bl
a
r
fe

an
r
t
n

no
a
s
Source code
a
h
Special builds
n) ide
v

m Gu
oinformation,
c
This list is continually being updated. For the most current

t please check the MySQL website at:


1
n
2
e
http://www.mysql.com .
s
pt Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
t
e
y
gu
Solaris

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-13

MySQL Developer Techniques

1.7

MySQL Certification Program


The MySQL Certification Program is a high quality certification program that provides developers and DBAs
with the credentials to prove they have the knowledge, experience and skills to use and manage MySQL
Server. MySQL provides several certification types and levels:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

27

Thi

Chapter 1: Introduction

1. Certified MySQL Associate (CMA) - an entry level certification. It is intended for those that are
relatively new to using the MySQL database server and covers basic database management system
concepts as well as basic SQL. We recommend the CMA certification for MySQL users that know
the basics, but have not yet obtained the experience gained by professional MySQL DBAs or
Developers.
2. Certified MySQL 5.0 Developer (CMDEV) - proves mastery of the fundamental skills of using
MySQL including creating and using databases and tables, inserting, modifying, deleting, and
retrieving data.
3. Certified MySQL 5.0 Database Administrator (CMDBA) - proves mastery of the ability to
manage MySQL Server including such advanced areas of database management, installation,
security, disaster prevention and optimization.

an
r
t
n

bl
a
r
fe

4. Certified MySQL Cluster DBA (CMCDBA) - part of the DBA track which represents an
advancement level exceeding CMDBA certification. In order to attain CMCDBA certification, you
must attain CMDBA certification and pass one CMCDBA exam.

no
a
s
a
h
Certification web page
nthe) exams,
esee our Certification web
v

d
For more information on the certification program and the content
of
i
om t Gu
page; http://www.mysql.com/certification/ .
c

21 den
Exam administration
s
t
tuPearson VUE testing centers available worldsp s 3,000
S
All exams are administered through one@
of more than
i proctor network. Visit the MySQL certification
n
h
wide. And through the Linux r
Professional
Institute
a
t
t
e
webpage, online forum or email
for more information.
s
u
oncertification@mysql.com
o
o
(m nse t
n
a ice
r
T
l
t
e
y
gu

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-14

MySQL Developer Techniques

1.8

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

28

Thi

Chapter 1: Introduction

Training Curriculum Paths


MySQL offers the most comprehensive set of MySQL training courses that enable the building of database solutions
and competitiveness now and into the future. In addition to our open courses, we also offer in-house training.
The MySQL training services staff has put together great courses designed for success, and an excellent training
path for each individual to reach his/her training goals. There are two tiers to the curriculum paths:
Developer path

Introduction courses
o

an
r
t
n

bl
a
r
fe

MySQL for Beginners This course covers the fundamentals of SQL and relational databases,
using MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous
experience with any command-line program (such as MS-DOS or the MS Windows command
prompt) is beneficial. 4 Days in length.

no
a
s
a
h
Intermediate courses
n) ide
v

m is designed
u for students planning on developing
o MySQL for Developers - This instructor-led o
course
G
c

t
applications that make use of MySQL 5.0
(and
higher).
This
course
covers essential SQL statements
1 In addition,
n
2
e
for data design, querying, and programming.
it
will
prepare
students for the MySQL
s
d with Relational Databases
u
pt Some
t
Developer certification. Prerequisites:
experience
and SQL. 5 days
s
S
@
s
i
in length.
an e th
r
t

s New Features This instructor-led course will provide in-depth


nUpgradinguand
o MySQL 5.0
oneeded
o
o
knowledge
to
become
proficient using MySQL 5.0. This training course will provide quality
t
(m
time
both on s
theetopic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL
n
n
a usersiwho
r
e
want to become proficient using MySQL 5.0. 3 Days in length.
c
T
l
t

uy
g
N

MySQL and PHP : Developing Dynamic Web Application This course will provide the tools
needed for the development of dynamic web application. This course takes the student from the
basics to the advanced features in four hands-on, heavy duty training days. Each student will have
the opportunity to build a real-world application during the course. Prerequisites: Basic experience
with designing HTML pages including HTML forms and experience with any programming
language. Basic PHP skills. 4 days in length.

Advanced courses
o

MySQL Stored Procedure Techniques This instructor-led course with focus on labs is designed
to teach you how to maximize the use of stored procedures along with the knowledge to discern
when an application should contain stored procedures and when they should not. 2 Days in length.

MySQL Developer Techniques - This instructor-led course will provide database developers with
additional skills in the design, development and maintenance of data and the queries to mine such
data. This course places an emphasis on best practices for developers that improve response time of
query executions. The database developer will also learn additional query writing techniques for
creating reports, and creating trees and hierarchies, that can be used within MySQL to support the
needs of the end users. Prerequisites: Having attended the MySQL for Developers course, or similar
knowledge. 3 Days in length.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-15

MySQL Developer Techniques

Chapter 1: Introduction

Database Administrator (DBA) Path

Introduction courses

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

MySQL for Beginners This course covers the fundamentals of SQL and relational databases,
using MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous
experience with any command-line program (such as MS-DOS) is beneficial. 4 Days in length.

Intermediate courses
o

MySQL 5.0 Upgrading and New Features This instructor-led course will provide in-depth
knowledge needed to become proficient using MySQL 5.0. This training course will provide quality
time both on the topic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL
users who want to become proficient using MySQL 5.0. 3 Days in length.

MySQL for Database Administrators This course covers essential DBA tasks such as,
installation and upgrading, user management, disaster recovery, and optimization. In addition, it will
prepare students for the MySQL Database Administrator certification. Prerequisites: Some
experience with Relational Databases and SQL. 5 days in length.

Advanced courses

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

1
n
2
e
s
d
t
o MySQL Cluster - Learn how to
andtconfigure
the cluster nodes to ensure high availability.
u
sp install
S
Also learn about all of the
high availability
features that are implemented in the new storage engine
@
s
i
n fail-over
h
for the MySQL Cluster
between storage nodes, network partitioning protocol, two-phase
a
t
r
t
e
commit and much
Prerequisites:
Attendance to the MySQL for Database Administrators or an
s
n more.
u
omastery
o
equivalent
of
database
concepts,
SQL and the MySQL server. 3 Days in length.
o
t
m
(
e
nsis the sole course offering in the Cluster certification (CMCDBA) path.
This
an Note:
r
e
c
T
li High Availability - This course is designed for experienced database administrators and
et o MySQL
o

uy
g
N

bl
a
r
fe

MySQL Performance Tuning The MySQL Performance Tuning course is designed for Database
Administrators and others who wish to monitor and tune MySQL. This course will prepare each
student with the skills needed to utilize tools for monitoring, evaluating and tuning. Students will
evaluate the architecture, learn to use the tools, configure the database for performance, tune
application and SQL code, tune the server, examine the storage engines, assess the application
architecture, and learn general tuning concepts. 4 Days in Length.

system architects that want to analyze and form a basis of understanding different high availability
options, including clustering and replication solutions within MySQL. This course will provide the
tools required to make the decision of what high availability solution is appropriate and how to
implement a system with the correct design.. 3 days in length.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-16

MySQL Developer Techniques

Chapter 1: Introduction

Introductory

MySQL
and PHP

Thi

MySQL for
Beginners

4 days

Advanced/
Intermediate
Special Topics

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Curriculum Path Chart

Certification:
CMA

4 days

MySQL for
Developers

MySQL for
DBAs

Certification:
CMDEV I-II

Certification:
CMDBA I-II

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
Developer
DBA
e

s
n
u
o
o
to
m
(
e
an icens
r
T
l
t

MySQL
Stored
Procedure
Techniques
2 days

an
r
t
n

MySQL
Developer
Techniques

3 days

MySQL
Performance
Tuning

MySQL High
Availability

4 days

3 days

MySQL
Cluster
3 days
Certification:
CMCDBA

uy
g
N

bl
a
r
fe

5 days

5 days

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-17

MySQL Developer Techniques

Chapter 1: Introduction

Virtual Classroom
MySQL also offers virtual (online) courses covering various topics in relation to the MySQL suite of tools. These
classes are instructor-led and delivered synchronously, via the web. Information regarding the available courses can
be found on the MySQL training web page.

Introductory

PHP
Elements

Thi

Coming Soon!

4 hours

MySQL
Enterprise
Monitor
4 hours

Advanced/
Intermediate
Special Topics

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

29

MySQL
Transactions

MySQL
Storage
Engines

an
s
a
h
)
n ide
v

om t Gu
c

n
21ComingdSoon!
e
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
Developer
DBA
ce
T
i
l
t
e
4 hours

Writing
MySQL UDFs

2x4 hours

4 hours

4 hours

an
r
t
n

MySQL
Partitioning

bl
a
r
fe

MySQL Proxy

4 hours

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-18

MySQL Developer Techniques

1.9

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

30

Thi

Chapter 1: Introduction

MySQL Website
Everything you ever wanted to know about MySQL and more can be found on our website: http://www.mysql.com.
From the home page, you may navigate the web site with the tabs (pull-down menus) across the top, the menu along
the left side, and/or the many links on the page.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-19

MySQL Developer Techniques

1.9.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

31

Thi

Chapter 1: Introduction

MySQL Community Web Page


The MySQL Community web page is located at http://dev.mysql.com and is maintained by Sun Microsystems, Inc.
This is the main support tool for the MySQL open source community and can provide valuable insight for Enterprise
users also. Information such as the following can be found:

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-20

MySQL Developer Techniques

Information such as the following can be found on the Developer Zone web page:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

32

Thi

Chapter 1: Introduction

Current product and service promotions


Get Started with MySQL
o Installation information page for MySQL beginners
Developing with:
o Links to specific information on using MySQL with; PHP, Perl Python, Ruby, Java/JDBC and
.Net/C#/Visual Basic
Quality Contribution Program
o Structured program for software contributions and bug reports
o Contributors acknowledged/rewarded
MySQL Server Community Edition
o All current General Available (GA), Release Candidate (RC) and Alpha versions of the MySQL
Server
New Releases
o Latest code release for all MySQL products
Software Previews
o New features available for preview by users

an
What's New
s
a
o Latest news about the company, products, services, etc.
h
)
n ide
v
MySQL Training

m Services
u
oTraining
o List of upcoming courses provided by MySQL
G
c

t
MySQL Quickpoll
21 den
s
t
tu issues
o Give your opinions about current
spMySQL-related
S
@
s
i
Stay Connected
n
h
a
t
r
t
e
o Blogs, Lists, Guilds,
s Meetups and Quality Contributions
n MySQL
u
o
o
o
Resources m
( nsArticles,
e t Training, Webinars, and MySQL Newsletter
o n
White Papers,
a more!ice
r
T
And
much
l
t

an
r
t
n

bl
a
r
fe

e
guy

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-21

MySQL Developer Techniques

1.9.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

33

Thi

Chapter 1: Introduction

MySQL Online Documentation


Documentation for all MySQL products can be found from our website, on the Documentation page:
http://dev.mysql.com/doc/ . The page includes links for downloading the following;

MySQL Reference Manual


Excerpts from the Reference Manual
MySQL GUI Tools Manuals
Expert Guides
MySQL Help Tables
Example Databases
Meta Documentation
Community Contributed Documentation
Printed Books
Additional Resources

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-22

MySQL Developer Techniques

Chapter 1: Introduction

Inline
ne Lab 1-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this exercise you will review some web pages on the MySQL website using the web browser of your
choice.

Thi

Step 1. Place cursor in web browser URL field.


Click in the text area of the Address/Location Toolbar.
The URL currently listed will be selected.
Step 2. MySQL web address
In the toolbar text area, type:
www.mysql.com

bl
a
r
fe

The MySQL Homepage is displayed.


Step 3. Products

an
r
t
n

no
a
sand downloads.
a
A list of currently available products will appear, with links for further information
h
n) ide
v

Step 4. MySQL Enterprise


om t Gu
c

non the Learn More >> link.


Review the details of the new MySQL Enterprise program,
21 by clicking
e
s
d
t
p set ofSenterprise-grade
tu
MySQL Enterprise provides a comprehensive
software, support and services
s
@
s
directly from the developers of MySQL
to
ensure
the
highest
levels
of
reliability,
security and uptime.
i
n
h
a
t
r
t
e
on to us
Step 5. Services
o
(mtab located
senear the top of the MySQL home page.
Click on the Services
n
n
a
r
e
A listtofT
currently available
lic services will appear, with links for further information and downloads.
e
y
guStep
6. MySQL Training and Certification

Click on the Products tab located at the top of the MySQL home page. Scroll down the list to review the
various product information provided.

Review the details of the MySQL Training & Certification program, by clicking on the Learn More >>
link.
Featured information on this page will be updated periodically. For specific Training sub-topics select one
of the links in the sub-menu in the upper-left corner of the page.
Step 7. Certification
From the MySQL Training & Certification web page, select the Certification link in the sub-menu and
review the contents.
Featured information on this page will be updated periodically. For specific Certification sub-topics select
one of the links in the sub-menu in the upper-left corner of the page.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-23

MySQL Developer Techniques

Chapter 1: Introduction

Step 8. Return to Services


Click on the Services tab located at the top of the MySQL home page.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Returns to the top level Services page.

Thi

Step 9. Support Services


Review the details of MySQL Support programs, by clicking on the Learn More >> link.
Shows the various support programs available.
Step 10. MySQL Community
Click on the Developer Zone tab located at the top of the MySQL home page.
The community page contains many links to more information and access to various open communication
forums.

ns
a
r
t
Return to the MySQL home page using the MySQL.com tab at the top of the page, then click on- the News
on
& Events tab located near the top of the MySQL home page.
n
a
Shows the latest MySQL happenings, from news stories to upcoming seminars. s
a
h
)
n
v uide
m
o
c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
t
e
y
gu
Step 11. News & Events

bl
a
r
fe

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-24

MySQL Developer Techniques

Chapter 1: Introduction

1.10 Installing MySQL

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

34

Thi

It is recommended to download the MySQL database server from the MySQL web site downloads page:
http://dev.mysql.com/downloads/. There are several different platforms supported, and there are
differences in the specific installation details for each.
Inline Lab 1-B
This lab requires you to use the Windows OS Wizard for installing the MySQL server.
Step 1. Install MySQL
For installations on the Windows and Linux Operating systems, see Appendix A at the back of this training
guide.
See Appendix A.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-25

MySQL Developer Techniques

Chapter 1: Introduction

1.11 Installing the 'world' database

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

35

Thi

MySQL provides three example databases to be used for testing server features and training. These
databases can be found on the MySQL website documentation page; http://dev.mysql.com/doc/ . The
world database will be used throughout this course.
Inline Lab 1-C
This lab requires you to use the MySQL command line client in order to load the world database. In order
to use the pre-made tables in the world database, you will need to create the database (empty) and then
upload the file containing the table data.
Step 1. Create 'world' database
Type the following in the MySQL command line client:
mysql> CREATE DATABASE world;

Creates the (empty) world database. Returns following message;


Query OK, 1 row affected (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
Type:
a
h
n) ide
v
mysql> USE world;

u
om
G
Instructs client to use the newly created world database. Returns
following
message;
c

t
21 den
s
Database changed
t
sp s Stu
@
i
n
h
Step 3. Build 'world' database tables
a
t
r
t
e
on to us
Type:*
o
(m nse
mysql> n
SOURCE C:/world.sql
a
rwill giveliyou
cethe file path if it is different than shown here. Several Query messages will scroll
T
* Instructor
t
ye while tables and data are being uploaded for the world database.
gupassed
Step 2. Select the 'world' database

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-26

MySQL Developer Techniques

Chapter 1: Introduction

1.12 Chapter Summary

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

36

Thi

This chapter introduced you to MySQL products and services. In this chapter, you learned:

Explain the origin and status of the MySQL product

List the available MySQL products and professional services

Describe the MySQL Enterprise subscription

List the currently supported operating systems

Describe the MySQL Community web page

Describe the MySQL Certification program

List all the available MySQL courses

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

1-27

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 2

bl
a
r
fe

an
r
t
n

IMPROVING as a no
h
)
n
PERFORMANCE
v WITH
ide
m
u
o
c nt G
1
2
INDEXES
s
de
spt Stu

uy
g
N

@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

2.1

Thi

Chapter 2 - Improving Performance with Indexes

IMPROVING PERFORMANCE WITH INDEXES


Overview
This chapter provides the knowledge and skills necessary to improve the response time of queries by
choosing and utilizing the multiple indexing options available. At the completion of this chapter, you
will be able to:

38

Describe how MySQL handles query executions

Explain why MySQL uses indexes

List when MySQL uses and index and when it does not use an index

Describe ways to retrieve index information

Explain processes to optimize indexes

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-1

MySQL Developer Techniques

2.2

Query Executions in MySQL


When a query is executed in MySQL, there are multiple
layers of the process that must be considered.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

39

Thi

Chapter 2 - Improving Performance with Indexes

MySQL Database Management Level - within


this level, the actual parsing of the the query
statement is performed along with the optimization
and execution of that statement.
Storage Engine Implementation - within MySQL
there are multiple storage engines that can be used
to house the data that MySQL database
management level will execute against. The
storage engine itself will provide benefits (and
limitations) based on their intended purpose and
design.

MySQL Database Management Level

Optimize

Parse

Execute

Storage Engine Implementation


MyISAM

InnoDB

MySQL
Cluster

Falcon

Other ...

File System

Hardware

bl
a
r
fe

File System - the file system that is being used plays a vital role in the overall performance of the
query execution. There will be advantages (and disadvantages) for each file system and care must
be taken to ensure that the file system that is chosen for MySQL to run on will improve the
performance of query executions (and other vital database responsibilities) rather than provide a
hindrance to performance.

an
r
t
n

no
a
s
a
h
and managing
nfor)actually
eexecuting
v
Hardware - last but not least, the hardware that is responsible

d
i
m role inGtheuperformance and execution of
the operating system and the installed software plays o
a key
c

t
queries against the MySQL database.
1
n
2
e
s
pt Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-2

MySQL Developer Techniques

2.2.1

Query Optimizer
The task of the query optimizer in MySQL is to find the best or optimal plan for executing the SQL
query that is submitted to the server. The query optimizer searches for the best plan among all the
possible query evaluation plans. A process called greedy optimization enables the optimization engine
to significantly reduce the amount of time it spends calculating optimal execution paths by a process of
intelligent best-path reductions. For larger more complex queries, this became a bottleneck in the
servers performance. The following representation demonstrates the Query Process:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

40

Thi

Chapter 2 - Improving Performance with Indexes

MySQL DBMS
Query Compiler
Parser
Analyze Syntax
Preprocessor
Semantic checking, name resolution

Optimizer
generate
optimalQuery
QueryExecution
ExecutionPlans
Plans(QEPs)
(
Generate
Optimal

bl
a
r
fe

ns

Parse-tra
Tree
non

a
s
a
Query Transformations
h
)
n
v uide
m
o
cplan nt G
Search for optimal execution
1
2
e
pts Stud
s
@
is
Plan
refinement
Query
n
h
a
t
r
t
e
Execution
on to us
o
Plan
(m nse QEP
n
a ice
r
T
l
et

uy
g
N

Query execution engine

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-3

MySQL Developer Techniques

2.3

Why Indexes?
There is a cost associated with a query
execution needing to interact with a layer
outside of itself in the form of time. When a
storage engine has to be accessed, there is a
time associated with reading the data. When a
storage engine has to interact with the file
system (such as cache misses on the file
system), there is a time associated with that
interaction. When the file system must interact
with components of hardware that are slower
than others (such as disk seeks versus being
able to retrieve the data stored in memory)
there is a loss in time to complete a query.
With each of these there are steps that can be
followed to improve performance, but for the
most part the greatest gain in query execution
performance involves indexes.
Indexes
Indexing increases the performance of the database when it comes to data-access performance by allowing
quick access to the rows in a table. In addition, indexes can have positive effects on UPDATE and DELETE
operations by providing a faster means to locate the rows affected. Indexes are created using one or more
columns of a table. In addition to the index being smaller than the original table (due to having fewer
columns), indexes are optimized for quick searching, usually via a balanced tree. When indexes are not used
or are not matched by SQL statements submitted to the database, then a full-table scan is executed. A fulltable scan will read all rows in a table to find a specific row or set of rows, which can be extremely inefficient.

bl
a
r
fe

42

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
Full Table Scan vs. Utilizing Indexes

100,000

10,000

uy
g
N

Milliseconds

1,000
Full Table Scan
Utilizing Indexes

100

10

65
,5
36

32
,7
68

8,
19
2
16
,3
84

4,
09
6

2,
04
8

1,
02
4

51
2

25
6

12
8

64

32

16

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

41

Thi

Chapter 2 - Improving Performance with Indexes

Number of Rows

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-4

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

43

Thi

Chapter 2 - Improving Performance with Indexes

Index Issues
The following list decribes some index issues to be aware of:

mysql>

Speed versus maintenance Indexes help to speed up data retrieval but are expensive to
maintain. As indexes are added, the time to write to the data is increased to maintain index integrity.

UNIQUE indexes A UNIQUE index creates a constraint such that all values in the index
must be distinct. An error occurs if a new row of data is added with a key that matches an
existing row. If an index is unique, use the UNIQUE clause to force the column to be distinct
which in turn also improves the efficiency of searching on that column.

Index selectivity - The more selective an index is, the more benefit is obtained from using it.
The more unique values in an indexed field, the better chance that the index has of finding the
value being searched in an effective amount of time. Index selectivity of a field can be
determined by using the following SQL statement:

SELECT 1.0/NULLIF(COUNT(DISTINCT(index_field)),0) FROM table;

bl
a
r
The NULLIF is placed into the SQL to ensure that there is no chance of a divide by zero error.
sisfe
n
A response of 0.50 is the worst possible result and states that the effectiveness of thea
index
r
extremely poor. A response as close to 0.00 as possible is the best resultnto-tensure the
o
effectiveness of the index.
n
areference
s
For large datasets, this SELECT statement can be expensive to run
(in
to the time it
a
h
would take). Using the INFORMATION_SCHEMA database,
the
following
SQL
will list the
n) ide
v
percentage of rows that are not unique:

om t Gu
c

mysql> SELECT TABLES.TABLE_SCHEMA, TABLES.TABLE_NAME,


21 den
s
-> STATISTICS.INDEX_NAME,
t
sp s Stu'%') AS `Rows per value`
-> CONCAT((1 - CARDINALITY/TABLE_ROWS)*100,
@
-> FROM information_schema.STATISTICS
i
n
h
a
t
-> JOIN information_schema.TABLES
r
t
e
->
USING (TABLE_SCHEMA,
us
on ANDtoTABLE_NAME)
o
-> WHERE non_unique=1
TABLE_ROWS/CARDINALITY IS NOT NULL;
(m nse
n
a
- Highly duplicated data should not be indexed (for example, boolean data
r Cardinality
ceand columns
T
i
l
t
types,
that represent gender, state abbreviations, or country codes). However,
e
y
having a heavily skewed data distribution can make indexes useful when looking for some
u
g

values and not useful when looking for others.

Altering the execution of a query


In a situation where the end user understands the data, MySQL provides two means to alter the execution of the
query:
USE INDEX(index_list) - This clause will tell the MySQL server to only evalute the indexes in the
index_list to determine which index (if any) would be the best to use for the query. If the MySQL server
believes that a full table scan is the most appropriate execution to provide the best results, this clause makes that
possibility available.
FORCE INDEX(index_list) - This clause is identical to the USE INDEX clause but will force MySQL to
choose one of the indexes in the index_list. A full table scan would never be performed with this clause even if
the MySQL server felt it would provide the best results. This clause would be appropriate in those times when the
cardinality data type field indexed is searching for the more unique values in the field.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

Chapter 2 - Improving Performance with Indexes

Short keys - Keys that are shorter in length have benefits over longer keys due to the sheer
speed of eliminating the need to scan a long value. However, too short of a key reduces the
possibility of having index selectivity.

Integer data types - Keys that are based on the integer data type make the best indexes not only
for index operations but also for join and other types of database operations.

Dead Indexes - Make sure to avoid indexes that are never used by any queries. These cause
additional overhead that is not necessary, and removing them will improve overall efficiency,
especially during updates, deletes and inserts.

Duplicate indexes - Avoid more than one index on the same column(s). The optimizer must
determine which to use. Also there is more maintenance as the data changes.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

44

Thi

Chapter 2 - Improving Performance with Indexes

Composite Indexes
MySQL uses multiple-column indexes in such a way that queries are fast when a known quantity is
specified for the first column of the index in a WHERE clause, even if values are not specified for the
other columns. The following list of characteristics describe composite indexes:

Indexes can be created on several columns.

They can be used for searches on just the first column(s).

An index on (a,b,c) can be used for searches on [(a), (a,b), (a,b,c)].


In summary, a composite index spans multiple columns where each column is sorted based on the value
in the preceeding column. For example, the following describes what a composite index on the
Country table which includes the Continent, Region and Name columns would look like when it
is stored:
Continent
Antarctica
Antarctica
...
Africa
Africa
Africa
Africa
Africa
Africa
...
Africa
Asia
Asia
...
Asia
Europe
Europe
Europe
Europe
Europe
Europe
...

Region
Antarctica
Antarctica

Name
Antarctica
Bouvet Island

Central Africa
Central Africa
Central Africa
Eastern Africa
Eastern Africa
Eastern Africa

Equatorial Guinea
Gabon
Sao Tome and Principe
British Indian Ocean Territory
Burundi
Comoros

Eastern Europe

Belarus

an
r
t
n

an
s
a
h
)
n ide
v

Western Africa
Togo
om t Gu
Eastern Asia
China1c
2 Kong den
s
Eastern Asia
Hong
t
sp s Stu
@
i
Southern and Central
n Asia thUzbekistan
a
r
t
Baltic Countries
Estonia
e
n
us Latvia
oCountries
Baltic
o
o
t
Baltic Countries
Lithuania
(m
e
s
n
n
British
Islands
Ireland
a
r
e
c
T
i
British
Islands
United Kingdom
l
et

uy
g
N

bl
a
r
fe

In this scenario, the Continent column is sorted and then the Region column is sorted within the
Continent column and finally the Name is sorted within the Region column. This indexing method
allows the MySQL server to quickly locate records when the first column is called along with the other
columns in an ascending fashion. This type of index would not be useful if the third column was
searched without searching the first and second columns.
Leftmost index prefixes
In a table that has a composite (multiple column) index, MySQL can use leftmost index prefixes of that index. A
leftmost prefix of a composite index consists of one or more of the initial columns of the index. MySQL's capability
to use leftmost index prefixes enables you to avoid creating unnecessary indexes. In the composite index example
above, the index on column a along with the index on columns a and b are considered leftmost prefix indexes.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-7

MySQL Developer Techniques

2.4
45

When MySQL Uses an Index


The following list entails a checklist of sorts that describes when the MySQL server will actually use an
index:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 2 - Improving Performance with Indexes

Testing an index against a constant value - When an index value is tested against a constant,
the MySQL server will utilize the applicable index:

mysql > SELECT ... FROM Country WHERE Code = 'NOR';

Joining against an indexed column - When an index value is tested against a joined field, the
MySQL server will utilize the applicable index:

mysql> SELECT ... FROM Country

JOIN City ON Capital = Id;

Leftmost index prefixes - When utilizing indexes, the MySQL server allows for the use of the
leftmost part of a prefix index only. In the following example, the CountryLanguage table
has a composite index on the CountryCode and Language columns. In the following
example only the leftmost part of that composite index (the prefix index) is being utilized:

bl
a
r
fe

ns
a
r
t
-= 'USA';
n
mysql> SELECT Language, Percentage FROM CountryLanguage WHERE CountryCode
o
a nstatements. The only
s
LIKE searches - Index values can be searched using wildcards with
LIKE
a
h
limitation to this method is the wildcard can not be the first character:
)
n ide
v

u
mysql> SELECT Name, Code FROM Country WHERE Code
LIKE 'Ma%';
om
G
c

t
n
21 serverdeprovides
s
MAX() and MIN() searches - ThetMySQL
for MAX() and MIN() functions
p Stu
s
to utilize indexes as well:
@ this
n
a
r
t City;use
mysql> SELECT MAX(Id) FROM
n
o
oresults - tThe
o MySQL server will utilize the index to produce a result set that has
m
(
Presorting
e
n sortedewhen
ns the column being sorted is an indexed field:
abeen
r
c
T
li
t
e
y
mysql>
SELECT
Name
FROM City ORDER BY Id LIMIT 5;
gu

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-8

MySQL Developer Techniques

2.4.1
46

When MySQL Does Not Use an Index


The following list entails a checklist of sorts that describes when the MySQL server will not use an
index:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 2 - Improving Performance with Indexes

Specific wildcard searches - As stated earlier, the MySQL server will not use an index when a
wildcard search utilizes a LIKE statement that begins with a wildcard:

mysql> SELECT Name FROM Country WHERE Code LIKE '%M';

Later parts of a composite index - The MySQL server will not use an index if the first part of
a composite index is not used. Thus in the following example, the first part of the composite
index on the CountryLanguage table is not being used and thus the composite index made
up of the CountryCode and Language columns will not be used:

bl
a
r
fe

mysql> SELECT CountryCode, Language, Percentage FROM CountryLanguage


> WHERE Language = 'French';

an
r
t
n

Ignoring indexes for full-table scan - The MySQL server may, in some cases, calculate that a
full table scan is cheaper to execute, even though it may fill the conditions for using indexes:

no
a
s
a
h
) one eindex
Multiple index operations - If a statement is designed ton
utilize
for a WHERE clause
v

d
i
and one index for an ORDER BY clause, the MySQL
server
will
pick
one of the multiple
m Gu
o
operations to use and index for:
c

t
1
n
2
e
s
mysql> SELECT Name FROM City WHERE Id BETWEEN
pt St10udAND 50 ORDER BY Name;
s
@ clause
s ORDER BY clause exist, the MySQL server will use
In the case where both a n
WHERE
iand
h
a
t
r
the index for the WHERE
If no WHERE clause exists, the MySQL server may use the
t clause.
seThe
n
u
index for the o
ORDER
BY clause.
exception to this is if there is a composite index where the
o
o
t
first part
can
be
used
for
the
WHERE
clause
and the second part can be used for the ORDER BY
m
( nse
n
clause:
a
r
ce
T
i
l
t
e SELECT Language, Percentage FROM CountryLanguage WHERE CountryCode = 'GBR'
mysql>
guy > ORDER BY Language;
mysql> SELECT Name FROM City WHERE Id > 10;

Functions and Expressions - If a statement is designed with a function or expression in its


WHERE clause, any indexed columns imbedded in the function or expression will not be
utilized:

mysql> SELECT Name FROM Country WHERE LEFT(Code,1)='A';

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-9

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Quiz 2-A Objective

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Using the following table, evaluate each query and determine if the query optimizer will be able to
use the indexes assigned:

Thi

CREATE TABLE t1 (
a INT,
b INT,
c INT,
d INT,
INDEX (a, b, c),
INDEX (d)
);

1.

SELECT d FROM t1 WHERE d = 10

Yes

No

2.

SELECT a FROM t1 WHERE a = 10

Yes

No

3.

SELECT c FROM t1 WHERE c = 10

4.

SELECT d FROM t1 WHERE d = 10 ORDER BY a

ns
a
r
t
n- Yes
Yes

o
an

s
a
h
)
e
6. SELECT a FROM t1 WHERE a = 10 AND b = 5vn
d
i
m Gu
o
c

7. SELECT a FROM t1 WHERE a = 10 AND


1 c = e17nt
2
s
pt Stdud= 5
8. SELECT a FROM t1 WHERE as = 10 AND
@ this
n
a
r
t WHERE
9. SELECT a FROMnt1
sae= 1 AND b = 1 AND c = 5
u
o
o
o
tWHERE
m
(
e
10. SELECT
a
FROM
t1
a = 1 AND b = 1 AND c = 5 AND d = 1
n ens
a
r
c
e11.t TSELECT lai FROM t1 WHERE a = 10 GROUP BY b
5.

guy

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

12. SELECT MAX(c) FROM t1 WHERE a = 10 GROUP BY b

Yes

No

13. SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b

Yes

No

14. SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b

Yes

No

15. SELECT a FROM t1 WHERE a = 10 AND b < 3 ORDER BY c

Yes

No

SELECT a FROM t1 WHERE a = 10 ORDER BY b

bl
a
r
fe No

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-10

MySQL Developer Techniques

2.5
47

Retrieving Index Information


In MySQL, there are a multiple ways to retrieve information regarding the indexes that are assigned to
tables within the databases. The following are the four most common approaches:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 2 - Improving Performance with Indexes

SHOW COLUMNS FROM - This command provides information about the columns in a table. The
DESCRIBE command is a shortcut for the SHOW COLUMNS FROM command. These commands
provide a quick view of the columns in the table referenced along with a small number of
additional fields that provide data types of the columns, if the column can accept NULL values, if
the column is indexed, the default value for the column and any additional information that are
considered important concerning the column.
The LIKE clause can be used with this SHOW command to limit the output to only those columns
desired (ex. LIKE 'P%' to obtain on those columns that start with the letter 'P')

mysql> SHOW COLUMNS FROM City;


+-------------+----------+------+-----+---------+----------------+
| Field
| Type
| Null | Key | Default | Extra
|
+-------------+----------+------+-----+---------+----------------+
| ID
| int(11) | NO
| PRI | NULL
| auto_increment |
| Name
| char(35) | NO
|
|
|
|
| CountryCode | char(3) | NO
|
|
|
|
| District
| char(20) | NO
|
|
|
|
| Population | int(11) | NO
|
| 0
|
|
+-------------+----------+------+-----+---------+----------------+
5 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

Key field valuesm


u
o
G
c

t
The Key field that is displayed in numerous of the commands1
n table and/or column information can have one
2 used to ddisplay
e
s
of four values associated with it:
t
sp onlySastua secondary column in a multiple-column, non-unique
Empty - the column either is not indexed or
is indexed s
@
i
n
h
index.
a
t
r
t
e
or is uonesof the columns in a multiple-column PRIMARY KEY.
PRI - the column is a PRIMARY
onKEY
o
o
UNI - the column is the
first columneofta unique-valued index that cannot contain NULL values.
m
(
s
n multiple
noccurrences
athat
MUL - this means
of a given value are allowed within the column. The column is the first column
r
e
c
T
i
l
t
of a non-unique
index
or
a
unique-valued
index
that can contain NULL values
e
y
u
g
SHOW CREATE TABLE - This command displays the SQL syntax that was used to create the
N
48
i
h
table referenced. This can be useful in being able to see the indexing that is set up along with
additional information such as the data types being used and any comments associated with the
table.

mysql> SHOW CREATE TABLE City\G


*************************** 1. row ***************************
Table: City
Create Table: CREATE TABLE `city` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
...
`Population` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`ID`),
) ENGINE=MyISAM AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1 COMMENT='This stores the cities
associated with the world database'
1 row in set (#.## sec)

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-11

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

49

Thi

Chapter 2 - Improving Performance with Indexes

SHOW INDEX FROM - This command returns detailed table index information to include the
column indexes, the cardinality (estimate of the number of unique values in the index) and the
type of index used.

mysql> SHOW INDEX FROM City\G


*************************** 1. row ***************************
Table: City
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: ID
Collation: A
Cardinality: 4079
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:

50

SELECT ... FROM INFORMATION_SCHEMA.STATISTICS - The MySQL


data dictionary implementation (INFORMATION_SCHEMA) is the place that
stores information about all the other databases that the MySQL server maintains.
The STATISTICS table provides information about table indexes. The output
from this table is equivalent to the SHOW INDEX FROM command.
The following SQL statement displays a few of the columns associated with the
STATISTICS table to show the similarity with the SHOW INDEX FROM
command.

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
mysql> SELECT INDEX_NAME, COLUMN_NAME, tSEQ_IN_INDEX,
p Stu
-> CARDINALITY, INDEX_TYPE
s
@
-> FROM INFORMATION_SCHEMA.STATISTICS
is
n
h
a
t
->
WHERE TABLE_NAME='City'
AND
TABLE_SCHEMA='world';
r
t use
n
+-------------+-------------+--------------+-------------+------------+
o
o
| INDEX_NAME | COLUMN_NAME
t|oSEQ_IN_INDEX | CARDINALITY | INDEX_TYPE |
m
(
e
+-------------+-------------+--------------+-------------+------------+
s |
| PRIMARY ran| ID en
1 |
4079 | BTREE
|
c
T
i
+-------------+-------------+--------------+-------------+------------+
l
etin set (#.## sec)
1 y
row
u
g

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-12

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Lab 2-B

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will use some of the SQL commands presented to retrieve information concerning the
world.CountryLanguage table.

Thi

Step 1: Verify that you are in the world database


In a the mysql client, type the following commands:
mysql> USE world;
mysql> SHOW TABLES;
+-----------------------+
| Tables_in_world_clean |
+-----------------------+
| city
|
| country
|
| countrylanguage
|
+-----------------------+
3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v
mysql> DROP DATABASE world;

m Gu
o
c

t
mysql> CREATE DATABASE world;
1
n
2
e
s
pt Stud
s
mysql> USE world;
@ this
n
a
r
mysql> SOURCE C:/world.sql
t use
n
o
o
tofile may different than the one displayed, please change it accordingly.
The location of(m
the world.sql
e
n indexes
nsof the CountryLanguage table using MySQL SHOW commands
e
StepT2:ra
View theic
l
t
e
y
In
the
mysql
client,
the following command to view the columns and associated information using
gu the SHOW COLUMNStypeFROM
command:

You may have more tables than shown here in your output. This can be corrected by removing the world
database and recreating it with the following commands:

mysql> SHOW COLUMNS FROM CountryLanguage;


+-------------+---------------+------+-----+---------+-------+
| Field
| Type
| Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| CountryCode | char(3)
| NO
| PRI |
|
|
| Language
| char(30)
| NO
| PRI |
|
|
| IsOfficial | enum('T','F') | NO
|
| F
|
|
| Percentage | float(4,1)
| NO
|
| 0.0
|
|
+-------------+---------------+------+-----+---------+-------+
4 rows in set (#.## sec)

You may wish to shorten the characters typed by using the DESC command (which is the shorter version
of the DESCRIBE command which is equivalent): DESC CountryLanguage;
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-13

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Utilize the SHOW CREATE TABLE command to see even more detailed information concerning the
CountryLanguage table to include the indexes associated with the table:

Thi

mysql> SHOW CREATE TABLE CountryLanguage\G


*************************** 1. row ***************************
Table: CountryLanguage
Create Table: CREATE TABLE `countrylanguage` (
`CountryCode` char(3) NOT NULL DEFAULT '',
`Language` char(30) NOT NULL DEFAULT '',
`IsOfficial` enum('T','F') NOT NULL DEFAULT 'F',
`Percentage` float(4,1) NOT NULL DEFAULT '0.0',
PRIMARY KEY (`CountryCode`,`Language`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

This explains the reason that both the CountryCode and the Language columns were both identied
as primary keys in the SHOW COLUMNS command. From this output, it is easy to see that both of these
columns are used to make up the primary key.
Utilize the last of the SHOW commands presented by typing the following command in the mysql client:

bl
a
r
fe

mysql> SHOW INDEX FROM CountryLanguage\G


*************************** 1. row ***************************
Table: CountryLanguage
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: CountryCode
Collation: A
Cardinality: NULL
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:
*************************** 2. row ***************************
Table: CountryLanguage
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 2
Column_name: Language
Collation: A
Cardinality: 984
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:
2 rows in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

The reason the first column does not show a cardinality is due to the fact that it is the first part of a multicolumn primary key. In this case, the cardinality will be associated with the last column of the multicolumn primary key.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-14

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Step 3: View the indexes of the CountryLanguage table using INFORMATION_SCHEMA database

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the mysql client, type the following command to view the indexes associated with the
world.CountryLanguage table using the INFORMATION_SCHEMA database:

Thi

mysql> SELECT INDEX_NAME, COLUMN_NAME, SEQ_IN_INDEX,


-> CARDINALITY, INDEX_TYPE
->
FROM INFORMATION_SCHEMA.STATISTICS
->
WHERE TABLE_NAME='CountryLanguage' AND TABLE_SCHEMA='world';
+------------+-------------+--------------+-------------+------------+
| INDEX_NAME | COLUMN_NAME | SEQ_IN_INDEX | CARDINALITY | INDEX_TYPE |
+------------+-------------+--------------+-------------+------------+
| PRIMARY
| CountryCode |
1 |
NULL | BTREE
|
| PRIMARY
| Language
|
2 |
984 | BTREE
|
+------------+-------------+--------------+-------------+------------+
2 rows in set (#.## sec)

bl
a
r
fe

From the SEQ_IN_INDEX column, it is easy to see which column is first in the multi-column primary
key index and which one is second. Again, this command is equivalent to the SHOW INDEX command
but for many, the INFORMATION_SCHEMA tables are easier to work with.

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-15

MySQL Developer Techniques

2.6

Optimize Indexes
Due to the potential for indexes to be a source of great return on investment when it comes to MySQL
performance, the following tricks of index optimization are presented:

51

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 2 - Improving Performance with Indexes

Prefix of Column Indexes- An index on a whole column is not always necessary. Instead, it may
be best to create an index on just a prefix of a column. Prefix indexes take less space and the
operations are faster. Prefix index for data selective by first few characters to help the optimizer
determine statistically if it is useful for a given query.

Character types only


Prefixing of column indexes works only with character data types. In addition, prefixes usually produce a noticeable
time saving when applied to longer character data types.

Composite Indexes - Composite indexes can be used for searches on the first column(s) in the
index. In addition, if a query is using MIN, MAX or ORDER BY on secondary (or subsequent)
columns it can also see improvements in performance due to the composite index.

bl
Primary Key Minimizing the size of the primary keys that are used as references in other tables ra
fe
can gain incremental performance benefits. Additionally, using an AUTO_INCREMENT column
s
n
can be more optimal for the primary key column.
tra
n
Covering index - A covering index, which may be a composite index, includes
of the columns
noofall this,
referenced in the SELECT, JOIN, and WHERE clauses of a query. Because
the index
a
s
a
contains the data being looked for and MySQL server doesn't have
to
look
up
the
actual
data in
h
)
the table, reducing logical and/or physical I/O, thus boosting
performance.
While
non-covering

vn indexes
composite indexes can hinder performance, covering composite
idecan be very useful, and in
m
u
o
many cases, really boost the performance of a query
data reads.
G
c by saving
t
1
n
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-16

MySQL Developer Techniques

Chapter 2 - Improving Performance with Indexes

Further Practice 2-C

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this further practice, you will utilize the fundamentals of improving indexes that were presented
in this chapter.

Thi

1.
2.

If necessary, install a fresh copy of the world database to remove any changes that may have
been made to the data or the data structures.
Create three new indexes in the world.Country table:
a.

A unique index named uName that indexes the Name column.

b.

A composite index named Cont_Pop that indexes the Continent and Population
columns together.
A unique index named uCapital that indexes the Capital column.

c.
3.
4.

Create a new index in the world.City table against the CountryCode column.
Determine which indexes (if any) the MySQL query optimizer would consider using in the
following SQL statement:

bl
a
r
fe

an
r
t
n

mysql> SELECT DISTINCT Country.Region FROM Country INNER JOIN CountryLanguage ON


> CountryLanguage.CountryCode = Country.Code WHERE Language = 'Spanish';

no
a
Based on the query execution plan, is the CountryLanguage table thes
first to be evaluated (to
a
h
look up the language 'Spanish')? Why or why not?
n) ide
v

________________________________________________________________________________
m Gu
o
c

t
________________________________________________________________________________
1
n
2
e
s
ud optimizer would consider using in the
5. Determine which indexes (if any)sthe
ptMySQLSquery
t
following SQL statement:
@ this
n
a
r
t FROM
seCountry WHERE Population > 200000000;
n
mysql> SELECT Name, Continent
u
o
o
to
m
(
e
s
Basedn
on the query
plan, are there any indexes that could be utilized in the execution of
norexecution
aquery?
r
e
this
Why
why
not?
c
T
li
t
e
y
________________________________________________________________________________
gu
________________________________________________________________________________
Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer
would consider using now:
mysql> SELECT DISTINCT Continent FROM Country WHERE Population > 200000000;

Based on the output, would this modified SQL statement change the query execution plan? Why or
why not?
________________________________________________________________________________
________________________________________________________________________________

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-17

MySQL Developer Techniques


6.

Chapter 2 - Improving Performance with Indexes

Determine which indexes (if any) the MySQL query optimizer would consider using in the
following SQL statement:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> SELECT CountryCode, MAX(Population) FROM City GROUP BY CountryCode;

Thi

Would the CountryCode index, which was created in step three, be considered in the execution of
this query? Why or why not?
________________________________________________________________________________
________________________________________________________________________________
Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer
would consider using now:
mysql> SELECT CountryCode, MAX(Population) FROM City WHERE CountryCode LIKE 'C%'
> GROUP BY CountryCode;

bl
a
r
fe

Based on the output, would this modified SQL statement change the query execution plan? Why or
why not?

an
r
t
n

no
a
________________________________________________________________________________
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
lice
t Tr
________________________________________________________________________________

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-18

MySQL Developer Techniques

2.7

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

52

Thi

Chapter 2 - Improving Performance with Indexes

Describe how MySQL handles query executions

Explain why MySQL uses indexes

List when MySQL uses and index and when it does not use an index

Describe ways to retrieve index information

Explain processes to optimize indexes

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

2-19

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 3

bl
a
r
fe

an
r
t
n

no
SEARCHING DATE
a
s
a
h
n) ide
v
AND TEXT oFIELDS

m
u

uy
g
N

c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

3
3.1

SEARCHING DATE AND TEXT FIELDS


Overview
This chapter provides the knowledge and skills necessary to improve the writing and use of queries that search
date and text fields. At the completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

54

Thi

Chapter 3: Searching Date and Text Fields

Search for a string in multiple columns

Compare case sensitivity for text and binary searches

Utilize INET_ATON and INET_NTOA functions

Convert string dates to SQL dates

Convert dates to integer values

Convert a date to a string

Search for a non-specific date

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-1

MySQL Developer Techniques

3.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

55

Chapter 3: Searching Date and Text Fields

Strings in Multiple Columns


There are times when developers (either by inheriting existing code or attempting to simplify their own
code) will create SQL statements that look harmless but turn out to be anything but harmless. Searching
for strings in multiple columns can lead to many expensive SQL statements that cost the end user in time
and performance.
CONCAT in a WHERE clause
Once of the ways that developers have avoided using multiple search means to come up with the result
set they are looking for is by using the CONCAT function in a WHERE clause. For example, to find the
rows that contain any users with a first name that begins with the letter 'M' and contains the last name
'Keitel', it might make sense to use a CONCAT function:

mysql> SELECT actor_id FROM actor


-> WHERE CONCAT(LEFT(first_name,1),last_name) = 'MKeitel';
+----------+
| actor_id |
+----------+
|
74 |
|
198 |
+----------+
2 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
This would produce the result intended, but it might be a good practice
to
determine
how the database
) iteispossible
nfunction,
56
v
server actually performed the requested query. With the EXPLAIN
to remove the

d
iserver.
m
u
guess work and see exactly how the queries are going to be
executed
by
the
o
c nt G
1
2
ts WHEREtude
mysql> EXPLAIN SELECT actor_id FROM actor
p
s
-> CONCAT(LEFT(first_name, 1), last_name)
@ this S = 'MKeitel'\G
*************************** a
1.nrow ***************************
tr se
id: 1

n
u
select_type: SIMPLEo
o
o
t
table: actor
m
( nse
type:
n ALL
a
r
possible_keys:
NULL
ce
T
i
l
t
key:
NULL
NULL
uye key_len:
g
ref: NULL
N
rows: 200
hi
Extra: Using where
1 row in set (#.## sec)

Using the CONCAT function results in a problem, are there any keys that could be used that are not being
used? This query would result in a full table scan, which for all practical purposes is not the most
effective means to return results.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

57

Chapter 3: Searching Date and Text Fields

Now that this query did not work as well as expected, it is time to try a more traditional means to search
multiple columns:
mysql> EXPLAIN SELECT actor_id FROM actor
> WHERE LEFT(first_name, 1) = 'M' AND last_name = 'Keitel'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: actor
type: ref
possible_keys: idx_actor_last_name
key: idx_actor_last_name
key_len: 137
ref: const
rows: 3
Extra: Using where
1 row in set (#.## sec)

bl
a
r
The results of this query would take advantage of an index containing the idx_actor_last_name
sfe
key and improve the response time of a large data set immensely by avoiding a full table scan. a
So,neven
-trapproach.
though it may look cleaner, using the CONCAT function in the WHERE clause is not then
best
o
However, is the LEFT function in this WHERE clause still limiting the query from
n reaching its full
a
potential. The last query to review with the EXPLAIN function may be the best
possible
solution to take
s
a
58
h
full advantage of all possible indexes:
n) ide
v

u
mysql> EXPLAIN SELECT actor_id FROM actor
o=m'Keitel'\G
G
c

t
> WHERE first_name LIKE 'M%' AND last_name
21 den
*************************** 1. row ***************************
s
t
id: 1
sp s Stu
select_type: SIMPLE
@
i
n
h
table: actor
a
t
r
t
e
type: ref
on to us
possible_keys: idx_actor_last_name,idx_actor_first_name
o
key: idx_actor_last_name
(m nse
n
key_len:
137
a
rref: const
ce
T
i
l
t
rows: 3
uye Extra:
Using where
g
N
1 row in set (#.## sec)
i
h

The potential results may not have changed much for this particular query, but removing the LEFT
function did provide the MySQL server with another possible key to utilize in providing the most
efficient results.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-3

MySQL Developer Techniques

3.3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

59

Chapter 3: Searching Date and Text Fields

Case Sensitivity
Any text search in MySQL is based on the character set and collation that is being used. You can review
which collations are being used in the server by issuing the following SQL statement:

mysql> SHOW VARIABLES LIKE 'coll%';


+----------------------+-------------------+
| Variable_name
| Value
|
+----------------------+-------------------+
| collation_connection | latin1_swedish_ci |
| collation_database
| latin1_swedish_ci |
| collation_server
| latin1_swedish_ci |
+----------------------+-------------------+
3 rows in set (#.## sec)

Types of searches
Searches can be made in a case sensitive, case insensitive or binary fashion. The corresponding collations
will end in _cs, _ci and _bin. The following SQL statement demonstrates how a case insensitivity
collation works:

bl
a
r
fe

mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin';


+------+---------+
| Code | Name
|
+------+---------+
| FIN | Finland |
+------+---------+
1 row in set (#.## sec)

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

1
n
2
e
s
d
t
To alter the collation setting for the currentp
60
ttheu following SQL can be used:
s connection,
S
@
s
i
n = latin1_general_cs;
h
mysql> SET COLLATION_CONNECTION
a
t
r
t
e
on to us
o
With the current
(mconnections
e collation set to case-sensitive, the following SQL should produce an empty
s
n
set:
n
a
r
ce
T
i
l
t
ye SELECT Code, Name FROM Country WHERE Code = 'Fin';
umysql>
+------+---------+
g
hi N | Code | Name |
+------+---------+
| FIN | Finland |
+------+---------+
1 row in set (#.## sec)

The fact that the SQL statement actually worked and did not produce an empty set could be an issue if it
was not for the fact that each column's defined collation can override the collation of the connection (or
table, database, server, etc.).

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-4

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this case, the Country.Code column has a collation that is different from the current connections
collation:

Thi

mysql> SHOW FULL COLUMNS FROM Country WHERE Field='Code'\G


*************************** 1. row ***************************
Field: Code
Type: char(3)
Collation: latin1_swedish_ci
Null: NO
Key: PRI
Default:
Extra:
Privileges: select,insert,update
Comment:
1 row in set (#.## sec)

61

bl
a
r
fe

CHARSET defaults
When a DEFAULT CHARSET clause is part of the table creation syntax and there is no defined collation,
a default collation will be implied based on the defined character set. The default collation for the
latin1 character set (and all character sets) can be seen by issuing the following SQL statement:

an
r
t
n

o
n
a
mysql> SHOW COLLATION;
s
+----------------------+----------+-----+---------+----------+---------+
a
h
| Collation
| Charset | Id | Default | Compiled
n) i|dSortlen
e |
v
+----------------------+----------+-----+---------+----------+---------+

u|
| big5_chinese_ci
| big5
|
1 | Yes om
| Yes
1 |
G
c

t
... ... ...
1
n
| latin1_swedish_ci
| latin1
|
8
|
1 |
s|2Yes tud|eYes
t
p
... ... ...
s
S
+----------------------+----------+-----+---------+----------+---------+
@
s
i
n
h
126 rows in set (#.## sec) ra
t
t
e

s
n
o to u
odisplay,
As seen in (this
the default collation connected to the latin1 character set is
m
eThus,
s
n
latin1_swedish_ci.
the collation is case insensitive for the table no matter what the
n
a
r
e
c
T
i
connection
collation
may
be.
l
et
y
u
g

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-5

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Alter a column's collation


The collation for a column can be specifically set in the table definition by identifying the collation to be
used either in the table creation or afterwards by altering the table:

62

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> ALTER TABLE Country MODIFY Code CHAR(3) COLLATE latin1_general_cs;

Thi

This SQL has overwritten the default collation and defined a case sensitive collation to the Country
table. Issuing the same SQL presented earlier will now produce the intended result of an empty set:
mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin';
Empty set (#.## sec)

To actually return a result, the SQL must be specifically set to include case sensitivity when searching
against the table. The following SQL takes into account the case sensitive collation that the Country
table is using:
mysql> SELECT Code, Name FROM Country WHERE Code = 'FIN';
+------+---------+
| Code | Name
|
+------+---------+
| FIN | Finland |
+------+---------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-6

MySQL Developer Techniques

3.3.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

63

Thi

Chapter 3: Searching Date and Text Fields

Binary searches
The difference between binary comparisons versus collation based (text searches) is that there is no need
to consider sort order or multiple characters evaluating to the same sort-value. Thus binary comparisons
will produce better performance; however, all the specific features associated with language and
character sets will not be available.
Binary searches are accomplished by forcing one side of the comparison operation to become binary thus
forcing the whole comparison into a binary context. To demonstrate how this would work, the following
SQL returns the Country table back to a case insensitive collation:

mysql> ALTER TABLE Country MODIFY Code CHAR(3) COLLATE latin1_general_ci;

With the Country table being case insensitive again, the following SQL will produce the desired output
without taking into account that the actual search does not have the same case as the stored data:
mysql> SELECT Code, Name FROM Country WHERE Code = 'Fin';
+------+---------+
| Code | Name
|
+------+---------+
| FIN | Finland |
+------+---------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

By adding the BINARY clause to either side of the comparison


uthe MySQL server is forced to
om operation,
G
c
make the SELECT statement a binary comparison operation:

t
21 den
s
t
tu = BINARY 'Fin';
mysql> SELECT Code, Name FROM Country
Code
spWHERE
S
Empty set (#.## sec)
@
s
i
n
h
a
t
r
t
e
mysql> SELECT Code, Name
us WHERE BINARY Code = 'Fin';
onFROMtoCountry
o
Empty set (#.## sec)
(m nse
n
a
r
ce
T
i
l
t
e
guy

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-7

MySQL Developer Techniques

Possible poor performance


There is a possible down side to using the BINARY clause with an SQL statement that could utilize an
index that is associated with the content of the comparison operation. The following SQL describes how
the SQL server would execute the binary comparison operation previously shown:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

64

Thi

Chapter 3: Searching Date and Text Fields

mysql> EXPLAIN SELECT Code, Name FROM Country WHERE BINARY Code = 'Fin'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 239
Extra: Using where
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

Even though the Code column in the Country table is the primary key, the binary comparison
operation will perform a full table scan because the index was not created for binary comparisons. This
can be corrected by utilizing a binary collation.

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-8

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Inline Lab 3-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will search for values in multiple columns from the world.CountryLanguage
table.

Thi

Step 1: Verify that you are in the world database


In a the mysql client, type the following commands:
mysql> USE world;
mysql> SHOW TABLES;
+-----------------------+
| Tables_in_world_clean |
+-----------------------+
| city
|
| country
|
| countrylanguage
|
+-----------------------+
3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v
mysql> DROP DATABASE world;

m Gu
o
c

t
mysql> CREATE DATABASE world;
1
n
2
e
s
pt Stud
s
mysql> USE world;
@ this
n
a
r
mysql> SOURCE C:\world.sql
t use
n
o
o
tofile may different than the one displayed, please change it accordingly.
The location of(m
the world.sql
e
n indexes
nsof the CountryLanguage table using MySQL SHOW commands
e
StepT2:ra
View theic
l
t
e
y
In
the
mysql
client,
the following command to view the columns and associated information using
gu the SHOW COLUMNStypeFROM
command:

You may have more tables than shown here in your output. This can be corrected by removing the world
database and recreating it with the following commands:

mysql> SHOW COLUMNS FROM CountryLanguage;


+-------------+---------------+------+-----+---------+-------+
| Field
| Type
| Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| CountryCode | char(3)
| NO
| PRI |
|
|
| Language
| char(30)
| NO
| PRI |
|
|
| IsOfficial | enum('T','F') | NO
|
| F
|
|
| Percentage | float(4,1)
| NO
|
| 0.0
|
|
+-------------+---------------+------+-----+---------+-------+
4 rows in set (#.## sec)

You may wish to shorten the characters typed by using the DESC command (which is the shorter version
of the DESCRIBE command which is equivalent): DESC CountryLanguage;
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-9

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Step 3: Search the columns of the CountryLanguage table using functions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the mysql client, type the following command to view how the MySQL server will execute the
SELECT statement that is utilizing functions in the WHERE clause:

Thi

mysql> EXPLAIN SELECT CountryCode FROM CountryLanguage


-> WHERE LEFT(CountryCode,1)='D' AND Language = 'German'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: CountryLanguage
type: index
possible_keys: NULL
key: PRIMARY
key_len: 33
ref: NULL
rows: 984
Extra: Using where; Using index
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

Even though the EXPLAIN output shows that there are indexed columns that are part of the WHERE
clause, none of the indexes can be used due to a function being used. However, reviewing how the
MySQL server will execute the following SELECT statement reveals something interesting:

an
s
ha
mysql> EXPLAIN SELECT CountryCode FROM CountryLanguage n)
v uide
-> WHERE CountryCode LIKE 'D%' AND LEFT(Language,1)='G'\G
m
o
*************************** 1. row ***************************
c nt G
id: 1
1
2
e
select_type: SIMPLE
pts Stud
table: CountryLanguage
s
@ this
type: range
n
a
possible_keys: PRIMARY
r
t
se
key: PRIMARY n
u
o
o
key_len: 3
to
m
(
ref: NULL
e
n 23 ens
rows:
a
r
T
Using
lic where; Using index
tinExtra:
e
1 y
row
set (#.## sec)
gu

Here, the left portion (CountryCode) of the composite index (CountryCode,Language) is not
using a function and thus the index can be used for this SELECT statement. Of course, not using the
function in the second portion of the index would have a positive affect on a larger table.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-10

MySQL Developer Techniques

3.4
65

Chapter 3: Searching Date and Text Fields

INET_ATON and INET_NTOA


Storing network addresses can be made more efficient by using the INET_ATON and INET_NTOA
functions.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

INET_ATON (InterNET Address TO Network address)

Thi

Given the dotted-quad representation of a network address as a string, this function returns an integer
that represents the numeric value of the address. A 4-byte address (213.136.52.29 as an example)
would be represented in the following binary depiction:
11010101.10001000.00110100.00011101 (4-byte representation)
The following is an example of how a 4-byte IP address would be converted using INET_ATON:
mysql> SELECT INET_ATON('213.136.52.29');
+----------------------------+
| INET_ATON('213.136.52.29') |
+----------------------------+
|
3582473245 |
+----------------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

o
n
a
The generated number is always in network byte order. For the example just
s shown, the number is
a
calculated as 213256 + 136256 + 52256 + 29.
h
n) ide
v
What is this number?

u
oism
c
The number that is returned (in this case 3582473245),
a 10 t G

n
21 that dcame
e
number base representation of the binary numbers
s
t
u
p SQL
tstatement
from the IP address. For example, the following
srepresentation
S
converts the number returned to an
binary
of
the 10
@
s
i
h
a
t
based number:
r
t use
n
o
o
to
mysql> SELECT CONV(3582473245,10,2);
m
(
e
+----------------------------------+
an icens
r
| CONV(3582473245,10,2)
|
T
l
t
+----------------------------------+
e
y11010101100010000011010000011101 |
gu|+----------------------------------+
3

1 row in set (#.## sec)

Here the number provided by the INET_ATON function is converted from the 10 base to binary
representation which connects with the IP address.
In this case the returned result is
11010101100010000011010000011101 which is equivalent to 11010101.10001000.
00110100.00011101 which is equivalent to 213.136.52.29.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-11

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Short form IP addresses

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

66

Thi

In addition to the 4-byte addresses, the INET_ATON function understands short form IP addresses:
mysql> SELECT INET_ATON('127.0.0.1'), INET_ATON('127.1');
+------------------------+--------------------+
| INET_ATON('127.0.0.1') | INET_ATON('127.1') |
+------------------------+--------------------+
|
2130706433 |
2130706433 |
+------------------------+--------------------+
1 row in set (#.## sec)

Storing INET_ATON values


When storing values generated by INET_ATON(), it is recommended that you use an INT UNSIGNED column. If you
use a (signed) INT column, values corresponding to IP addresses for which the first octet is greater than 127 cannot be
stored correctly.

an
r
t
n

INET_NTOA (InterNET Network TO Address)

Given a numeric network address, returns the dotted-quad representation of the address as a string.

an
s
a
mysql> SELECT INET_NTOA(3582473245);
h
)
+-----------------------+
n ide
v

| INET_NTOA(3582473245) |
om t Gu
+-----------------------+
c

| 213.136.52.29
|
21 den
s
+-----------------------+
t
1 row in set (#.## sec)
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
guy

bl
a
r
fe

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-12

MySQL Developer Techniques

3.5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

67

Chapter 3: Searching Date and Text Fields

Searching Dates
Similar to strings in multiple columns, dates can become an issue for developers who wish to improve
the look and effectiveness of their code. The issue is which searches provides the results required but
also using the most effective means to return those results.
Month and year
One of the most common searches against a date column is obtaining all the rows for a specific month in
a specific year. Now like any good programming language, SQL provides multiple ways to obtain these
results. The first way that some developers have attempted to search for dates is to use a similar LIKE
function that is used with strings:

mysql> EXPLAIN SELECT inventory_id FROM rental WHERE return_date LIKE '2006-02%';
Warning (Code 1292): Incorrect datetime value: '2006-02%'
for column 'return_date' at row 1

bl
a
r
Not exactly the output desired. This of course is an incorrect statement in relation to dates. Even though
fe
s
n
this approach may seem logical, it is not proper SQL.
ra
t
n
YEAR and MONTH functions
no
68
a
The next possible solution for solving the searching for dates dilemma is to
date functions in the
s useseems
a
WHERE clause. Using the YEAR and MONTH functions in the WHEREh
clause
like a reasonable
)

n
solution, but what does EXPLAIN say:
v uide
m
o
c nt G
mysql> EXPLAIN SELECT inventory_id FROM rental
1
2
-> WHERE YEAR(return_date)=2006 AND
de
u
ptsMONTH(return_date)=2\G
t
*************************** 1. row s
***************************
S
@
s
id: 1
i
n
h
select_type: SIMPLE
tra use t
table: rentalon
type: ALL o
to
m
(
e
possible_keys: NULL
ns
key:
an NULL
r
e
c
T
NULL
li
tkey_len:
e
ref: NULL
y
rows: 15609
gu
N
Extra: Using where
i
h
1 row in set (#.## sec)

This query will result in a full table scan, no indexes will be evaluated due to the functions modifying the
value of return_date. This would result in the correct output but a very expensive means to achieve
it. Could a more creative approach work better.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-13

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Getting between the dates

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

69

The most likely solution to obtain the results expected while also utilizing an available indexes is by
using the BETWEEN clause to evaluate a date between two dates. In this case, the dates are the beginning
of the month and the end of the month:
mysql> EXPLAIN SELECT inventory_id FROM rental
-> WHERE return_date BETWEEN '2006-02-01' AND '2006-02-28'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: rental
type: range
possible_keys: return_date
key: return_date
key_len: 8
ref: NULL
rows: 2614
Extra: Using where
1 row in set (#.## sec)

bl
a
r
fe

ns
a
r
t
Success, indexes are used. In this case, the rows will be evaluated utilizing o
annindex with a
n Here are some
return_date key; however, do we actually get the results expected. Maybe
not.
a
s
issues to consider:
a
70
h
)
into account those
nwould inot
Leap years - The dreaded month of February. This query
take
e
v

d
years where February 29 appears.
om t Gu
c

1 is a DATETIME
n
Last day of the month - Since return_date
column, a DATE implicitly
2
e
s
d
t
converted to DATETIME is midnight
on that date.
This would result in not seeing any rentals
u
p
t
s ands2006-02-28
S
between 2006-02-28 00:00:00
23:59:59.
@
i
n
h
a
t
r
Compensating for these potential
sethe following query would result in the correct output but void
nt issues,
u
the index usage: oo
to
m
(
e
s
ninventory_id
an SELECT
mysql> EXPLAIN
FROM rental WHERE return_date
r
e
c
T
i
l
->t BETWEEN '2006-02-01
00:00:00' AND
AS DATETIME) - INTERVAL 1 SECOND \G
ye> CAST('2006-03-01 00:00:00'
u***************************
g
1. row ***************************
id: 1
hi N
th

select_type: SIMPLE
table: rental
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 16352
Extra: Using where
1 row in set (#.## sec)

This query takes into account every possible second of the last day of February by utilizing the first
day of March minus 1 second. This may be extreme, but there is no chance of missing data or
incorrect data being displayed.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-14

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

End of month report

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

71

The next logical step to consider with the rental query up to this point is running it every single month.
This is a common practice and using some creative (and valid) SQL can ensure that the results are
easily and accurately obtained. The following SQL uses three functions called DATE_FORMAT,
CURDATE and LAST_DAY in the WHERE clause that provides a valid date for the query:
mysql>
->
->
->
->
->
->
->
->
->

SELECT inventory_id FROM rental WHERE return_date BETWEEN


CAST(DATE_FORMAT(
CURDATE(), '%Y-%m-01 00:00:00'
)
AS DATETIME)
AND
CAST(DATE_FORMAT(
LAST_DAY(CURDATE()),'%Y-%m-%d 23:59:59'
)
AS DATETIME);

bl
a
r
fe

an
r
t
n

This query would take the current day (no matter what day it is in the month) and determine what the
first day of the month would be (using the DATE_FORMAT function) and the last day of the month
would be (using the LAST_DAY function). These two values are then connected to their proper time
for each day and then that data is evaluated to produce the correct and accurate output. This could
easily be stored in a stored procedure or an external application to eliminate having to type this at the
end of every month.

no
a
s
a
h
n) ide
v

com nt Gu

Is CAST()1needed?
s2however,
diteis a good practice to be explicit when you work
t
The CAST() function in the above queries is not needed;
u
p
t
s
S the right thing. The CAST() function will return results
with data type conversions rather than hope that
the server
does
@
s
i
n
h
as strings.
tra se t

n
o to u
o
m
n ( ense
a
r
T
lic
t
e
uy
g
hi N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-15

MySQL Developer Techniques

3.6

String Dates to SQL Dates


Converting a string date to an actual date within MySQL can be accomplished with the STR_TO_DATE
function. This function is the opposite of the DATE_FORMAT function. The STR_TO_DATE function
takes a string date as the first argument followed by the format type (of the string date) as the second
argument. The following demonstrates how the this function works:

72

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 3: Searching Date and Text Fields

mysql> SELECT STR_TO_DATE('07/14/2007','%m/%d/%Y');


+--------------------------------------+|
| STR_TO_DATE('07/14/2007','%m/%d/%Y') |
+--------------------------------------+
| 2007-07-14
|
+--------------------------------------+
1 row in set (#.## sec)

bl
a
r
fe

The second argument in this function tells the STR_TO_DATE function that the characters up to the first
forward slash ('07') identify the month, the characters between the first forward slash and the second
forward slash ('14') identify the day and the 4 characters after the second forward slash ('2007') identify
the 4-digit year. The MySQL server then displays the date in the ISO format for dates.

no
a
The following chart displays the specifier syntax and the description for the most
s common date and time
a
h
format types.
n) ide
v

Specifier
Description
om t Gu
%a
Abbreviated weekday name (Sun..Sat)
c

1
n
%b
Abbreviated month name (Jan..Dec) 2
e
s
d
t
%c
p Stu
Month, numeric (1..12)
s
%D
@
s ... 31st)
Day of the month with
English suffix i(1st
n
h
a
t
%d
r
Day of the month,
e
t numeric
s(01..31)
n
%e
u
Day of the
month, numeric
(1..31)
o
o (000000..999999)
to
%f
m
Microseconds
(
e
%H
(00..23)
ns
an Hour
r
e
c
T
li numeric (00..59)
Minutes,
t%i
e
y
%j
Day
of year (001..366)
u
Format Specifiers

N
Thi

an
r
t
n

%k

Hour (0..23)

%l
%M

Hour (1..12)
Month name (January..December)

%m

Month, numeric (01..12)

%r
%S

Time, 12-hour (hh:mm:ss followed by AM or PM)


Seconds (00..59)

%T
%U

Time, 24-hour (hh:mm:ss)


Week (01..53), where Sunday is the first day of the week

%W

Weekday name (Sunday..Saturday)

%w
%Y

Day of the week (0=Sunday..6=Saturday)


Year, numeric, four digits

%y
%%

Year, numeric (two digits)


A literal % character

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

73

Chapter 3: Searching Date and Text Fields

Illegal entries
In the event that the values entered in the first argument for the STR_TO_DATE function do not line up
with the format expected from the second argument, the response will be a NULL value:
mysql> SELECT STR_TO_DATE ('07/14/2007','%y/%m/%d');
+---------------------------------------+
| STR_TO_DATE ('07/14/2007','%y/%m/%d') |
+---------------------------------------+
| NULL
|
+---------------------------------------+
1 row in set, 1 warning (#.## sec)
mysql> SHOW WARNINGS;
+-------+------+-----------------------------------------------------------------+
| Level | Code | Message
|
+-------+------+-----------------------------------------------------------------+
| Error | 1411 | Incorrect datetime value: '07/14/2007' for function str_to_time |
+-------+------+-----------------------------------------------------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
p Stu
s
No
TIME
like
the
present
@
74
is
n
h
a
t
r
The resulting data type from
function is the DATETIME data type. This of course
t the STR_TO_DATE
se can handle
n
u
implies that the STR_TO_DATE
function
time string conversions also. The following
o
o
o
t
example demonstrates
how
this
would
work:
m
n ( ense
a
r
T STR_TO_DATE
lic
mysql>t SELECT
('1:23 PM','%l:%i %p');
e
y
+------------------------------------+
gu| STR_TO_DATE ('1:23 PM','%l:%i %p') |
N
hi +------------------------------------+
In the STR_TO_DATE function above, the second argument tells the function to expect a two digit year
in the first two characters of the string. That would have been met with no problems; however, the
characters after the first forward slash and before the second forward slash ('14') are to contain a value
that would be appropriate for a numeric month identifier. In this case, the numeric identifier '14' is not a
valid month entry and causes the function to error out. The third set of characters after the second
forward slash are ignored due to the function already erroring out; however, this would of also caused
problems due to no month containing 2007 days.

| 13:23:00
|
+------------------------------------+
1 row in set (#.## sec)

In this example, the time of '1:23 PM' is converted to the ISO format of 13:23:00 using the format
structure of hour (%1) followed by a colon and minutes (%i). In addition, due to this time not identifying
if it is one in the morning or one in the afternoon, the string contains the identification of PM which is
specified by %p in the format string.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-17

MySQL Developer Techniques

3.7

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

75

Chapter 3: Searching Date and Text Fields

Dates to Integer Values


Within standard SQL, there is a function called TO_DAYS that converts a date into an integer value.
The resulting integer value is the number of days that have occurred since the year 0000. The
following example shows the integer value associated with the acquisition of MySQL AB by Sun
Microsystems:

mysql> SELECT TO_DAYS('2008-02-26');


+-----------------------+
| TO_DAYS('2008-02-26') |
+-----------------------+
|
733463 |
+-----------------------+
1 row in set (#.## sec)

Calculating date differences in days


76

Using the TO_DAYS SQL function, an end user can calculate the number of days between two
different dates. The following example shows the length of days from the first release of MySQL
(version 1.0 released internal to the company) to the acquisition of MySQL AB by Sun Microsystems:

bl
a
r
fe

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
tu a similar result:
spthat swillSproduce
With standard SQL there is another function
@
77
i
n
h
a
t
r
t
e
mysql> SELECT DATEDIFF('2008-02-26','1995-05-23');
on to us
o
+-------------------------------------+
(m nse
| DATEDIFF('2008-02-26','1995-05-23')
|
n
+-------------------------------------+
a
r
e
T
|
4662 |
lic
t
e
+-------------------------------------+
y
gu1 row in set (#.## sec)
N
hi
mysql> SELECT TO_DAYS('2008-02-26') - TO_DAYS('1995-05-23');
+-----------------------------------------------+
| TO_DAYS('2008-02-26') - TO_DAYS('1995-05-23') |
+-----------------------------------------------+
|
4662 |
+-----------------------------------------------+
1 row in set (#.## sec)

an
r
t
n

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-18

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

78

Chapter 3: Searching Date and Text Fields

Calculating date differences with a date


The FROM_DAYS function displays the year, month and days associated with the integer value entered.
The function can be considered the matching function for the TO_DAYS function:
mysql> SELECT FROM_DAYS(4662);
+-----------------+
| FROM_DAYS(4662) |
+-----------------+
| 0012-10-06
|
+-----------------+
1 row in set (#.## sec)

This value displays a date which is equivalent to 12 years, 10 months and 6 days. This length of time
is an approximate length that shows how long in a more readable format (versus 4,662 days) of the
date difference from when MySQL version 1.0 was released until Sun Microsystems acquired MySQL
AB. It is considered approximate because the months between and leap years are not taken into
account exactly, so plus or minus up to 2 days should be considered the range.
For any date difference that is 365 days or less, the following output will be the result:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
Proleptic
Gregorian
calendar
e

s
n
u
o
o from thetoJulian to the Gregorian calendar has had to discard at least ten days during the
Every country that has switched
m
(
switch. There are no
dates between
seOctober 4 and October 15. This discontinuity is called the cutover. Any dates before
n
n
a
r
e
the cutover are
TJulian, andlicany dates following the cutover are Gregorian. Dates during a cutover are non-existent.
t
e
y applied to dates when it wasn't actually in use is called proleptic. Thus, if we assume there was never a
Au
calendar
g
cutover
and Gregorian rules always rule, we have a proleptic Gregorian calendar. This is what is used by MySQL, as is
hi Nrequired by standard SQL. For this reason, dates prior to the cutover stored as MySQL DATE or DATETIME values must
mysql> SELECT FROM_DAYS(365);
+----------------+
| FROM_DAYS(365) |
+----------------+
| 0000-00-00
|
+----------------+
1 row in set (#.## sec)

be adjusted to compensate for the difference. It is important to realize that the cutover did not occur at the same time in
all countries, and that the later it happened, the more days were lost. For example, in Great Britain, it took place in 1752,
when Wednesday September 2 was followed by Thursday September 14. Russia remained on the Julian calendar until
1918, losing 13 days in the process, and what is popularly referred to as its October Revolution occurred in November
according to the Gregorian calendar.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-19

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Inline Lab 3-B

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will use some of the tools you learned in these last few sections to answer questions
related to countries and the date of their independence from data stored in the Country table.

Thi

Step 1: Review the data type for the column we are going to reference
Using the world database, type the following command in the mysql client:
mysql> SHOW COLUMNS FROM Country LIKE 'I%';
+-----------+-------------+------+-----+---------+-------+
| Field
| Type
| Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+-------+
| IndepYear | smallint(6) | YES |
| NULL
|
|
+-----------+-------------+------+-----+---------+-------+
1 row in set (0.00 sec)

bl
a
r
This shows that the year of indepence (IndepYear) for a country is an integer data type versus a date fe
ns
data type. Any queries involving this column and dates will need to take this into account.
a
r
t
n
Step 2: Convert the IndepYear Integer data type to a valid date type
o
an
With the IndepYear value being an integer that contains only the 4 digit year,
converting the value to a
s
a
h
date takes some creative SQL. There are multiple ways to accomplish
this. The following command is
)
n ide
one way to accomplish this task:
v

u
om FROM
G
c

t
mysql> SELECT IndepYear, FROM_DAYS(IndepYear*365.25)
Country
LIMIT 5;
1
n
2
e
+-----------+-----------------------------+
s
pt | Stud
| IndepYear | FROM_DAYS(IndepYear*365.25)
s
+-----------+-----------------------------+
@ this|
|
1919 | 1919-01-16 ran
t use |
|
1581 | 1581-01-12
n
o
|
NULL | NULL o
|
to
m
|
1912 | 1912-01-16
|
(
e
s
|
1962a|n1962-01-16
|
n
r
e
...
c
T
i
l
t
| ye NULL | NULL
|
u
|
NULL | NULL
|
g
+-----------+-----------------------------+
239 rows in set (#.## sec)

With the year being calculated to take into account leap years (the .25 value), a date value is created that
is equivalent to a date early in the year of the independence year stored in the Country table.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-20

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Step 3: Calculate the number of years since independence of a country

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the mysql client, type the following command to calculate the current date minus the independence
year of a country to calculate how long an individual country has been independent:

Thi

mysql> SELECT IndepYear, DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25))


-> FROM Country;
+-----------+---------------------------------------------+
| IndepYear | DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25)) |
+-----------+---------------------------------------------+
|
1919 |
32787 |
|
1581 |
156242 |
|
NULL |
NULL |
|
1912 |
35344 |
|
1962 |
17081 |
|
NULL |
NULL |
...
|
NULL |
NULL |
|
NULL |
NULL |
+-----------+---------------------------------------------+
239 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v
mysql> SELECT IndepYear,

-> DATEDIFF(NOW(),FROM_DAYS(IndepYear*365.25))/365.25
om t GASu Years_Indep
c

-> FROM Country LIMIT 10;


21 den
+-----------+-------------+
s
t
| IndepYear | Years_Indep |
sp s Stu
+-----------+-------------+
@
i
h
|
1919 |
89.7659 |ran
t
t
e
|
1581 |
427.7673
n || us
oNULL
|
NULL |
o
o
|
1912 | (m
96.7666e| t
s|
|
1962a|n
46.7652
n
r
e
|
NULL
|
NULL
|
c
T
li
... et
NULL |
gu| y NULL |

The output is the number of days difference, which of course is not very useful to most end users. A
format of years would be more useful. This is accomplished with the following command:

|
NULL |
NULL |
+-----------+-------------+
239 rows in set (#.## sec)

Not only has the column name been changed to be easier to read, but also the data is easier to read with
the addition of the year divisor.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-21

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Step 4: Cleaning up the output


For the most part, the output is readable and could be used for multiple purposes if their was a need.
However, cleaning up the output to minimize any irrelevant data (like an output of years with decimal
values) would be more professional. The following command cleans up the output and provides
additional value to that output.
mysql> SELECT Name,
->
CAST(
->
(DATEDIFF(
->
NOW(), FROM_DAYS(IndepYear*365.25) ) / 365.25
->
) AS UNSIGNED
->
) AS Years_Indep
-> FROM Country WHERE IndepYear IS NOT NULL;
+----------------------+-------------+
| Name
| Years_Indep |
+----------------------+-------------+
| Afghanistan
|
90 |
| Netherlands
|
428 |
| Albania
|
97 |
| Algeria
|
47 |
| Andorra
|
731 |
| Angola
|
34 |
...
| Estonia
|
18 |
| United States
|
233 |
| Zimbabwe
|
29 |
+--------------- ------+-------------+
192 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
tu
sp s S
This output provides the name of the
country, then
uses the CAST function to turn the Years_Indep
@
n places)thfori only countries that have a value in the IndepYear
output into an integer (no decimal
a
r
t
NOT uNULL).
se Your output may be slightly different than the output
column (IndepYear n
IS
o
othe NOW() function which will affect the output based on when this
displayed here m
dueo
to the use tof
(
e
statement is
anrun.icens
r
T
l
t
e
y
u

N
Thi

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-22

MySQL Developer Techniques

3.8

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

79

Thi

Chapter 3: Searching Date and Text Fields

Dates to Strings
Extracting portions of a date field to display a string (or integer) value can be accomplished with
numerous SQL functions. The following list demonstrates a portion of these functions:

Function
DAY()
DAYOFMONTH()

Description
Returns the day of the month (1-31)
Synonym for DAY() function

Usage
DAY('2008-02-26')
DAYOFMONTH('2008-02-26')

Result
26
26

DAYNAME()
DAYOFWEEK()

Returns the name of the weekday


Returns the weekday index (1-7)

DAYNAME('2008-02-26')
DAYOFWEEK('2008-02-26')

Tuesday
3

DAYOFYEAR()
MONTH()

Returns the day of the year (1-366)


Returns the month (1-12)

DAYOFYEAR('2008-02-26')
MONTH('2008-02-26')

57
2

MONTHNAME()
QUARTER()

Returns the name of the month


Returns the quarter of the year

MONTHNAME('2008-02-26')
QUARTER('2008-02-26')

February
1

WEEK()
WEEKDAY()

Returns the week of the year


Returns the weekday index (0-6)

WEEK('2008-02-26',3)
WEEKDAY('2008-02-26')

8
1

YEAR()
YEARWEEK()

Returns the 4-digit year


Returns the year and week

YEAR('2008-02-26')
YEARWEEK('2008-02-26',3)

bl
a
r
fe

ns
a
2008
r
t
n- 200808

no
a
s
a
h
How is the first week of the year calculated?
n) ide
v

When using the WEEK() function, there are eight different modes
that can be u
used to determine how the weeks are
om
G
c

t
calculated. The modes are entered into the second argument
of
the
function
and
range from 0 (which states that if the
1week forethenyear using
2
first week of the year contains a Sunday, consider if the
zero
range of 0-53) to 7 (which states
s
t week
tuofdthe year using aarange
that if the first week contains a Monday, considersitpthe first S
of 1-53). The other modes
@
s
(1-6) provide more options to fine tune n
the
WEEK()
function.
The
WEEKDAY()
function
does not take a mode
i
h
a
t
r
t
argument and is equivalent to WEEK(date,3)
which
states
that
if
the
first
week
contains
more
than 3 days and begins
e
usingusa range of 1-53.
n
o
with a Monday, consider it the
first
week
o
to
m
(
e
an icens
r
T
l
t
e
y
u
Ng

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-23

MySQL Developer Techniques

3.8.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

80

Thi

Chapter 3: Searching Date and Text Fields

Time values to Strings


As seen, MySQL has numerous date functions that will extract specific values from the dates entered.
In addition to date functions, MySQL offers multiple time functions that can be valuable to
administrators and developers alike.

TIME() - This function will return the time portion of a date (and/or time) value. This
function should only be used with valid time values to assure accurate responses.

mysql> SELECT TIME('1955-11-12 22:04:00');


+-----------------------------+
| TIME('1955-11-12 22:04:00') |
+-----------------------------+
| 22:04:00
|
+-----------------------------+
1 row in set (#.## sec)

bl
a
r
fe

HOUR() - This function will return the hours for a day from 0 to 23 and greater if the value
given contains a greater hour value.

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
MINUTE() - This function
ithe minutes from a given time in the range of 0-59.
n will return
h
a
t
r
t
e
mysql> SELECT MINUTE('1955-11-12
on to us22:04:00');
o
+-------------------------------+
(m n
se
| MINUTE('1955-11-12
22:04:00')
|
n
a
r
e
+-------------------------------+
c
T
li
| et
4 |
y
+-------------------------------+
u
g1 row in set (#.## sec)
mysql> SELECT HOUR('1955-11-12 22:04:00');
+-----------------------------+
| HOUR('1955-11-12 22:04:00') |
+-----------------------------+
|
22 |
+-----------------------------+
1 row in set (#.## sec)

SECOND() - This function will return seconds from a given time also in the range of 0-59.

mysql> SELECT SECOND('1955-11-12 22:04:05');


+-------------------------------+
| SECOND('1955-11-12 22:04:00') |
+-------------------------------+
|
5 |
+-------------------------------+
1 row in set (#.## sec)

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-24

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 3: Searching Date and Text Fields

MICROSECOND() - This function will return the microseconds from a given time expression
as a number in the range from 0 to 999999.

mysql> SELECT MICROSECOND('1955-11-12 22:04:00.00078405');


+---------------------------------------------+
| MICROSECOND('1955-11-12 22:04:00.00078405') |
+---------------------------------------------+
|
78405 |
+---------------------------------------------+
1 row in set (#.## sec)

3.8.1.1 Storing Microsecond Precision Times


81

Microseconds can not be stored into a column of any of the MySQL temporal data types. If an attempt
is made to store a time value with a microsecond into a temporal data type, the microsecond value is
discarded.
Storing as a DOUBLE

bl
a
r
fe

an
r
t
n

One of the ways that a time value can be stored is by converting the time to a numeric Unix timestamp.
The Unix timestamp evaluates the current time (or the time provided) into an unsigned integer that
equates to the number of seconds since '1970-01-01 00:00:00'. This Unix timestamp can be evaluated
by issuing the UNIX_TIMESTAMP function:

no
a
s
a
h
n) ide
mysql> SELECT UNIX_TIMESTAMP('2008-02-26 09:00:00');v
+---------------------------------------+
om t Gu
| UNIX_TIMESTAMP('2008-02-26 09:00:00') | 1c
n
2
e
+---------------------------------------+
s
d
t
|
1204034400
sp |s Stu
+---------------------------------------+
@
i
n
h
1 row in set (#.## sec)
a
t
r
t
e
s
umicrosecond
onvalue,
Once the time is an o
integer
the
value can be added to this value and then stored in a
o
t
m
(
e
data type value
such assDOUBLE.
For example, the following table could be used to store the
n that ea web
n page was requested by a browser and the actual time that the server took to
atime
requested
r
c
T
i
l
t the request and begin a response back to the browser:
eevaluate
y
u
g
mysql> CREATE TABLE web_request (
->
web_page VARCHAR(100),
->
request_time DOUBLE(16,6),
->
response_time DOUBLE(16,6)
-> );
Query OK, 0 rows affected (#.## sec)

With this table in place, statistics can begin to be evaluated for each web page requested to the
microsecond (up to 6 values) based on the DOUBLE data type being used (16 precision of significant
digits being stored, with up to 6 of those digits being stored after the decimal point).

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-25

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

82

Chapter 3: Searching Date and Text Fields

Continuing with the example, the web_request table that was created will be filled with a record
that will store the requested time that a web page on the mysql.com domain server received the
request for access to one of its pages and when the server started the response back to the browser
requesting the page:
mysql> INSERT INTO web_request VALUES (
->
'www.mysql.com/training/catalog.html',
->
UNIX_TIMESTAMP('2008-02-26 09:00:00')+.001356,
->
UNIX_TIMESTAMP('2008-02-26 09:00:00')+.978650
-> );
Query OK, 1 row affected (#.## sec)

The following query will show how the record just added was actually stored in the web_request
table:
mysql> SELECT web_page, request_time, response_time FROM web_request;
+-------------------------------------+-------------------+-------------------+
| web_page
| request_time
| response_time
|
+-------------------------------------+-------------------+-------------------+
| www.mysql.com/training/catalog.html | 1204034400.001356 | 1204034400.978650 |
+-------------------------------------+-------------------+-------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
astored
h
With these values, along with additional requests over a set period of) time,
in the web_request

n
e
v
table,
statistics
can
be
evaluated
and
monitored.
The
following
query
uses
the
one
record stored to

d
iand
83
mis requested
u
determine a simple length of time scenario from when c
ao
page
when
the
server begins
G

t
to respond in milliseconds:
1
n
s2 tude
t
p
s s ASS diff FROM web_request;
mysql> SELECT response_time - request_time
@
i
n
+----------+
h
a
t
r
t
e
| diff
|
on to us
+----------+
o
| 0.977294 |
(m nse
+----------+ n
a
r (#.##
e
1 row inTset
csec)
i
l
t
uye
g
hi N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-26

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

One of many

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

The solution to storing microseconds presented is one of many ways to accomplish this task. The following are a list of
a few other ways to accomplish this:

Thi

BIGINT - Using the BIGINT data type, the time value would be stored as an integer (which implies no decimals).
This would be accomplished by multiplying the Unix timestamp by 100,000 to make room for the microseconds to be
stored in the last six values of the integer:
mysql> SELECT UNIX_TIMESTAMP('2008-02-26 09:00:00') * 100000 + 001356;
+---------------------------------------------------------+
| UNIX_TIMESTAMP('2008-02-26 09:00:00') * 100000 + 001356 |
+---------------------------------------------------------+
|
120403440001356 |
+---------------------------------------------------------+
1 row in set (#.## sec)

mysql> SELECT '2008-02-26 09:00:00' AS sec, 978650 AS micro;


+---------------------+--------+
| sec
| micro |
+---------------------+--------+
| 2008-02-26 09:00:00 | 978650 |
+---------------------+--------+
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

Two columns - Storing the date/time value in one column and the microseconds in a second column can eliminate the
need to use Unix timestamps:

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-27

MySQL Developer Techniques

3.9

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

84

Chapter 3: Searching Date and Text Fields

Nonspecific Date Searches


There are times when a request is made to provide data that is not specific to an actual date that can be
searched easily using standard SQL. Such a request could be "Give me all the times that employees have
been absent from work the third Monday of each month for the last year." For the most part, this could
simply be an exercise in pulling out a calendar, looking for each third Monday of the months for the last
year, writing an SQL statement that searches for the specific dates that were obtained from this manual
view and then providing the results to the requestor. However, what if there was someway to figure this out
logically and let SQL handle the majority of this tedious work.
The Logic
The first day (day of the week) that a month begins on provides a lot of information that can be used to
create the logic that can be used to obtain non-specific dates. The following chart shows how the first day
of the month provides valuable information about the 1st and 3rd Mondays of the month:
First Day of Month

# Weekday

1st Monday 3rd Monday

Sunday
Monday
Tuesday

1
2
3

2
1
7

Wednesday
Thursday
Friday

4
5
6

6
5
4

16
15
21

bl
a
r
fe

an
r
t
n

20 o
n
19
a
s
a 18
h
)
n3 ide17
Saturday
7
v

om t Gu
c

The Formula
1
n
2
e
85
s
d
t
Due to the fact that there is a consistency associated
with
tuthe first day of the month affecting the date of the
spfollowing
S
1 and 3 Monday's (or any week day),
the
formula can be derived:
@
s
hi + (2 - [1 first day of week for
anday ofe tmonth]
r
3 Monday = [1
t
the month]
onMOD t7)o u+ s21
o
... and utilizing(m
syntax
se ...
n MySQL
n
a
r
e
mysql> SELECT
'2008-02-01'
+ INTERVAL ((2 - DAYOFWEEK('2008-02-01') % 7) + 21) DAY
T Third_Monday;
lic
t
>
AS
e
y
u+--------------+
g
hi N | Third_Monday |
st

rd

rd

st

st

+--------------+
| 2008-02-18
|
+--------------+

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-28

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

Further Practice 3-C

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this further practice, you will utilize the fundamentals of searching date and text fields that were
presented in this chapter.

Thi

1.

2.
3.
4.

Install a copy of the sakila database into a database with the same name. This is a sample
database provided by MySQL and found in the Example Databases section of the
http://dev.mysql.com/doc/ page. This database contains data associated with a madeup video rental store. If assistance is needed in installing this database, please contact your
instructor for specific directions.
Using the sakila database, review the structure of the customer, film, inventory and
rental tables.
Using the rental table, determine the maximum rental days where a customer checked out a
video and returned it. In addition, include the average length of days that customers (who have
returned the videos) have kept the videos.
Create a report that identifies videos that are currently out. Include in the report the customers
name and contact info, the video that they have checked out and the number of days they have had
the movie checked out (from the current date). Note: Due to this data being slightly outdated, the
days checked out will be quite lengthy.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-29

MySQL Developer Techniques

Chapter 3: Searching Date and Text Fields

3.10 Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

86

Thi

Search for a string in multiple columns

Compare case sensitivity for text and binary searches

Utilize INET_ATON and INET_NTOA functions

Convert string dates to SQL dates

Convert dates to integer values

Convert a date to a string

Search for a non-specific date

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

3-30

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 4

bl
a
r
fe

an
r
t
n

no
IMPROVING INSERTS
a
as

uy
g
N

h
)
n
v uide
m
o
c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4.1

Thi

Chapter 4: Improving Inserts

IMPROVING INSERTS
Overview
This chapter provides the knowledge and skills necessary to improve the response time of queries that are
responsible for inserting data. At the completion of this chapter, you will be able to:

88

Describe the process that takes place when an INSERT is performed

Use multiple-row insert statements to store many rows with one SQL statement

Utilize the INSERT ... SELECT statement to add/modify rows of data from one or more tables

Improve data inserts by utilizing LOAD DATA INFILE

Speed up bulk inserts with MyISAM

Speed up bulk inserts with InnoDB

Utilize MySQL extensions to standard SQL to improve data inserts

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-1

MySQL Developer Techniques

4.2

INSERT Process
When an INSERT is performed against MySQL, there is a defined process that takes place:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

89

Thi

Chapter 4: Improving Inserts

Connection to the server is opened

Query sent to the server

Query is parsed

Row is inserted into table

Index is created and inserted into index structure

Connection to the server is closed

Each of these steps has an associated cost connected to it in the form


of performance. In addition, there is an initial overhead cost that
takes place to open tables for each concurrently running query.

Connect
Send
Parse
Insert
Index
Disconnect

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-2

MySQL Developer Techniques

4.2.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

90

Thi

Chapter 4: Improving Inserts

Multiple Row Inserts


When there is a need to insert multiple rows of data, consider using INSERT
statements with multiple VALUES lists to minimize the cost overhead related to
creating a connection, sending the query to the server, parsing the server, opening the
table and then closing the table and connection. This method can improve the speed
of inserts considerably over the single row inserts method.
The following statement demonstrates how a multiple insert would look:

mysql> INSERT INTO ID_table (ID,Name) VALUES


-> (1,'Max'),(2,'Pat'),(3,'Roland'),(4,'Jeff'),(5,'Sarah');

For the most part this is a very simple approach to improving performance when
inserting multiple rows of data. However, with almost every process, there are issues
that need to be addressed and multiple-row inserts or updates are no exception. The
following is a list of characteristics of multiple row data manipulation statements:
91

Multi-row statements must fully succeed - When a transaction has multiple


rows being modified (like an INSERT statement adding multiple rows of in one
transaction), all the components of the transaction must succeed for the
individual transaction to complete. Consider the following statement:

Connect
Send
Parse
Insert
Index
Send
Parse
Insert
Index

bl
a
r
fe

Send
Parse

Insert
ns
a
r
t
n- Index

no Disconnect
a
s
a
h
mysql> INSERT INTO employees (employeeID) VALUES
n) ide
v

-> (10001),(10002),(10003),(10004),(10001),(10005);
om t Gu
c

1 inserts ewould
n take place because the next to last
2the
With a transactional storage engine, none
of
s
d
t
tu (assuming that the employeeID field is a
value would cause a duplicate value
key
sinpa primary
S
@
s
primary key). For a non-transactional
hi engine, the first four values would be inserted and
ancauseeantstorage
r
the next to last valuetwould
error preventing the last two values from being inserted
(due to the execution
us terminating after the error was encountered).
on of thetostatement
o
(m- Multiple-row
Portability
se statements are less portable and less likely to be understood by other
n
n
a
database
systems.
r
e
T
lic
t
Too big of inserts - It is important to consider the database administrator (and the database
e
guy system) when determining how much data should be committed at any given time. This is also

important when considering how much data should and can be rolled back at one time in relation
to transactional storage engines.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-3

MySQL Developer Techniques

4.2.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

92

Chapter 4: Improving Inserts

INSERT ... SELECT


With the INSERT ... SELECT syntax, it is possible to insert many rows into a table from one or
many tables. Similar to the multiple row inserts, this process can be used to minimize the cost associated
with connecting and disconnecting in the INSERT process. The SELECT statement used can be any
kind of SELECT statement, including joins and subqueries. In this case, we have created a table like
City and called it City2, then added all the column/row data from the original City table in to this
new table;

mysql> INSERT INTO City2 SELECT * FROM City;

The resulting rows will be the same as those of the original City table.
REPLACE ... SELECT

an
s
a
h
)
4.2.3 LOAD DATA INFILE
n ide
v

u
oa m
G
The LOAD DATA INFILE statement reads rows from
text file
into
c

t
1 wayento insert
93
an existing table. This method is a much more
2efficient
s
t
data then using INSERT statements (either
p individual
torudmultiple row
s
S
INSERT statements). In most cases
@wheretthe
s DATA INFILE
iaLOAD
n
h
a
statement is used, the data was
exported
using
SELECT ... INTO
r
t is theuSQL
se complement.
n
OUTFILE statement o
which
The following
o
o
t
shows an example
of
exporting
data
and
then
importing
that same data
(m nse
into another
table:
n
a ice
r
T
l
t
e
y
gumysql> SELECT LastName, FirstName, SchoolID
N
-> FROM class_roster
hi

an
r
t
n

->
->

WHERE ClassCode LIKE 'HIS%'


INTO OUTFILE 'C:/tmp/HIS_students.txt';

bl
a
r
fe

REPLACE is a MySQL extension to the SQL standard. It either inserts, or deletes and inserts. REPLACE works exactly
like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE
index, the old row is deleted before the new row is inserted. The REPLACE .. SELECT syntax can be used instead of
the INSERT ... SELECT statement to either insert a new row or update an existing row (which would actually be a
delete and followed by an insert).

Connect
Send
Parse
Insert
Insert
Insert
Insert
Insert
Insert
Insert
Index
Disconnect

mysql> LOAD DATA INFILE 'C:/tmp/HIS_students.txt'


-> INTO TABLE db.History;

The first statement uses the SELECT ... INTO OUTFILE to load the text file
'\tmp\HIS_students.txt' with all students that are enrolled in history classes (HIS is located in
the first three characters of the class code). The second statement loads the History table with the
values that were stored in the HIS_students.txt file.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-4

MySQL Developer Techniques

Chapter 4: Improving Inserts

Importing tab delimited or comma separated files:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

94

Thi

To import a data file, containing tab delimited or comma separated table data, the LOAD DATA
INFILE command can be used. Suppose there is a file called City.csv that contains the following
data:
"1", "Kabul", "AFG", "Kabol", "1780000"
"2", "Qandahar", "AFG", "Qandahar", "237500"
"3", "Herat", "AFG", "Herat", "186800"
"4", "Mazar-e-Sharif", "AFG", "Balkh", "127800"
"5", "Amsterdam", "NLD", "Noord-Holland", "731200"
"6", "Rotterdam", "NLD", "Zuid-Holland", "593321"
"7", "Haag", "NLD", "Zuid-Holland", "440900"
"8", "Utrecht", "NLD", "Utrecht", "234323"
"9", "Eindhoven", "NLD", "Noord-Brabant", "201843"
The following LOAD DATA INFILE statement could be used to import this data:
mysql> LOAD DATA INFILE 'C:/tmp/City.csv' INTO TABLE City
-> FIELDS TERMINATED BY ','
-> ENCLOSED BY '"'
-> LINES TERMINATED BY '\n';

bl
a
r
fe

an
r
t
n

no
a
s
a
h
Files on local host
n) ide
v

By default, MySQL assumes that the file is located on the serverm


statement
begins with LOAD DATA
u client host
o host. IfttheGthe
LOCAL INFILE rather than with LOAD DATA INFILE, thec
file is read from
on which the statement is
1from which
n
2
e
issued. In other words, LOCAL means local to the client
host
the
statement
is
issued.
s
dover the network to the server.In this case, the client
t its contents
u
psends
t
program reads the data file and
s
S
@
s
i
n
h
tra se t

n
o to u
o
m
n ( ense
a
r
T
lic
t
e
guy

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-5

MySQL Developer Techniques


Skipping lines in the data files

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

95

To ignore the initial part of the data file, use the IGNORE n LINES clause, where n is an integer that
indicates the number of input lines to skip. This clause commonly is used when a file begins with a row
of column names rather than data values. Suppose the file called City.csv contains header lines to
identify the data columns as shown below:
Note: Data exported from the world.City table
"ID","Name","CountryCode","District","Population"
"1", "Kabul", "AFG", "Kabol", "1780000"
"2", "Qandahar", "AFG", "Qandahar", "237500"
"3", "Herat", "AFG", "Herat", "186800"
"4", "Mazar-e-Sharif", "AFG", "Balkh", "127800"
"5", "Amsterdam", "NLD", "Noord-Holland", "731200"
"6", "Rotterdam", "NLD", "Zuid-Holland", "593321"
"7", "Haag", "NLD", "Zuid-Holland", "440900"
"8", "Utrecht", "NLD", "Utrecht", "234323"
"9", "Eindhoven", "NLD", "Noord-Brabant", "201843"
To skip the first two input lines, a statement might be written like this:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

Skipping or Transforming Column Values


ndata values read from the file before
2or1to transform
e
s
It is possible to skip columns in the data file,
d
t
tuby specifying user variables in the column list
spare available
inserting them into the table. These features
S
@
s
i
and the optional SET clause.
n
h
a
t
r
t
e
to ua user
To assign an input data n
s variable rather than to a table column, provide the name of a
o column
user variable in the o
column
list.to
If the column is assigned to a user variable but nothing else is done with
(meffectnisstoeignore the column rather than to insert it into the table. Or, by including a
the variable, the
n
a
SET clause,
r expressions
ce can be used to transform the value before inserting it.
T
i
l
t
uyeThe following example will skip the ID (first) column along with the city name and district being
mysql>
->
->
->
->

96

N
Thi

Chapter 4: Improving Inserts

LOAD DATA INFILE 'C:/tmp/City.csv' INTO TABLE City


FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 2 LINES;

concatenated together with a space inbetween:

mysql> LOAD DATA INFILE 'C:/City.txt' INTO TABLE City


-> (@skip,@Name,CountryCode,@District,Population)
-> SET name=CONCAT(@Name,' ',@District);

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

97

Thi

Chapter 4: Improving Inserts

LOAD DATA INFILE and Duplicate Rows


When new rows are added to a table with the INSERT or REPLACE statement, there are means to
control how new rows are handled containing values that duplicate unique key values already present in
the table. An error can be allowed to occur, the new rows can be ignored, or the old rows can be replaced
with the new ones. LOAD DATA INFILE affords the same types of control over duplicate rows by
means of two modifier keywords (IGNORE or REPLACE). However, its duplicate-handling behavior
differs slightly depending on whether the data file is on the server host or the client host, so the data file
location must be taken into account.
When loading a file that is located on the server host, LOAD DATA INFILE handles rows that contain
duplicate unique keys as follows:

By default, an input record that causes a duplicate-key violation results in an error and the rest
of the data file is not loaded. Records processed up to that point are loaded into the table.

If the IGNORE keyword is specified following the filename, new records that cause duplicatekey violations are ignored and no error occurs. LOAD DATA INFILE processes the entire file,
loads all records not containing duplicate keys, and discards the rest.

bl
a
r
fe

an
r
t
n

If the REPLACE keyword is specified after the filename, new records that cause duplicate-key
violations replace any records already in the table that contain the duplicated key values. LOAD
DATA INFILE processes the entire file and loads all its records into the table.

no
a
IGNORE and REPLACE are mutually exclusive. One or the other can be specified,
s but not both.
a
h
For data files located on the client host, duplicate unique key handling
is similar, except that the default
n) with
v

is to ignore records that contain duplicate keys rather than to terminate


daneerror. That is, the default
i
m
u
o
is as though the IGNORE modifier is specified. The reason
isG
that the client/server protocol does
c for this
t
1
n
not allow transfer of the data file from the client 2
host
to the server
to
be
interrupted after it has started, so
s in the tmiddle.
de
t
there is no convenient way to abort the operation
u
p
s sS
@
i KEY values
n PRIMARY
h
a
t
r
t
e
INFILE
The biggest danger with LOAD
us is the ability to replace or delete the primary key values. This is
onDATA
o
o
t
definitely a danger and(must
m be taken
e into account when performing inserts or updates using this method. With storage
s
n
engines that support
foreign keys
(such as InnoDB and Falcon), the foreign key can be created with options (such as
n
a
r
ce NULL, etc.) that handle any deletes or replacements of the primary or candidate keys.
RESTRICT,
CASCADE,
SET
T
i
l
t
This can
yeminimize the danger associated with LOAD DATE INFILE but for those storage engines that do not support
u
g
Nforeign keys, caution is still required.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-7

MySQL Developer Techniques

4.2.4

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

98

Thi

Chapter 4: Improving Inserts

Transactional Multiple Inserts


With transactional storage engines, it is possible to execute multiple insert statements in a single
transaction. This is usually performed to provide atomicity to the data (all the actions are performed or
none of the actions are performed); however, multiple insert statements in a single transaction can have a
positive effect on performance. The following SQL is an example of performing multiple insert
statements in a single transaction:

mysql>
mysql>
mysql>
mysql>
mysql>
mysql>

START TRANSACTION;
INSERT INTO city (NULL,'Bryson City','USA','North Carolina', 7800);
INSERT INTO city (NULL,'Virginia City','USA','Virginia',5467);
INSERT INTO city (NULL,'Morgantown','USA','West Virginia',11435);
INSERT INTO city (NULL,'Julian','USA','Pennsylvania',4356);
COMMIT;

Combining multiple insert statements into a single transaction can be beneficial to performance up to a
point. Not only is it more costly to rollback large transactions, anything over approximately 2 megabytes
of data will degrade performance rather than improve performance. In addition, the size of the
log_buffer must be considered when considering the size of a transaction.

bl
a
r
fe

an
r
t
n

no
a
In MySQL, autocommit is a setting that tells the server to either consider each SQL statement
a single transaction or
s toasdetermine
a
wait for the user (or system) to enter an explicit command (either COMMIT or ROLLBACK)
when all the
h
)

n
statements for the transaction have been completed. If the autocommit mode
is
enabled
(SET
AUTOCOMMIT=1),
each
v starts
dnewe connections with autocommit
i
m
SQL statement forms a single transaction of its own. By default,
MySQL
u
co nt G

enabled.
1
2 However,
signored.
de for transactional storage engines it is either
t
In non-transactional storage engines, these settingsp
are
u
t
s
S
necessary to disable the autocommit setting@
(SET AUTOCOMMIT=0)
or explicitly tell the server when a transaction
s
i
n
h
should begin. The START TRANSACTION
(or BEGIN)
t statement overrides the autocommit setting and tells the server
tracommand
e

s
everything that follows, up to an explicit
(either
COMMIT or ROLLBACK), will form a single transaction.
n
o to u
o
(m nse
n
a
r
ce
T
i
l
t
uye
g
N
START TRANSACTION, is it necessary?

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-8

MySQL Developer Techniques

4.3

Storage Engine Specifics


Most storage engines have their own specific processes that can be addressed to improve the
performance of the operations performed against them. Two of the main storage engines; MyISAM and
InnoDB are not exception. In the case of speeding up insert operations, these two storage engines will be
addressed.

99

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 4: Improving Inserts

4.3.1

Speeding up Bulk Inserts with MyISAM


One of the drawbacks of performing bulk inserts is the amount of time that takes place to accomplish
the task. Even though bulk inserts tend to be faster than other means of inserting large amounts of
data, there are still ways to fine-tune MySQL to improve the inherent performance gains from such an
operation. One of these ways is to disable non-unique index keys during the bulk insert operation.
The following discussion applies to bulk insert operations performed against tables utilizing the
MyISAM storage engine.
Disabling keys
The process by which non-unique indexes are disabled with MyISAM tables is to alter the table that
the operation will be performed against:

bl
a
r
fe

mysql> ALTER TABLE table_name DISABLE KEYS;

an
r
t
n

o
n
a
Once this statement is executed, the server itself will ignore these indexes when
the best
s determining
a
query execution plans either by using SELECT or evaluating any statements
with the EXPLAIN
h
n) ide
feature.
v

Enabling keys
om t Gu
c

The process by which non-unique indexes are enabled


n with MyISAM tables is to alter
21 (or dre-enabled)
e
s
t
the table that the operation will be performed
against:
sp s Stu
@
i
nENABLE KEYS;
mysql> ALTER TABLE table_name
h
a
t
r
t
e
s
n
userver
o
o
o
Once this statement
is
executed,
the
itself will evaluate these indexes when determining the best
t
(mplansneither
e by using SELECT
s
query execution
or evaluating any statements with the EXPLAIN
n
a ice
r
feature.
T
l
t
e
Delaying insert
y
100 u
g Bullk inserts can be improved by using the DELAYED option with INSERT. This is due to the client
N
i
h
not having to wait until the bulk insert is completed to execute other statements.

Rebuilding at one time


The advantage of disabling keys and then re-enabling them after a bulk insert operation comes in the
form of rebuilding the indexes once, rather than having the server performing a modification against
the physical storage of the indexes after each individual operation. Even though the re-enabling
operation requires the server to rebuild the indexes for the whole table, the time that it takes is much
faster than if the keys were enabled during the bulk insert operation. If there is a desire to achieve any
benefit from this process, it is best to perform these actions against an empty table.
When should this be used?
To gain benefit from this process, the rows being inserted must be significantly more than the rows already existing in
the table.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-9

MySQL Developer Techniques

4.3.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

101

Chapter 4: Improving Inserts

Speeding up Bulk Inserts with InnoDB


InnoDB Defaults
InnoDB default options are very conservative and in several situations increasing the buffers can have
a high impact on the performance. To speed up inserts on an InnoDB-only setup, the rule of a thumb is
to set the innodb_buffer_pool size to 50-80% of the computer's memory and to set
innodb_log_file_size to about 25% of the innodb_buffer_pool_size.
If the MySQL server is running on several threads, which all work on the InnoDB table(s),
performance can be gained by setting the innodb_flush_log_at_trx_commit value to 0.
This adds some risk of losing some transactions in the case of a power failure or an unclean mysqld
shutdown, but it can improve speed.
Primary key order inserts

Please note that when inserting several rows at once in InnoDB table, it is fastest, if the rows are inserted in primary key
order. This is because of the clustered index, which InnoDB always uses.
102

Disable AUTOCOMMIT

an
r
t
n

When importing data into InnoDB, make sure that the MySQL server does not have autocommit mode
enabled because that requires a log flush to disk for every insert. To disable autocommit during the
import operation, surround it with SET AUTOCOMMIT and COMMIT statements:

no
a
s
a
h
mysql> SET AUTOCOMMIT=0;
n) ide
v
... SQL import statements ...

mysql> COMMIT;
om t Gu
c

1 are fasterento import into an InnoDB table, even


2files
s
If using the mysqldump option --opt, dump
d
t
p SandtuCOMMIT
s
without wrapping them with the SET AUTOCOMMIT
statements.
@
s
i
n
h
Turn off UNIQUE_CHECKS ra
t
t onusecondary
103
se keys, table import performance can be improved by
n
If there are UNIQUE o
constraints
ooff the uniqueness
to checks during the import session:
temporarily turning
m
(
e
n ens
aUNIQUE_CHECKS=0;
r
mysql> SET
T
lic
t
... e
import
operation ...
y SET UNIQUE_CHECKS=1;
umysql>
g
hi N

104

bl
a
r
fe

For big tables, this saves a lot of disk I/O because InnoDB can use its insert buffer to write secondary
index records in a batch. Be certain that the data contains no duplicate keys. UNIQUE_CHECKS allows
but does not require storage engines to ignore duplicate keys.
Turn off FOREIGN_KEY_CHECKS

If there are FOREIGN KEY constraints in the InnoDB tables, table import performance can be
improved by turning foreign key checks off for the duration of the import session:
mysql> SET FOREIGN_KEY_CHECKS=0;
... import operation ...
mysql> SET FOREIGN_KEY_CHECKS=1;

For big tables, this can save a lot of disk I/O.


_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-10

MySQL Developer Techniques

4.4

Chapter 4: Improving Inserts

MySQL Extensions
The MySQL server contains additions to the SQL standard when it comes to functions associated with
inserts.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

105

INSERT ... SELECT - This statement syntax can quickly insert multiple rows into a table
from one or more tables. The following are some of the additional features of this statement
that should be understood:
o

INSERT IGNORE ... SELECT - The IGNORE clause will cause the statement to
ignore rows that would cause duplicate-key violations.

Same table - The target table of the INSERT statement may appear in the FROM
clause of the SELECT part of the query. In this case, MySQL creates a temporary table
to hold the rows from the SELECT and then inserts those rows into the target table.
However, the table referenced can not be a TEMPORARY table itself because
TEMPORARY tables cannot be referred to twice in the same statement.

bl
a
r
INSERT DELAYED - The DELAYED option for the INSERT statement is a MySQL extension
fe
s
n
to standard SQL that is very useful if there are transactions that cannot or need not wait
for
the
a there
trand
INSERT to complete. This is a common situation when MySQL is used for logging
n
are also periodically run SELECT and UPDATE statements that take a long
notime to complete.
a
However, the negative side to this syntax is it is not transactionally s
safe. The end user is never
a
given any response to know how long its going to take for the INSERT
to happen. If the server
h
)

n
crashes, or is forcefully shutdown, the delayed INSERT's
will
be
lost.
v uide
m
o
INSERT ... ON DUPLICATE KEY UPDATE
- ThisG
statement syntax allows a user to
c if there
t
106
1
n
perform an update against the data being2entered
is
already
an identical value in either a
e
s tuddemonstrates
t
UNIQUE index or PRIMARY KEY.
The following
how this statement syntax will
p
s sS
look in action:
@
i
n
h
a
t
r
t
e
s
n (ID,Name,Old_Name)
mysql> INSERT INTO ID_table
VALUES (1,'John','')
uOld_Name=Name,Name='John';
o
o
o
-> ON DUPLICATE
KEY
UPDATE
t
(m nse
n
a
r This statement
ce will perform an INSERT into the ID_table with a value of 1 for the ID
T
i
l
t
column 'John' for the Name column, and an empty string for the Old_Name column if a row
uye
g
in
the table does not already have an ID column with the value of 1. If there is a row with an
hi N
ID column with a value of 1 already in the table, then the following action would be performed:

mysql> UPDATE ID_table SET Old_Name=Name,Name='John' WHERE ID=1;

This syntax (INSERT ... ON DUPLICATE KEY UPDATE) eliminates the need to do two
round trips to the database of an insert or update which ultimately improves performance.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-11

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

107

Thi

Chapter 4: Improving Inserts

REPLACE - This statement syntax works exactly like INSERT, except that if an old row in the
table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is
deleted before the new row is inserted. The following demonstrates how this statement syntax
will look in action:

mysql> REPLACE ID_table (ID,Name) VALUES (1,'John');

This statement will perform an INSERT into the ID_table with a value of 1 for the ID
column and 'John' for the Name column if a row in the table does not already have an ID
column with the value of 1. If there is a row with an ID column with a value of 1 already in the
table, then the following actions would be performed:
mysql> DELETE FROM ID_table WHERE ID=1;

bl
a
r
fe

mysql> INSERT INTO ID_table (ID,Name) VALUES (1,'John');

an
r
t
n

This syntax (REPLACE) can save round trips to the database which ultimately improves
performance.

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-12

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Lab 4-A

Thi

In the following lab, large inserts will be performed against InnoDB tables with referential integrity
checking turned on. Then, the same operations will take place with the referential integrity checking
turned off to evaluate the amount of difference performing large insert operations between the two
processes.
1.

Action (you perform). Alter the existing world database tables to utilize the InnoDB storage
engine:

mysql> ALTER TABLE Country ENGINE=InnoDB;


mysql> ALTER TABLE CountryLanguage ENGINE=InnoDB;
mysql> ALTER TABLE City ENGINE=InnoDB;

bl
a
r
fe

Response: Each of the statements returns a Query OK response with the number of rows of data
that was affected by the change. These tables will need to be InnoDB before we can add foreign
keys to them (InnoDB is the most popular MySQL transactional storage engine) in steps to follow.

an
r
t
n

no
a
s
a
h
n) ide
v

mysql> CREATE TABLE City_copy LIKE City;


om t Gu
c

n
e
s21thetstructure
d
t
Response: A new table is created thatp
contains
u of the City table but with none of the
data of the table. This table will@
be s
used to test S
the inserts of large amounts of data to determine the
is checks are turned on and turned off.
n
difference in performance a
when
referential
integrity
h
t
r
tAdd foreign
sekey constraints to the original world tables that were altered in
n
3. Action (you perform).
u
o
o
to
step 1: (m
e
an icens
r
T
mysql>t ALTER TABLE
l City_copy ADD FOREIGN KEY (CountryCode) REFERENCES Country (Code);
e
y
gumysql> ALTER TABLE Country ADD FOREIGN KEY (Capital) REFERENCES City_copy (Id);
2.

Action (you perform). Create a copy of the City table structure to a new table called
City_copy:

Response: Each of the statements returns a Query OK response with the number of rows of data
that was affected by the change.
Notes: These tables now have foreign key constraints that will be checked.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-13

MySQL Developer Techniques


4.

Chapter 4: Improving Inserts

Action (you perform). Insert the contents from the City table into the City_copy table:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> INSERT INTO City_copy SELECT * FROM City;

Thi

Response: The City_copy table now contains the same number of rows as the City table. How
long did it take for the City_copy table to be populated with the City data? _________ seconds.
5.

Action (you perform). Insert the contents of the City_copy table back into itself a total of five
times by executing the following SQL:

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Response (1): The City_copy table now contains double the number of rows that it originally
contained. How long did it take for the City_copy table to insert the contents of itself
(approximately 4,000 records)? _________ seconds.

bl
a
r
fe

an
r
t
n

Note: The NULL in the SELECT statement is due to the first column being an auto_increment
primary key column. The NULL allows the column to utilize the auto_increment feature which
ensures that the contents of that column will be unique.

no
a
s
a
h
mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode,
District,
n) ide Population
-> FROM City_copy;
v

om t Gu
c

1 approximately
n four times the number of rows
Response (2): The City_copy table now2
contains
e
s
d
t
that it originally contained. How longp
did it take for
tuthe City_copy table to insert the contents of
s _________
S
itself (approximately 8,000 records)?
seconds.
@
s
i
n
h
a
t
r
t
e
n SELECT
mysql> INSERT INTO City_copy
us NULL, Name, CountryCode, District, Population
o
o
o
-> FROM City_copy;
t
(m nse
n
a
r
ceThe City_copy table now contains approximately eight times the number of rows
T
i
l
Response (3):
t
e
it originally contained. How long did it take for the City_copy table to insert the contents of
guy that
itself (approximately 16,000 records)? _________ seconds.

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Response (4): The City_copy table now contains approximately sixteen times the number of
rows that it originally contained. How long did it take for the City_copy table to insert the
contents of itself (approximately 32,000 records)? _________ seconds.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-14

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Thi

Response (5): The City_copy table now contains approximately thirty two times the number of
rows that it originally contained. How long did it take for the City_copy table to insert the
contents of itself (approximately 65,000 records)? _________ seconds.
mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Response (6): The City_copy table now contains approximately sixty four times the number of
rows that it originally contained. How long did it take for the City_copy table to insert the
contents of itself (approximately 130,000 records)? _________ seconds.
6.

bl
a
r
fe

Action (you perform). Remove the contents from the City_copy table:

mysql> TRUNCATE City_copy;

an
r
t
n

no
a
s
a
h
)
ninto
e
v
7. Action (you perform). Insert the contents from the City table
the
City_copy
table:
d
i
m
u
o
c
tG
1
mysql> INSERT INTO City_copy SELECT * FROM
City; en
2
pts Stud
s
@nowtcontains
is the same number of rows as the City table. How
Response: The City_copyntable
h
a
r
t
long did it take for the City_copy
setable to be populated with the City data? _________ seconds.
n
u
o
o Turn
8. Action (you perform).
tooff the foreign key constraints:
m
(
e
n ens
aFOREIGN_KEY_CHECKS=0;
r
mysql> SET
T
lic
t
e
guy Response: This will affect any actions against the contents of the InnoDB tables that are

Response: The data contained within the City_copy table has been deleted but the structure of the
City_copy table remains the same. This will provide us with the ability to perform a comparable test
with the foreign key constraints turned off for the remainder of this lab.

manipulated during the current session.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-15

MySQL Developer Techniques


9.

Chapter 4: Improving Inserts

Action (you perform). With the foreign key constraints turned off, insert the contents of the
City_copy table back into itself a total of five times by executing the following SQL:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Thi

Response (1): The City_copy table now contains double the number of rows that it originally
contained. How long did it take for the City_copy table to insert the contents of itself
(approximately 4,000 records)? _________ seconds.
Note: The NULL in the SELECT statement is due to the first column being an auto_increment
primary key column. The NULL allows the column to utilize the auto_increment feature which
ensures that the contents of that column will be unique.

bl
a
r
fe

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

an
r
t
n

Response (2): The City_copy table now contains approximately four times the number of rows
that it originally contained. How long did it take for the City_copy table to insert the contents of
itself (approximately 8,000 records)? _________ seconds.

no
a
s
a
h
mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode,
n) iDistrict,
e Population
v

-> FROM City_copy;


d
om t Gu
c

1 approximately
n eight times the number of rows
2contains
e
Response (3): The City_copy table now
s
d
t
that it originally contained. How long
spdid itstakeSfortuthe City_copy table to insert the contents of
@
itself (approximately 16,000 n
records)? _________
i seconds.
h
a
t
r
t
e
us NULL, Name, CountryCode, District, Population
on toSELECT
mysql> INSERT INTO o
City_copy
-> FROM City_copy;
(m nse
n
a
r
ceThe City_copy table now contains approximately sixteen times the number of
T
i
l
t
Response
(4):
e
guy rows that it originally contained. How long did it take for the City_copy table to insert the

contents of itself (approximately 32,000 records)? _________ seconds.

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Response (5): The City_copy table now contains approximately thirty two times the number of
rows that it originally contained. How long did it take for the City_copy table to insert the
contents of itself (approximately 65,000 records)? _________ seconds.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-16

MySQL Developer Techniques

Chapter 4: Improving Inserts

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> INSERT INTO City_copy SELECT NULL, Name, CountryCode, District, Population
-> FROM City_copy;

Thi

Response (6): The City_copy table now contains approximately sixty four times the number of
rows that it originally contained. How long did it take for the City_copy table to insert the
contents of itself (approximately 130,000 records)? _________ seconds.
10. Action (you perform). Evaluate the amount of time that each insert took in step 9 against each
associated insert in step 5. Which step consistently had a better response time and why?
________________________________________________________________________________
________________________________________________________________________________
________________________________________________________________________________

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-17

MySQL Developer Techniques

4.5

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

108

Thi

Chapter 4: Improving Inserts

Describe the process that takes place when an INSERT is performed

Use multiple-row insert statements to store many rows with one SQL statement

Utilize the INSERT ... SELECT statement to add/modify rows of data from one or more tables

Improve data inserts by utilizing LOAD DATA INFILE

Speed up bulk inserts with MyISAM

Speed up bulk inserts with InnoDB

Utilize MySQL extensions to standard SQL to improve data inserts

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

4-18

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 5

bl
a
r
fe

an
r
t
n

COMPLEX as a no
h
)
n
CALCULATIONS
v uide
om

uy
g
N

c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

5.1

Thi

Chapter 5:Complex Calculations

COMPLEX CALCULATIONS
Overview
This chapter provides the knowledge and skills necessary to perform simple to complex calculations using SQL.
At the completion of this chapter, you will be able to:

110

Simulate an aggregate multiplication function

Use variables to create a running total query

Utilize COALESCE to avoid divide by zero errors

Calculate the median value

Simulate the SQL RANK function in MySQL

Utilize SQL to solve complex problems

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-1

MySQL Developer Techniques

5.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

111

Thi

Chapter 5:Complex Calculations

Aggregate Multiplication
MySQL provides multiple aggregate functions that can be used to compute a single result value from a
collection of records. However, MySQL does not provide a means to create an aggregate function for
multiplication purposes. This limitation can be corrected with some creative SQL.
Using logarithms
The following formula shows that a list of logarithms that are added together is equivalent to the
logarithms of that list being multiplied together:
log(x) + log(y) + log(z) = log(x*y*z)
When the opposite function EXP is used, the resulting formula would look like this:
exp(log(x) + log(y) + log(z)) = x*y*z
This process can then be used in aggregate functions.

112

bl
a
r
fe

How can this be used?


The following chart shows the rate of return for a publicly traded company over a 10 year span:
YEAR
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007

ROR*
44%
-25%
8%
-2%
35%
57%
1%
11%
-8%
-21%

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
* Rate of return

uy With an initial investment of $1,000 in 1997, the current value of the investment of 2007 would need to
g
be calculated. The first thought may be to average the rate of return and then multiply this by the initial
N
$1,000 investment:

mysql> SELECT AVERAGE(ROR) FROM investment;

This would result in an average return of approximately 9.9% over the 10 year time span. However, this
does not take into account compound interest.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

113

Thi

Chapter 5:Complex Calculations

Calculating compound interest


The first step in calculating compound interest is to obtain the logarithm of the rate of return for each
year:
mysql> SELECT YEAR, 1+ROR AS multiplier, LOG(1+ROR) FROM investment;
+------+------------+--------------------+
| YEAR | multiplier | LOG(1+ROR)
|
+------+------------+--------------------+
| 1998 |
1.44 |
0.36464311358791 |
| 1999 |
0.75 | -0.28768207245178 |
| 2000 |
1.08 | 0.076961041136128 |
| 2001 |
0.98 | -0.020202707317519 |
| 2002 |
1.35 |
0.30010459245034 |
| 2003 |
1.57 |
0.45107561936022 |
| 2004 |
1.01 | 0.0099503308531681 |
| 2005 |
1.11 |
0.10436001532424 |
| 2006 |
0.92 | -0.083381608939051 |
| 2007 |
0.79 | -0.23572233352107 |
+------+------------+--------------------+
10 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
114
s
a
h
n) ide
v
mysql> SELECT SUM(LOG(1+ROR)) FROM investment;

+------------------+
om t Gu
c

| SUM(LOG(1+ROR)) |
21 den
+------------------+
s
t
| 0.68010599048258 |
sp s Stu
+------------------+
@
i
n
h
1 row in set (#.## sec)
a
t
r
t
e
on to us
o
This of course(only
m showssethe logarithmic results. The next step is to use the EXP function to turn the
n
logarithmic
results intonthe
multiplier that determines the return percentage with compound interest:
a
r
e
c
T
i
l
et SELECT EXP(SUM(LOG(1+ROR))) FROM investment;
y
mysql>
u
Ng+----------------------+
With the formula created and tested for the logarithm that is going to be used, the next step is to put this
formula into an aggregate function that adds all the logarithmic results together:

| EXP(SUM(LOG(1+ROR))) |
+----------------------+
|
1.9740869555715 |
+----------------------+
1 row in set (#.## sec)

This percentage, 97% and some change, is quite different than the average of 9.9% calculated by just
averaging the rate of returns.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-3

MySQL Developer Techniques


The final step is to then take this value and multiply it by the initial investment:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

115

Thi

Chapter 5:Complex Calculations

mysql> SELECT EXP(SUM(LOG(1+ROR)))*1000 FROM investment;


+---------------------------+
| EXP(SUM(LOG(1+ROR)))*1000 |
+---------------------------+
|
1974.0869555715 |
+---------------------------+
1 row in set (#.## sec)

Over ten years, even including the bad years, the initial investment of $1,000 nearly doubled. Of course,
the purpose of this example was not to show how much a person can make (or lose) in stocks, but to
show how creative SQL can be used to solve perceived limitations in the language.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-4

MySQL Developer Techniques

5.3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

116

Thi

Chapter 5:Complex Calculations

Running Total Query


In the course of almost every analytical presentation, someone will inevitably request that a running total
be presented. In the case of the aggregate multiplication example shown in the last section, an analytical
minded person may request to see a running total showing how the investment looked at the end of each
year. With external software, this could be accomplished with some creative functions to look something
like the following:
YEAR
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007

ROR
44%
-25%
8%
-2%
35%
57%
1%
11%
-8%
-21%

EOY*
$1,440.00
$1,080.00
$1,166.40
$1,143.07
$1,543.15
$2,422.74
$2,446.97
$2,716.14
$2,498.84
$1,974.09

bl
a
r
fe

an
r
t
n

no
a
s
a
h
Creative SQL
n) idcould
e create a running total
v
Sure, using an external application and some number crunching
a presenter

ulet MySQL handle it all. The


like the one displayed above, but why not just be creative
with SQLG
and
om
c

t
following SQL example uses a self-join on the investment
to
ncreate the running total requested:
21 table
e
s
d
t
sp s Stu
mysql> SELECT a.YEAR, a.ROR, EXP(SUM(LOG(1+b.ROR)))*1000
AS balance
@
i
-> FROM investment a JOIN n
investment
b ON (a.YEAR >= b.YEAR)
h
-> GROUP BY a.YEAR, a.ROR;
tra se t

n
+------+-------+-----------------+
o to u|
o
| YEAR | ROR
|m
balance
+------+-------+-----------------+
se
n |( 1439.9999976158
n
a
| 1998 | r0.44
|
e
c
T
i
l
t
| 1999
|
-0.25
|
1079.9999982119
|
e
| y
2000 | 0.08 | 1166.3999961376 |
u
g| 2001 | -0.02 | 1143.0719967363 |
* End of year

| 2002 | 0.35 | 1543.1471887807 |


| 2003 | 0.57 | 2422.7410753483 |
| 2004 | 0.01 | 2446.9684855603 |
| 2005 | 0.11 | 2716.1350175134 |
| 2006 | -0.08 | 2498.8442209692 |
| 2007 | -0.21 | 1974.0869509493 |
+------+-------+-----------------+
10 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-5

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

117

Thi

Chapter 5:Complex Calculations

Looking at how this works


For the most part, the self join is straight forward, but what is actually going on to provide the results that
is being requested? To see what is happening, the following SQL removes the SUM function and the
corresponding GROUP BY to show the contents that make up the aggregate results:
mysql> SELECT a.YEAR, a.ROR, EXP((LOG(1+b.ROR)))*1000 AS balance
-> FROM investment a JOIN investment b ON (a.YEAR >= b.YEAR)
-> ORDER BY a.YEAR, a.ROR;
+------+-------+------------------+
| YEAR | ROR
| balance
|
+------+-------+------------------+
| 1998 | 0.44 | 1439.99999761581 |
| 1999 | -0.25 | 1439.99999761581 |
| 1999 | -0.25 |
750 |
| 2000 | 0.08 | 1439.99999761581 |
| 2000 | 0.08 |
750 |
| 2000 | 0.08 | 1079.99999821186 |
| 2001 | -0.02 | 1439.99999761581 |
| 2001 | -0.02 |
750 |
| 2001 | -0.02 | 1079.99999821186 |
| 2001 | -0.02 | 980.000000447035 |
...
| 2007 | -0.21 | 980.000000447035 |
| 2007 | -0.21 | 1349.99999403954 |
| 2007 | -0.21 | 1569.99999284744 |
| 2007 | -0.21 | 1009.99999977648 |
| 2007 | -0.21 | 1109.99999940395 |
| 2007 | -0.21 | 920.000001788139 |
| 2007 | -0.21 | 790.000006556511 |
| 2007 | -0.21 | 1439.99999761581 |
| 2007 | -0.21 |
750 |
| 2007 | -0.21 | 1079.99999821186 |
+------+-------+------------------+
55 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
The resulting
one row for the first year, two rows for the second year and so on. This is
r query
ceproduces
T
i
l
t
due
to
the
first
year
resulting
in one row that meets the join condition (a.YEAR >= b.YEAR), the
e

uy second year resulting in two rows that meet the join condition, and so on. The values (represented by the
g
N
balance column) are then aggregated together to produce the running total that is displayed when the
SUM and GROUP BY clauses are used.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-6

MySQL Developer Techniques

5.4

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

118

Chapter 5:Complex Calculations

Avoiding Divide by Zero


Divide by zero is a problem in computational equations and must be dealt with when developing any
mathematical equation, either in SQL or any other programming language. When MySQL faces a
divide-by-zero error, it fails silently and produces a NULL response.

mysql> SELECT 50/0 AS response


+----------+
| response |
+----------+
|
NULL |
+----------+
1 row in set (#.## sec)

This of course may not be the best way to handle a divide-by-zero error and using creative SQL to
avoid divide-by-zero errors should be implemented.
Feeding a NULL into an equation
The COALESCE function is an ANSI standard function that takes one to numerous arguments and
provides the first non-NULL argument in response. There is an additional function called ISNULL that
acts similarly to COALESCE but is limited in that it only takes two arguments, in not an ANSI standard
function and most importantly takes on the datatype of first argument entered. COALESCE takes on
the datatype of the response (or first non-NULL value) which can eliminate potential future problems.
The following demonstrates the use of the COALESCE function:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

mysql> SELECT COALESCE(NULL, NULL, NULL, 0) AS response;


om t Gu
c

+----------+
21 den
| response |
s
t
+----------+
sp s Stu
|
0 |
@
i
n
h
+----------+
a
t
r
t
e
1 row in set (#.## sec)n
o to us
o
(mfunction
e itself does not solve the divide-by-zero error, but is the first step. The
The COALESCE
by
sNULLIF
n
n
a
second
step
is
using
the
function. The NULLIF function is an ANSI standard function that
r
e
119
c If both arguments
T
i
l
t
takes
two
arguments.
are identical, the function returns a NULL. If the two
earguments are not identical the first value is returned.
y
u
g
N
i
h
mysql> SELECT NULLIF(10,10) AS response;
+----------+
| response |
+----------+
|
NULL |
+----------+
1 row in set (#.## sec)
mysql> SELECT NULLIF(10,0) AS response;
+----------+
| response |
+----------+
|
10 |
+----------+
1 row in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-7

MySQL Developer Techniques

The final step to providing a solution to divide-by-zero errors is to combine the COALESCE and
NULLIF functions. The following produces a numeric zero when a divide-by-zero would otherwise
cause a problem:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

120

Thi

Chapter 5:Complex Calculations

mysql> SET @INPUT1 = 0;


Query OK, 0 rows affected (#.## sec)
mysql> SELECT COALESCE(50/NULLIF(@INPUT1,0),0) AS response;
+----------+
| response |
+----------+
|
0.0000 |
+----------+
1 row in set (#.## sec)

The response produced is a numeric zero because the statement 50/NULLIF(@INPUT1,0) will
produce a NULL output thus causing the COALESCE function to ignore that first response and go with
the next one: 0. However, if @INPUT1 is set to a numeric value that is not zero (thus would not
produce a divide-by-zero error), the first value of the COALESCE would not be a NULL and become
the response provided.

bl
a
r
fe

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

mysql> SET @INPUT1 = 5;


Query OK, 0 rows affected (#.## sec)
mysql> SELECT COALESCE(50/NULLIF(@INPUT1,0),0) AS response;
+----------+
| response |
+----------+
| 10.0000 |
+----------+
1 row in set (#.## sec)

an
r
t
n

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-8

MySQL Developer Techniques

5.5

Calculate the Median Value


In many cases the average value of a set of data is not a consistent sampling of the data. Wide ranges of
values can skew average results and provide an unrealistic assessment of data. Consider the following
chart representing the continents and their populations:

121

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 5:Complex Calculations

Name

Population

Asia

3,705,025,700

Africa

784,475,000

Europe

730,074,600

North America

482,993,000

South America

345,780,000

Oceania

30,401,150

Antarctica

bl
a
r
fe

an
r
t
n

Based on the data in this chart, the average population for the continents is 868,392,779. A quick glance
at the data reveals this is not accurate. Asia is so much larger in population than the other continents
along with Antarctica having no population. This results in a skewed outcome and not representative of
the data. A more accurate calculation would be the median value.

no
a
s
a
h
Calculating the median value
n) ide
v

The median value is calculated by using the middle valueoofm


a data set.
Ifuthere was a grouping of values
G
c

t
that consisted of 1,3,5,20,99, the median value would
5. In the
the population chart, the single
ncaseIfof there
21ofbe 482,993,000.
e
s
d
middle value is North America with a population
was an even number of
t
p Stu located in the center
continents, this would require averagingsthe two populations
of the data. Easy
@
s
i
enough to determine when doing
it
by
hand,
but
how
is
this
done
with
SQL.
n
h
tra use t
n
Multiple paths to theo
median
o
o to accomplish the same task. Some are more efficient than others,
tways
m
In MySQL there
are multiple
(
e
nsto create. Either way, in MySQL determining the median value can be
while rothers
an areiceasier
e
T
accomplished
inl one of three ways:
t
e
y
u

N
Thi

Creating an implicit temporary table

Creating a stored procedure

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-9

MySQL Developer Techniques

5.5.1

Median Value via Implicit Temporary Tables


Using an SQL user variable and nested SELECT can provide creative solutions to perceived limitations
of SQL. In the following SQL statements, a user variable called @counter is created for use in a
nested SELECT statement:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

122

Thi

Chapter 5:Complex Calculations

mysql> SET @counter := 0;


Query OK, 0 rows affected (0.00 sec)
mysql> SELECT FLOOR(AVG(table_a.Population)) AS median_pop
-> FROM (SELECT @counter := @counter+1 AS Count,
->
Population
->
FROM Continents ORDER BY Population) AS table_a,
->
(SELECT COUNT(*) AS totalrows FROM Continents) AS table_b
-> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1);
+------------+
| median_pop |
+------------+
| 482993000 |
+------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

123

no
a
s
a
h
n)the values
AVG(value) - The first portion AVG(value) will v
take
ethat are returned and

d
i
average them together. If there are an odd numberm
theu
values returned would be the
o of rows,
G
c
same values so averaging them would simply
provide
the
one
value
in response. If there are

t
n
21 would
e
an even number of rows, the valuests
returned
be
the
middle
two
values which would
p Stud
then be averaged together.
s
@
is
n
h
(SELECT @counter
:= @counter+1
AS Count, Population FROM
a
t
r
t BY sPopulation)
e

Continents n
ORDER
AS table_a - This portion of the script acts
o to u that is being used in the SQL statement. This table acts as the
as the firsto
table (table_a)
m
( table nwhich
counter
sewill produce a value from 1 to 7 based on the values in the Continents
n
a
r
e
table. ic
l
et T

uy
g
N

For the most part the SQL is pretty standard; however, the following explains portions of the SQL that
may seem a little awkward:

(SELECT COUNT(*) AS totalrows FROM Continents) AS table_b - This


portion of the script acts as the second table (table_b) that is being used in the SQL. The
only column in this table, called totalrows, will contain the number of rows from the
incomes table. This totalrows column will be used to determine the middle rows which is
necessary for determining the median value.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-10

MySQL Developer Techniques

Chapter 5:Complex Calculations

WHERE counter IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1) - The

totalrows column that is obtained from the second table (table_b), is used here to
determine the middle rows.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Odd number of rows - When there are an odd number of rows, the formula
FLOOR((totalrows+1)/2) will evaluate to (totalrows+1)/2. The second
formula FLOOR(totalrows/2)+1 will also evaluate to (totalrows+1)/2.
Thus providing two values that are identical to the AVG(value) aggregate
function. In a list that contains 1, 2, 3, 4, 5 the following results would exist:

FLOOR((totalrows+1)/2) => FLOOR((5+1)/2) => 3

FLOOR(totalrows/2)+1 => FLOOR(5/2)+1 => 3

Even number of rows - Where there are an even number of rows, the formula
FLOOR((totalrows+1)/2) will evaluate to totalrows/2. The second formula
FLOOR(totalrows/2)+1 will evaluate to (position/2+1). This results in the two
values returned being the values from the two middle rows in the list. In a list that
contains 1, 2, 3, 4, 5, 6 the following results would exist:

an
r
t
n

FLOOR((totalrows+1)/2) => FLOOR((6+1)/2) => 3

FLOOR(totalrows/2)+1 => FLOOR(6/2)+1 => 4

bl
a
r
fe

no
a
s
a
h
n) ide
FLOOR() function v
om Guof the numeric value that is supplied.
The SQL FLOOR() function returns the largest integer that1isc
less than ortequal
n
2value thatdisesupplied.
This function will return the same data type of the numeric
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
uye
g
hi N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-11

MySQL Developer Techniques

Chapter 5:Complex Calculations

5.5.1.1 Evaluating Performance

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

124

The SQL statement that was used to determine the median value for the Continents table produced the
correct output but at what cost to performance. Utilizing the EXPLAIN function can provide clues to
performance issues:
mysql> EXPLAIN
-> SELECT FLOOR(AVG(table_a.Population)) AS median_pop
-> FROM (SELECT @counter := @counter+1 AS Count, Population
->
FROM Continents ORDER BY Population)
->
AS table_a,
->
(SELECT COUNT(*) AS totalrows FROM Continents)
->
AS table_b
-> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1);
+----+---------+------------+----+----+-------+----+----+------+------------------------------+
| id | selec.. | table
| .. | .. | key
| .. | .. | rows | Extra
|
+----+---------+------------+----+----+-------+----+----+------+------------------------------+
| 1 | PRIMARY | <derived3> | .. | .. | NULL | .. | .. |
1 |
|
| 1 | PRIMARY | <derived2> | .. | .. | NULL | .. | .. |
7 | Using where
|
| 3 | DERIVED | Continents | .. | .. | Pop.. | .. | .. |
7 | Select tables optimized away |
| 2 | DERIVED | Continents | .. | .. | Pop.. | .. | .. |
7 | Using index
|
+----+---------+------------+----+----+-------+----+----+------+------------------------------+
4 rows in set (0.00 sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
The first nested SELECT statement (id #2) is using the Continents.Population
n) ide index, but the
v

second nested SELECT statement (id #3) is able to completely


uthe SELECT statement due to
om remove
G
c
the MyISAM storage engine having an internal row count.

t
n
21to attempt
e
s
d
t
Other storage engines (such as InnoDB) would
have
to utilize an index. With the SELECT
125
spbe ables toSbetuused when
statement as it currently is, no index@
would
requesting a count on all columns:
i
n
COUNT(*). To overcome thisalimitation
intnon-MyISAM
storage engines, an indexed column can be
h
tr sfeatures:
e

used instead of * to utilize


optimization
n
u
o
o
to
m
(
e
an icens
r
T
l
t
e
y
gu
N
hi
Note: Some of the content from this output has been removed for printing purposes to ensure that the
more important columns for this discussion are displayed.

mysql> EXPLAIN
-> SELECT FLOOR(AVG(table_a.Population)) AS median_pop
-> FROM (SELECT @counter := @counter+1 AS Count,
->
Population
->
FROM Continents ORDER BY Population) AS table_a,
->
(SELECT COUNT(Population) AS totalrows FROM Continents) AS table_b
-> WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1);
+----+---------+------------+----+----+------------+----+----+------+-------------+
| id | selec.. | table
| .. | .. | key
| .. | .. | rows | Extra
|
+----+---------+------------+----+----+------------+----+----+------+-------------+
| 1 | PRIMARY | <derived3> | .. | .. | NULL
| .. | .. |
1 |
|
| 1 | PRIMARY | <derived2> | .. | .. | NULL
| .. | .. |
7 | Using where |
| 3 | DERIVED | Continents | .. | .. | Population | .. | .. |
7 | Using index |
| 2 | DERIVED | Continents | .. | .. | Population | .. | .. |
7 | Using index |
+----+---------+------------+----+----+------------+----+----+------+-------------+
4 rows in set (#.## sec)

Using the Continents.Population index in the COUNT function has allowed the second table
(id #3) to now use the index and thus improves the performance of the SELECT statement.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-12

MySQL Developer Techniques

5.5.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

126

Thi

Chapter 5:Complex Calculations

Creating a Median Stored Procedure


The MySQL stored programming language is a block-structured programming language based on the
ANSI SQL:2003 SQL/PSM (Persistent Stored Module) specification. This language includes many of
the common programming tools expected in a block-structured programming language such as
manipulating variables, managing execution through conditions, repeated routine processing along
with handling errors. In the course of using MySQL, a developer may find it easier to produce a stored
procedure to eliminate external coding or repetition of tasks. Thus, if there is a need to obtain the
median value on a regular basis against datasets, it may make sense to create a stored procedure to
handle the task.
A median procedure
Using the process shown for obtaining a median value using implicit temporary tables can be moved
easily into a stored procedure that can be reused over and over again. The following is the median
stored procedure in its entirety which will be followed up with a detailed look at how it works:

mysql> DELIMITER //

an
r
t
n

mysql> CREATE PROCEDURE median (src_field VARCHAR (32), src_table VARCHAR (32))
-> SQL SECURITY INVOKER
-> BEGIN
-> SET @counter = 0;
-> SET @stmt = CONCAT(
->
'SELECT FLOOR(AVG(table_a.', src_field,')) AS MEDIAN
'>
FROM (SELECT @counter := @counter+1 Count, ',
->
src_field,
->
' FROM ',
->
src_table,
->
' ORDER BY ',
->
src_field,
->
') AS table_a,
127
'>
(SELECT COUNT(*) totalrows FROM ',src_table, ') AS table_b
'>
WHERE Count IN (FLOOR((totalrows+1)/2), FLOOR(totalrows/2)+1)
'>
');
-> PREPARE stmt_exec FROM @stmt;
-> EXECUTE stmt_exec;
-> DEALLOCATE PREPARE stmt_exec;
-> END //
Query OK, 0 rows affected (#.## sec)

bl
a
r
fe

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

mysql> DELIMITER ;
mysql> CALL median('Population','Continents');
+-----------+
| MEDIAN
|
+-----------+
| 482993000 |
+-----------+
1 row in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-13

MySQL Developer Techniques


DELIMITER

128

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 5:Complex Calculations

Within the BEGIN END syntax, statements must be terminated with a semicolon (;). This is due to
the fact that this is the same default terminating character for SQL statements, it is important to change
the SQL terminating character with the DELIMITER statement when using the MySQL command line
client or batch processing.
SQL SECURITY
Stored routines bring a feature to database execution that is many times overlooked in discussions; this
feature is the ability to give an end user a different privilege to execute a defined statement than they
have in any other circumstance. This feature is implemented by use of the SQL SECURITY optional
clause in the development of the stored routine:

DEFINER This value causes the routine to have the privileges of the user who created it
(or another user set by the developer). This is the default if none is selected.

INVOKER This value causes the routine to run with the privilege of the user who invoked
it or is running the stored routine. This means the routine has access to database objects only
if the invoker (user) already has access to them.

bl
a
r
fe

an
r
t
n

In the case of the median stored procedure, the SQL SECURITY needs to be set to INVOKER to allow
the stored procedure to create the temporary table that is used and then to access that temporary table.
If the SQL SECURITY setting is DEFINER (which is the default), then the temporary table would be
created but then would not be able to be accessed by the remaining portions of the stored procedure.

no
a
s
a
h
BEGIN END

n) idTheecompound
vtriggers.
The BEGIN END syntax can be used in stored routinesm
and
statements
u (there is no limit)
G
within the BEGIN END syntax can contain onec
oromore statements
or no
t
1
n
statements (an empty compound statement is legal).
2
ts tude
p
s
S
SET
@
s
i
n
h
t a value to defined variables using either = or := as the
The SET statement allowsthe
trauser tosassign
e
n
assignment operator.
o to u
o
m
n ( ense
Dynamic
SQL
a
r
T one of ltheicmost exciting features of MySQL stored routines is the use of server-side prepared
t
Perhaps
e
uy statements, which allows for an independent (from the API) method to create efficient and secure SQL

N
Thi

for repeated execution. This in essence is a dynamic method to create SQL, thus the term Dynamic
SQL. SQL syntax for prepared statements is based on three SQL statements:
PREPARE
o An SQL statement is precompiled with PREPARE - The PREPARE statement prepares
a statement and assigns it a name, stmt_name, by which to refer to the statement later.
Statement names are not case sensitive. preparable_stmt is either a string literal or
a user variable that contains the text of the statement. The text must represent a single
SQL statement, not multiple statements.

EXECUTE
o Prepared statements are executed with the EXECUTE statement - After preparing a
statement, it can only be executed with an EXECUTE statement that refers to the prepared
statement name.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-14

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 5:Complex Calculations

DEALLOCATE PREPARE
o To deallocate a prepared statement, use the DEALLOCATE PREPARE statement An alternative to using DEALLOCATE PREPARE is DROP PREPARE. Attempting to
execute a prepared statement after deallocating it results in an error. If you terminate a
client session without deallocating a previously prepared statement, the server deallocates
it automatically.
{DEALLOCATE | DROP} PREPARE stmt_name

It is a good practice to deallocate resources in general if no longer needed.

Stored functions are not able to utilize prepared statements, this is the reason that a stored
procedure had to be used to obtain the median value.

CALL
The CALL statement invokes a procedure that was defined previously with CREATE PROCEDURE. In
the case of the median stored procedure, CALL median ('Population','Continents'),
tells the median stored procedure to execute using the Continents tables Population column.

bl
a
r
fe

ns
a
r
t
- is written, the
User defined functions (UDF) contain code that becomes part of the running server, so when n
a UDF
o
designer is bound by any and all constraints that otherwise apply to writing server code. UDF's
written in C or
atheynwillmustbebeinstalled
C++ (or another language that can use C calling conventions), and the operating system
on must
s
a
h
support dynamic loading. With that said, there are numerous repositories of UDF's
that
designers
all
over
the
world
have

n)haveisolved
created and made available to the MySQL community. A number of designers
the limitation of not having a
e
v

d the requirements for a specific


m doGnotu meet
median aggregate function with their own UDF's. If the solutionso
presented
c
database house, UDF's are another possible solution.
1 ent
2
s
pt Stud
s
@ thiThe
s MySQL Training division offers a virtual training course
n
a
r
targeted to developers who would like to learn how
t use specifically
n
o
to
write
MySQL
user defined functions. This 8-hour eLearning
o
o
t
m
course
(broken
up
over 2 days), will provide MySQL users
( nse
n
with the skills necessary to write, manage and optimize
a
r
ce
T
MySQL user defined functions. Additional information about
i
l
t
e
this and any other training offering from the MySQL Training
y
u
g
division can be found at www.mysql.com/training.
N
MEDIAN UDF

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-15

MySQL Developer Techniques

5.6

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

129

Chapter 5:Complex Calculations

Simulating RANK
When performing an analysis against a dataset, there are times when determining where a row of data
fits in against the entire dataset. This is called ranking the data. However, There is no standard SQL
solution that provides ranking. Other database vendors have created non-standard SQL functions to
handle this limitation but as of yet, MySQL has not provided a function to solve this. With that said,
using standard SQL there are a few ways to resolve this limitation and simulate the analytical ranking
function in MySQL.
Ranking without gaps
The first way to simulate the ranking of rows is by creating a subquery that will be used to add an
additional column to the dataset. This additional column would be responsible for determining the
ranking by allowing more than one row to contain the same ranking without creating a gap in the
ranking. For example, consider the following table:
+--------+----------+-------+
| userid | username | score |
+--------+----------+-------+
|
3 | Sarah
|
97 |
|
1 | Max
|
95 |
|
2 | Kai
|
93 |
|
5 | Pat
|
93 |
|
4 | Jeff
|
72 |
+--------+----------+-------+
In the case of this dataset, there are two users who have received the same score. Their rank (Kai and
Pat) is a tie for 3rd, but does Jeff obtain 4th or 5th place in the ranking. Using the following standard
SQL, Jeff would be assigned the ranking of 4th place (no gaps in the ranking):

bl
a
r
fe

an
s
a
h
)
n ide
v

om t Gu
c

21 den
130
s
t
sp s Stu
@
i AS r_score, (
n score,thscore
a
mysql> SELECT userid, username,
r
t
e

->
SELECT COUNT(DISTINCT(score))+1
FROM scores
us
o>nr_score
o
->
WHERE score
o
t
(m
-> ) AS rank
FROM scores
se ORDER BY rank;
n
n
a
+--------+----------+-------+---------+------+
r
e
T| username
ic | score | r_score | rank |
| userid
l
t
e
y 3 | Sarah | 97 |
u+--------+----------+-------+---------+------+
|
97 |
1 |
g
N
|
1 | Max
|
95 |
95 |
2 |
i
h

an
r
t
n

|
2 | Kai
|
93 |
93 |
3 |
|
5 | Pat
|
93 |
93 |
3 |
|
4 | Jeff
|
72 |
72 |
4 |
+--------+----------+-------+---------+------+
5 rows in set (#.## sec)

In this case, two additional columns are added to the table (r_score and rank). The r_score
column is added to assist with the subquery that will determine the rank column by performing a
"pseudo" self-join against the score column. The resulting ranking will eliminate any gaps in the
ranks and provide identical ranking when ties (or more than one) occur in the column being ranked.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

131

Thi

Chapter 5:Complex Calculations

Ranking with gaps


The second way to simulate the ranking of rows is similar to the first. There is still an additional
column that is added to the dataset that will display the ranks, but rather than creating a subquery this
solution utilizes WHERE clauses along with the ORDER BY clauses to manage the ranking. The big
difference between the two solutions is that this solution will create a natural gap in the ranking and
will assign any ties to a lower ranking in the scope of ranks. For example, consider the same table
presented with the first solution:
+--------+----------+-------+
| userid | username | score |
+--------+----------+-------+
|
3 | Sarah
|
97 |
|
1 | Max
|
95 |
|
2 | Kai
|
93 |
|
5 | Pat
|
93 |
|
4 | Jeff
|
72 |
+--------+----------+-------+
In the case of this dataset, there is still the issue of two users who have received the same score. Jeff
still would question if he obtains 4th or 5th place in the ranking. However, more importantly to the two
individuals who tied (Kai and Pat), will they receive 3rd or 4th place. Using the following standard
SQL, Jeff would be assigned the ranking of 5th place (gaps in the ranking):

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

mysql> SELECT t1.username, t1.score, COUNT(t2.score)+1


AS rank
u< t2.score
oONmt1.score
G
-> FROM scores AS t1 LEFT JOIN scores AS t2
c
t
n DESC;
->
GROUP BY t1.username, t1.score ORDER
21 BY dt1.score
e
s
t
+----------+-------+------+
sp s Stu
| username | score | rank |
@
+----------+-------+------+ n
thi
| Sarah
|
97 |
1 |tra
e
| Max
|
95 | on
2 |
us
o
o
| Kai
|
93
|
3
|
t
m s3 e|
| Pat
| n (93 |
n5 |
a
| Jeff
|
72 |
r
e
c
T
i
l
+----------+-------+------+
et
5 y
rows in set (#.## sec)
u
g

The resulting ranking will include gaps in the ranks and provide identical ranking when ties (or more
than one) occur in the column being ranked.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-17

MySQL Developer Techniques

5.7

Divide and Conquer


Over the years, techniques have evolved that describe the process of breaking down a problem into
manageable chunks. One such technique has come to be known as "Divide and Conquer". There is a
formal mathematical model behind this technique, but basically it involves breaking down a complex
problem into smaller, related steps, solving those smaller problems and then coalescing the results into
the ultimate solution of the complex problem.
The scenario
Consider this scenario: In an effort to increase sales volume and customer loyalty, an online retailer of
consumable products decided to offer a monthly purchase volume rebate to its customers who choose to
register for this program. The rebate program would be a stepped program that works as follows:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

132

Thi

Chapter 5:Complex Calculations

133

Total volume of purchases calculated


o Purchases accumulated over the period of a month are entitled to a variable, stepped
rebate based upon the total "commvalue" volume purchased. This value is assigned to
each product and the percentage of the total price of the product represented by this value
varies from one product to another. This is done to make the rebate cost more "uniform"
from the seller's point of view.
What is "commvalue"?

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

Rebate accumulated incrementally


u
G
coismaccumulated

t
o The rebate is based on this volume1and
incrementally. If a certain
n
2 the customer
e
threshold level of purchases is tattained,
is
granted
of a specified
s
pto that S
tud amount. If a highera rebate
percentage on all purchasessup
threshold
threshold amount of
@ thatthmonth,
purchases is attainednwithin
is the volume above the lower threshold returns a
a
r
higher percentage
- otherwise
the additional purchases above the original threshold level
tpercentage
seof that
n
return the
base
level. If the lowest threshold level is not attained with
u
o
o
o
t
that
months
total
purchases,
no
rebate
is given. A sample rebate rate chart would be as
m
(
e
n
follows: ns
a
r
e
cCummulative
T
i
l
sales Rebate Achieved
t
e

Commvalue is short for commercial value. Calculating commvalue is done by removing the value of the product after
the cost of packaging and other overhead, etc. along with excluding the amount the customer pays for shipping and
handling. So products with simpler packaging have a higher percentage commvalue relative to the purchase price.

uy
g
N

250.00

5%

500.00

7%

1000.00

10%

1500.00

12%

2000.00

15%

So any additional purchases beyond the $2,000.00 level (which this retailer considers
unlikely, but desirable) continue to earn at the highest rebate rate. A customer buying lots
of products during a month gets an increasingly good deal, but never quite attains a rebate
of 15% of the cv - and that is a substantially lower percentage of what was actually paid
by the customer. Still, it is a generous offer for customer loyalty in a competitive market
and can be an incentive to purchase more at the end of the month as well if a customer is
tracking their cumulative purchases.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-18

MySQL Developer Techniques


134

Chapter 5:Complex Calculations

Calculating the rebate


Here are some examples of how a rebate might be calculated:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

800.00 cumulative cv

Thi

= 250.00 * 5% + 250.00 * 7% + 300 * 7%


= 12.50 + 17.50 + 21.00
= 51.00

1200.00 cumulative cv

bl
a
r
fe

= 250.00 * 5% + 250.00 * 7% + 500.00 * 10% + 200.00 * 10%


= 12.50 + 17.50 + 50.00 + 20.00
= 100.00

an
r
t
n

o
n
a
Notice that to even calculate these rebates, the purchase volume must be broken down into the amount
s
that falls into each percentage range to apply that percentage. This h
isafollowed by summing those
to perform the same
intermediate values to obtain the total rebate amount. Any query created
n) will
ehave
v

d
i
feat.
om t Gu
c
Step 1: Rebate percentage schedule table

n
135
21schedule.
e
s
Let's begin by characterizing the rebate percentage
The simplest table that could hold these
d
t
p andSpopulated
tu as follows:
sdefined
threshold values and rebate rates might be
@
s
i
n
h
a
t
r
t
mysql> CREATE TABLE commrate
(
e
s NULL,
n(8, 2)uNOT
-> threshold DECIMAL
o
o
o
t
-> rate DECIMAL
(3,2)
NOT
NULL);
(m nse
n
a
r
e could then be populated with the following values (250.00, .05), (500.00, .07),
ctable
T
i
l
The
commrate
t
uye(1000.00, .10), (1500.00, .12), (2000.00, .15):
g
N
mysql> INSERT INTO commrate VALUES (250.00, .05), (500.00, .07),
-> (1000.00, .10), (1500.00, .12), (2000.00, .15);

This would yield the following table:


mysql> SELECT * FROM commrate;
+-------------+------+
| threshold
| rate |
+-------------+------+
|
250.00 | 0.05 |
|
500.00 | 0.07 |
|
1000.00 | 0.10 |
|
1500.00 | 0.12 |
|
2000.00 | 0.15 |
+-------------+------+

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-19

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

136

Chapter 5:Complex Calculations

But while this is descriptive, it does not offer enough tools for the task at hand. It would be more helpful
if the endpoints were defined (or "bracketed") for each range, as well as the two rebate percentages that
could potentially apply to each range. So a more thorough table definition for the rebate levels and rates
that affords more flexibility might be:
mysql>
->
->
->
->

CREATE TABLE commrate (


base DECIMAL(8,2) NOT NULL,
threshold DECIMAL (8,2) NOT NULL,
baserate DECIMAL (3,2) NOT NULL,
rate DECIMAL (3,2) NOT NULL);

The *new* commrate table could then be populated with the following values (0, 250, 0, .05), (250,
500, .05, .07), (500, 1000.00, .07, .1), (1000, 1500, .1, .12), (1500, 2000, .12, .15):
mysql> INSERT INTO commrate VALUES (0, 250, 0, .05), (250, 500, .05, .07),
-> (500, 1000.00, .07, .1), (1000, 1500, .1, .12),
-> (1500, 2000, .12, .15);

bl
a
r
fe

ns
a
r
t
- month,
But because someone might occasionally exceed $2,000.00 in cumulative purchases in n
a given
o
there is also a need to have a row that defines that range. Safely assuming no one will
buy
$1,000,000.00
afor nthe data type of the
cv in a single month, that upper bound could bet set as the maximum value
s
a
threshold column:
h
)
n ide
v

mysql> INSERT INTO commrate VALUES (2000, 999999.99,.15,.15);


om t Gu
c

21 den
s
t
So the commrate table would now lookslike:
p Stu
@
is
n
h
a
t
r
mysql> SELECT * FROM commrate;
t use
n
+---------+-------------+----------+-----------+
o
o | baserate
| base
| threshold
to | bonusrate |
m
(
e
+---------+-------------+----------+-----------+
n 250.00
ns| 0.00 | 0.05 |
|
0.00 r|a
e
c
T
i500.00 | 0.05 | 0.07 |
| 250.00
t || l1000.00
e
y
|
500.00
|
0.07 |
0.10 |
u

g| 1000.00 |

N
Thi

1500.00 |
0.10 |
0.12 |
| 1500.00 |
2000.00 |
0.12 |
0.15 |
| 2000.00 |
999999.99 |
0.15 |
0.15 |
+---------+-------------+----------+-----------+

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-20

MySQL Developer Techniques

Step 2: Customer purchases table


Now lets define the table that will hold the purchases made by customers. It is assumed that a customer
could make more than one purchase in a given month, so there must be a column that holds the date of
each purchase. Then there is a need to accumulate the total purchase amounts for a given month. For
reference (and for other interesting statistical queries), there would need to be the full price of each
purchase as well as the commvalue associated with it - and, of course, a pointer to the customer. The
resulting table structure might be defined like this:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

137

Thi

Chapter 5:Complex Calculations

mysql>
->
->
->
->

CREATE TABLE purchases (


cuid CHAR (8) NOT NULL,
saledate DATE NOT NULL,
totalpurchase DECIMAL (8,2) NOT NULL,
commvalue DECIMAL (8,2) NOT NULL);

There could be other columns in this table as well, such as an invoice number, but those other columns
would not be involved in this calculation, so they will be left out for the sake of simplicity.
The table could then be populated with some test data:

bl
a
r
fe

mysql>
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->

INSERT INTO purchases VALUES


('DFH34789', '2008-04-02', 82.27, 74.16),
('ADK89230', '2008-04-03', 36.82, 32.28),
('DFH34789', '2008-04-06', 187.36, 168.59),
('ADK89230', '2008-04-08', 95.48, 88.71),
('DFH34789', '2008-04-10', 83.90, 74.83),
('DFH34789', '2008-04-14', 61.62, 55.07),
('DFH34789', '2008-04-16', 57.75, 52.15),
('ADK89230', '2008-04-17', 114.65, 103.92),
('DFH34789', '2008-04-21', 85.36, 77.16),
('ADK89230', '2008-04-22', 41.25, 38.05),
('DFH34789', '2008-04-25', 118.83, 102.14),
('ADK89230', '2008-04-26', 98.20, 89.98),
('DFH34789', '2008-04-30', 132.66, 116.45),
('ADK89230', '2008-04-30', 62.69, 56.30),
('DFH34789', '2008-05-03', 97.78, 88.42),
('DFH34789', '2008-05-05', 236.41, 210.59),
('ADK89230', '2008-05-06', 45.44, 41.31),
('DFH34789', '2008-05-07', 37.92, 33.80),
('ADK89230', '2008-05-10', 82.35, 74.78),
('DFH34789', '2008-05-11', 61.72, 55.97),
('DFH34789', '2008-05-15', 77.15, 69.12),
('ADK89230', '2008-05-16', 51.64, 46.25),
('DFH34789', '2008-05-20', 375.86, 335.27),
('DFH34789', '2008-05-24', 68.16, 62.87),
('ADK89230', '2008-05-28', 120.18, 109.00),
('DFH34789', '2008-05-28', 92.30, 82.97),
('DFH34789', '2008-06-01', 112.57, 106.12),
('ADK89230', '2008-06-04', 68.79, 62.18),
('DFH34789', '2008-06-05', 217.62, 208.89),
('DFH34789', '2008-06-08', 86.32, 82.80),
('ADK89230', '2008-06-09', 85.22, 78.31),
('DFH34789', '2008-06-12', 64.72, 58.37),
('DFH34789', '2008-06-16', 47.18, 43.00),
('ADK89230', '2008-06-16', 122.37, 114.98),
('ADK89230', '2008-06-21', 91.38, 86.25),
('DFH34789', '2008-06-22', 225.86, 212.19),
('DFH34789', '2008-06-26', 98.15, 92.84),
('ADK89230', '2008-06-27', 148.00, 139.88),
('DFH34789', '2008-06-30', 102.40, 95.27);

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

an
r
t
n

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-21

MySQL Developer Techniques


138

Chapter 5:Complex Calculations

Step 3: Calculate the rebate

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Now it is time to build a query that calculates the rebate for each customer for a given month. The first
task is to build a query to accumulate the purchases for each customer for a given month. This is easy! A
simple query that does this is:

Thi

mysql> SELECT cuid AS Customer, SUM(totalpurchase) AS MonthlyPurchases,


-> SUM(commvalue) AS MonthlyCV FROM purchases
-> WHERE saledate BETWEEN '2008-05-01' AND '2008-05-31' GROUP BY cuid;
+----------+------------------+-----------+
| Customer | MonthlyPurchases | MonthlyCV |
+----------+------------------+-----------+
| ADK89230 |
299.61 |
271.34 |
| DFH34789 |
1047.30 |
939.01 |
+----------+------------------+-----------+

bl
a
r
fe

The column aliases are used here only to make the result set read nicely. Different "internal" aliases may
be used as the statement is morphed to perform the rest of the task. The internal aliases may be needed to
reference derived columns internally, but the ones shown here will be used again in the final result set.
So all that is needed now is to add a column for "Rebate" and the task is finished. But that isn't as easy as
it sounds! A number of intermediate tests will need to be performed before the final solution will be
realized.
The total purchase is not used in the rebate calculation, so that can be eliminated from the exploratory
queries in this exercise and then can be added back in for the final result. To prepare to calculate the
amount of each customer's cumulative purchases that falls into each rebate range, a cross join needs to be
built between the result of the simple cumulative query and the commrate table, as follows:

an
r
t
n

an
s
a
h
)
139
n ide
v

om t Gu
c

1
n
2
e
s
d
t
mysql> SELECT * FROM (
spAS cvs Stu
->
SELECT cuid, SUM(commvalue)
@
nBY cuid thi
->
FROM purchases GROUP
a
r
t
-> ) AS sls CROSS JOIN
commrate

seORDER BY cuid, base;


n
u
o
+----------+---------+---------+-----------+----------+------+
o
| cuid
| cv(m | base to | threshold | baserate | rate |
se
+----------+---------+---------+-----------+----------+------+
n
n
a
r
e
| ADK89230
0.00 |
250.00 |
0.00 | 0.05 |
T || 1162.18
lic || 250.00
t
| ADK89230
1162.18
|
500.00 |
0.05 | 0.07 |
e
| y
ADK89230 | 1162.18 | 500.00 |
1000.00 |
0.07 | 0.10 |
u
Ng| ADK89230 | 1162.18 | 1000.00 | 1500.00 | 0.10 | 0.12 |
| ADK89230 | 1162.18 | 1500.00 |
2000.00 |
0.12 | 0.15 |
| ADK89230 | 1162.18 | 2000.00 | 999999.99 |
0.15 | 0.15 |
| DFH34789 | 2559.04 |
0.00 |
250.00 |
0.00 | 0.05 |
| DFH34789 | 2559.04 | 250.00 |
500.00 |
0.05 | 0.07 |
| DFH34789 | 2559.04 | 500.00 |
1000.00 |
0.07 | 0.10 |
| DFH34789 | 2559.04 | 1000.00 |
1500.00 |
0.10 | 0.12 |
| DFH34789 | 2559.04 | 1500.00 |
2000.00 |
0.12 | 0.15 |
| DFH34789 | 2559.04 | 2000.00 | 999999.99 |
0.15 | 0.15 |
+----------+---------+---------+-----------+----------+------+

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-22

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

140

Chapter 5:Complex Calculations

Then some additional columns can be injected to this result using a bit of logic to calculate the portion of
the accumulated total for each customer that applies to each range as well as the rebate amount for that
portion. Remember, the lower rebate percentage (baserate) applies to the portion if the next threshold
level is not reached, but the higher percentage applies if that threshold is reached or exceeded:
mysql> SELECT sls.*, crt.*,
-> IF(cv <= base, 0, IF(cv < threshold, cv - base, threshold - base)) AS cvpart,
-> ROUND(IF(cv <= base, 0, IF(cv < threshold, (cv - base) * rate,
-> (threshold - base) * rate)), 2) AS rpart
-> FROM (
->
SELECT cuid, SUM(commvalue) AS cv FROM purchases pch GROUP BY cuid
-> ) AS sls CROSS JOIN commrate AS crt ORDER BY cuid, base;
+----------+---------+---------+-----------+----------+------+--------+-------+
| cuid
| cv
| base
| threshold | baserate | rate | cvpart | rpart |
+----------+---------+---------+-----------+----------+------+--------+-------+
| ADK89230 | 1162.18 |
0.00 |
250.00 |
0.00 | 0.05 | 250.00 | 12.50 |
| ADK89230 | 1162.18 | 250.00 |
500.00 |
0.05 | 0.07 | 250.00 | 17.50 |
| ADK89230 | 1162.18 | 500.00 |
1000.00 |
0.07 | 0.10 | 500.00 | 50.00 |
| ADK89230 | 1162.18 | 1000.00 |
1500.00 |
0.10 | 0.12 | 162.18 | 16.22 |
| ADK89230 | 1162.18 | 1500.00 |
2000.00 |
0.12 | 0.15 |
0.00 | 0.00 |
| ADK89230 | 1162.18 | 2000.00 | 999999.99 |
0.15 | 0.15 |
0.00 | 0.00 |
| DFH34789 | 2559.04 |
0.00 |
250.00 |
0.00 | 0.05 | 250.00 | 12.50 |
| DFH34789 | 2559.04 | 250.00 |
500.00 |
0.05 | 0.07 | 250.00 | 17.50 |
| DFH34789 | 2559.04 | 500.00 |
1000.00 |
0.07 | 0.10 | 500.00 | 50.00 |
| DFH34789 | 2559.04 | 1000.00 |
1500.00 |
0.10 | 0.12 | 500.00 | 60.00 |
| DFH34789 | 2559.04 | 1500.00 |
2000.00 |
0.12 | 0.15 | 500.00 | 75.00 |
| DFH34789 | 2559.04 | 2000.00 | 999999.99 |
0.15 | 0.15 | 559.04 | 83.86 |
+----------+---------+---------+-----------+----------+------+--------+-------+

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

n
21base, dthreshold,
e
s
t
With the calculations being performed properly,
the
baserate, bonusrate and
u
pthe queryStotsimplify
s
cvpart
columns
can
be
removed
from
it
in
preparation
for building the final
141
@
s
i
n
h
query that will yield only the r
a eID (cuid)
t number, cumulative cv and rebate (rpart):
tcustomer's

s
n
u<= base, 0, IF(cv < threshold,
oROUND(IF(cv
o
o
mysql> SELECT sls.*,
t
m
-> (cv - base)
* baserate,
se (threshold - base) * rate)), 2) AS rpart FROM (
n (cuid,
n
a
->
SELECT
SUM(commvalue) AS cv FROM purchases AS pch GROUP BY cuid
r
e
ic JOIN commrate crt ORDER BY cuid, base;
->t )TAS sls lCROSS
e
y
u+----------+---------+-------+
| cuid
| cv
| rpart |
g
N
+----------+---------+-------+
i
h
| ADK89230 | 1162.18 | 12.50 |
| ADK89230 | 1162.18 | 17.50 |
| ADK89230 | 1162.18 | 50.00 |
| ADK89230 | 1162.18 | 16.22 |
| ADK89230 | 1162.18 | 0.00 |
| ADK89230 | 1162.18 | 0.00 |
| DFH34789 | 2559.04 | 12.50 |
| DFH34789 | 2559.04 | 17.50 |
| DFH34789 | 2559.04 | 50.00 |
| DFH34789 | 2559.04 | 60.00 |
| DFH34789 | 2559.04 | 75.00 |
| DFH34789 | 2559.04 | 83.86 |
+----------+---------+-------+

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-23

MySQL Developer Techniques


Step 4: The Final Query

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

142

Thi

Chapter 5:Complex Calculations

Now the only task left is to sum the results from the various portions for each customer to yield the total
rebate. The totpurch column will be added for the final display. The only problem is that an error will
be produced if there is only a sum() function added around the rpart calculation and a "GROUP BY
cuid" at the end. The reason for this is that the cv and totpurch columns are not involved in the
grouping if only cuid is grouped. There are two choices available:

Place a MAX() or MIN() or AVG() - although AVG() would then need to be rounded or
truncated back to 2 decimal places to function around the cv and totpurch columns in the
column list (which will also require additional aliasing for neatness).

Group by both cv and totpurch in addition to cuid.


The later will be chosen to complete the final query:

143

mysql> SELECT cuid AS Customer, totpurch AS MonthlyPurchases, cv AS


-> ROUND(SUM(IF(cv <= base, 0, IF(cv < threshold, (cv - base) *
-> (threshold - base) * rate))), 2) AS Rebate
-> FROM (
->
SELECT cuid, SUM(totalpurchase) AS totpurch, SUM(commvalue)
->
FROM purchases AS pch GROUP BY cuid
-> ) AS sls CROSS JOIN commrate AS crt GROUP BY cuid, totpurch,
+----------+------------------+-----------+--------+
| Customer | MonthlyPurchases | MonthlyCV | Rebate |
+----------+------------------+-----------+--------+
| ADK89230 |
1264.46 |
1162.18 | 96.22 |
| DFH34789 |
2811.87 |
2559.04 | 298.86 |
+----------+------------------+-----------+--------+

MonthlyCV,
baserate,

an
r
t
n

AS cv

bl
a
r
fe

ncv;o
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
p Stu table in this final statement rather than using
Notice that each column was listed fromsthe sls derived
@
s to each column. In addition, the query did not
n
sls.*. This is so that displaya
aliases
couldth
beiassigned
r
tcolumnsufrom
qualify the names of derived
se derived tables to keep their names distinct.
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-24

MySQL Developer Techniques

Chapter 5:Complex Calculations

Further Practice Lab 5-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In a certain country employees pay 18.5% social charges, including pension, unemployment etc funds.
On the remaining part they pay tax on revenue according to the following table:

Thi

Remaining Income

Tax Percentage

< 5,687

0.0%

5,687 - 11,344

5.5%

11,344 - 25,195

14.0%

25,195 - 67,546

30.0%

> 67,546

40.0%

Using the divide and conquer technique calculate the total tax, the tax on revenue % and the total % of
the salaries that the employees get to keep when we have the following list of employees:
Name

Salary

Johan

30,000

55,000

an
r
t
n

n
a
s
Tobias
90,000
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
Max

bl
a
r
fe

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-25

MySQL Developer Techniques

5.8

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

144

Thi

Chapter 5:Complex Calculations

Miscellaneous
There are multiple miscellaneous SQL tricks that can come in handy when working with MySQL. The
following list is just a few of them:
Avoiding exact counts
It is a common process for developers to use the SELECT COUNT(*) FROM tablename syntax to
produce an exact count of rows. However, for most storage engines (especially InnoDB tables) this is
costly in terms of performance. In many cases, the application does not require an exact count of rows,
rather an approximate number would suffice. A less costly approach would be to utilize the
INFORMATION_SCHEMA database with the following SQL:

mysql> SELECT table_rows FROM INFORMATION_SCHEMA.TABLES


-> WHERE TABLE_NAME = 'table_name' AND TABLE_SCHEMA = 'table_schema';

145

Random rows
Another common process that developers use is having the query statements choose random rows to return.
A common approach, and one of the worst for performance, is to utilize the RAND() function within the
query itself. The following is just one of many "poor" examples that are used by developers:

bl
a
r
fe

an
r
t
n

no
a
s with better performance.
There are other ways to get a "random" row that produce the same outcomeabut
For example, if the table has a numeric id column it is more efficient)toh
calculate a random number and
n ide
v
then look up the row by the random value id.

m Gu
o
c

t real number of rows the


mysql> SET @n = RAND() * 1000; -- substitute
by n
the
1 1000
2
e
->
table
likely
has
s
pt Stud
s
@ tWHERE
mysql> SELECT columns FROM table_name
is id = @n;
n
h
a
r
e which allows for overall better performance of indexes and
t usitself
n
This removes the function
from the query
o
o
tofeatures.
other MySQL server
optimization
m
(
e
an icens
r
T
l
t
e
y
gu
mysql> SELECT columns FROM table_name ORDER BY RAND() LIMIT 1;

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-26

MySQL Developer Techniques

5.9

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

146

Thi

Chapter 5:Complex Calculations

Simulate an aggregate multiplication function

Use variables to create a running total query

Utilize COALESCE to avoid divide by zero errors

Calculate the median value

Simulate the SQL RANK function in MySQL

Utilize SQL to solve complex problems

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

5-27

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 6

bl
a
r
fe

an
r
t
n

IMPROVING as a no
h
)
n
PERFORMANCE
v uiOF
de
m
o
c nt G
1
2
JOINS
s
de
spt Stu

uy
g
N

@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

6.1

Thi

Chapter 6: Improving Performance of Joins

IMPROVING PERFORMANCE OF JOINS


Overview
This chapter provides the knowledge and skills necessary to improve the response time of queries that
require the retrieval of data from more than one table. At the completion of this chapter, you will be able
to:

148

Describe how queries are optimized by the MySQL server

Describe the different types of MySQL JOIN's

Describe how MySQL performs JOIN's

Explain how to improve the performance of JOIN's

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-1

MySQL Developer Techniques

6.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

149

Thi

Chapter 6: Improving Performance of Joins

Query Optimizer
The task of the query optimizer in MySQL is to find the best or optimal plan for executing the SQL
query that is submitted to the server. The query optimizer searches for the best plan among all the
possible query evaluation plans. A process called greedy optimization enables the optimization engine to
significantly reduce the amount of time it spends calculating optimal execution paths by a process of
intelligent best-path reductions. For larger more complex queries, this can became a bottleneck in the
servers performance. The following example demonstrates the query process:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
Query Optimizer (Techniques
m se
n
n uses a list of known best practices to reduce not only the size of the query, but
a
As stated, the
query optimizer
r
e
c
T
i
l
more
the time that the query takes to execute. The following is a small example of some of the
etimportantly
y
many
best practices that are implemented:
u
g

150

Constant propagation - This best practice states that if the query is using constants (=, <, >, <=,
>=, <>, <=>, LIKE) and there are transitive conjunction (x=y=z), then x must equal z (x=z).
The optimizer will removes those inner equalities, thereby reducing the number of evaluations.
For example:

mysql> SELECT Name FROM City WHERE Name = District AND District = 'San Jose';

... would be optimized to read ...


mysql> SELECT Name FROM City WHERE Name = 'San Jose' AND District = 'San Jose';

This constant propagation best practice happens in a loop, so the output from one propagation
step can be input for the next step.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-2

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Dead code elimination - This best practice states that conditions that are always true or always
false can be removed from the query execution. For example:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> SELECT Name FROM Country WHERE Name=Name AND Code='USA';

Thi

... would be optimized to read ...


mysql> SELECT Name FROM Country WHERE Code='USA';

151

Range queries - In queries that have a clear list of disjunctions, the optimizer will transform the
query with the use of the IN clause. For example:
mysql> SELECT Name FROM Country WHERE Code IN('USA','CAN','DEU');

... would be optimized to read ...


mysql> SELECT Name FROM Country WHERE Code='CAN' OR Code='DEU' OR Code='USA';

bl
a
r
fe

ns
a
r
t
Notice that the order of the where clause was also rearranged to have them sortednto- assist the
optimizer.
no
a
s more ranges meeting the
Range optimizer - In queries that have multiple ranges with oneaor
h
same criteria, the range that meets all the requirements will) be kept while
n ide the others will be
v
dropped. For example:

om t Gu
c

mysql> SELECT Name FROM Country WHERE Population


n AND Population > 2000;
21 d> e10000
s
t
sp s Stu
... would be optimized to read
...
@
i
n
h
a
t
r
t
e
mysql> SELECT Name FROMnCountry
s Population > 10000;
o to uWHERE
o
(m nse
n
a
r
ce
T
i
l
t
e
guy

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-3

MySQL Developer Techniques

6.2.1

Greedy Optimizer
The MySQL server contains a greedy optimizer which can greatly reduce the time required to arrive at
a query execution plan for JOIN tables only. The greedy optimizer is most beneficial in cases where
several tables are joined with no useful join key. In earlier versions of MySQL, the complexity of the
search for a query execution plan was calculated as N!, where N is the number of tables to be joined.
Below is an example of all the possible query execution plans for 4 tables without any useful join keys
between them:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

152

Thi

Chapter 6: Improving Performance of Joins

Plan 1

Table1

Table2

Table3

Table4

Plan 2

Table1

Table2

Table4

Table3

Plan 3

Table1

Table3

Table2

Table4

Plan 4

Table1

Table3

Table4

Table2

Plan 5

Table1

Table4

Table2

Table3

Plan 6

Table1

Table4

Table3

Table2

Plan 7

Table2

Table1

Table3

Table4

Plan 8

Table2

Table1

Table4

Table3

Plan 9

Table2

Table3

Table1

Table4

Plan 10

Table2

Table3

Table4

Plan 22

Table4

Table2

Table3

Table1

Plan 23

Table4

Table3

Table1

Table2

Plan 24

Table4

Table3

Table2

Table1

bl
a
r
fe

an
r
t
n

no
Table1
a
s Table3
Table2
Table4
Table1
Plan 11
a
h
Table2
Table4
Table3
Plan 12
n) ideTable1
v

Table3
Table1
Plan 13
mTable2Gu Table4
o
c

t
Table3
Table11
Table4
Table1
Plan 14
n
2
e
s
d
t
Table3
Table2
Table1
Table4
Plan 15
u
p
t
s
S
Table3 @
Table2
Table4
Table1
Plan 16
is
n
h
a
t
Table3
Table4
Table1
Table2
Plan 17
r
t
e

s
n
u
Table4
Table2
Table1
Plan 18
oo Table3
o
t
m
Table4
Table1
Table2
Table3
Plan(19
e
s
n
n
a
Table1
Table3
Table2
r Plan 20lice Table4
T
t
Table4
Table2
Table1
Table3
Plan 21
e

uy
g
N

A more intense example of the complexity can be seen with a join involving 20 tables:
153

N! or 20! = 2,432,902,008,176,640,000
The greedy optimizer reduces this to N!/(N-D)!, where D is the maximum depth of the search. In other
words, the depth determines how far into the future the optimizer should look in order to evaluate
whether the query execution plan should be expanded further. In the event of a depth that is set to 7, the
following complexity will be realized:
N!/(N-D)! => 20!/(20-7)! => 20!/13! = 390,700,800
Thus reducing the complexity by a factor approximately 6x10^9.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-4

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Reducing query execution plans is not a guarantee

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Although the greedy optimizer does not guarantee the best possible of all query execution plans (this is currently being
worked on), it can reduce the time spent arriving at a query execution plan for a join involving a great many tables 30,
40, or more by a factor of as much as 1,000. This should eliminate most if not all situations where users thought that
the optimizer had hung when trying to perform joins across many tables.

Thi

optimizer_prune_level

154

The optimizer_prune_level variable tells the optimizer to skip certain plans based on estimates
of the number of rows accessed for each table. This kind of educated guess rarely misses optimal plans,
and may dramatically reduce query plan generation times. That is why this option is on
(optimizer_prune_level=1) by default.
However, this option can be switched off
(optimizer_prune_level=0) with the risk that query plan generation may take much longer. Note
that, even with the use of this heuristic, the optimizer still explores a roughly exponential number of
plans.
optimizer_search_depth

bl
a
r
fe

an
r
t
n

The optimizer_search_depth variable tells how far into the future of each incomplete plan the
optimizer should look in order to evaluate whether it should be expanded further. Smaller values of
optimizer_search_depth may result in orders of magnitude smaller query plan generation times. For
example, queries with 12, 13, or more tables may easily require hours and even days to generate if
optimizer_search_depth is close to the number of tables in the query. At the same time, if
generated with optimizer_search_depth is set to 3 or 4, the query plan may generate in less than
a minute for the same query.
If you are unsure of what a reasonable value is for
optimizer_search_depth, this variable can be set to 0 to tell the optimizer to determine the value
automatically. The default value set is 62.

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-5

MySQL Developer Techniques

6.3
155

Chapter 6: Improving Performance of Joins

EXPLAIN
The EXPLAIN command describes how the MySQL optimizer has decided to execute a SELECT
statement and access the data. The information that is displayed can be used in several ways:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

EXPLAIN can provide information that points out the need to add an index.

Thi

If a table already has indexes, EXPLAIN can be used to find out whether the optimizer is using
them.
If indexes exist but aren't being used, query rewrites in different ways can be tested prior to
executing them. EXPLAIN will provide information that can be used to determine if the rewrites
will perform better by helping the server use the available indexes.
When performing a JOIN, EXPLAIN will provide what type of JOIN is being performed and
provide additional details that can provide assistance in improving the performance of the JOIN.
When using EXPLAIN to analyze a query, it's helpful to have a good understanding of the tables
involved. DESCRIBE can be used to obtain information about a table's columns, and SHOW INDEX can
be used to display information about its indexes.

bl
a
r
fe

an
r
t
n

EXPLAIN works with SELECT queries, but can be used in an indirect way for UPDATE and DELETE
statements as well: A SELECT statement can be written that has the same WHERE clause as the UPDATE
or DELETE and use EXPLAIN to analyze the SELECT.

no
a
s
a
h
Using EXPLAIN
n) query
ethat is to be analyzed For
v
To use EXPLAIN, place the keyword EXPLAIN in front of the SELECT
d
156
i
u command:
omwith thet G
example, the following SELECT statement will be evaluated
EXPLAIN
c

21 den WHERE Language='English'\G


s
t
mysql> EXPLAIN SELECT CountryCode FROM
CountryLanguage
p Stu
*************************** 1. row s
***************************
@
is
id: 1
n
h
a
t
r
select_type: SIMPLE
t use
n
table: CountryLanguage
o
o
to
type: index
m
(
e
possible_keys:
ns
an NULL
r
key:
PRIMARY
e
c
T
tkey_len: 33li
e
y
ref: NULL
u
g
rows:
984
N
Extra: Using where; Using index
1 row in set (#.## sec)

157

The output of this explain tells the end user:


select_type - the query execution is considered SIMPLE (it does not contain subqueries or
unions).
table - the query execution is using only one table (CountryLanguage).
type - that the server will complete a full table scan against the index (instead of against the
data rows themselves).
possible_keys - that there are no possible (NULL) candidate keys to utilize to satisfy the
query.

key - the PRIMARY key will be used to satisfy this query (in spite of the fact that the optimizer
did not list this as a possible candidate key)
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-6

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

key_len - the length (measured in bytes) of the key that will be used. In this case the length of
the primary key is CountryCode is 3 bytes and the length of the Language column is 30
bytes.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

ref - that neither a constant nor another column is being used (NULL), indicating selection by an
expression or range of values.

Thi

rows - the optimizer's approximation of how many rows from the table (or index in this case) it
will need to examine when the actual query execution takes place.
Extra - the query optimizer will use a WHERE clause to identify rows that satisfy the query
(Using where) and can optimize the query by reading values from the index without having
to read the corresponding data rows (Using index).
158

Extending EXPLAIN
Adding the word EXTENDED after the EXPLAIN command will provide not only additional information
to the EXPLAIN output, but will also provide additional information into the SHOW WARNINGS output.
mysql> EXPLAIN EXTENDED SELECT CountryCode FROM CountryLanguage
-> WHERE Language='English'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: CountryLanguage
type: index
possible_keys: NULL
key: PRIMARY
key_len: 33
ref: NULL
rows: 984
filtered: 100.00
Extra: Using where; Using index
1 row in set, 1 warning (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
n
us
o
o
o
t
mysql> SHOW WARNINGS\G
(m nse 1. row ***************************
***************************
n
a
Level: Note
r
ce
T
i
l
t
Code:
1003
ye select `world`.`countrylanguage`.`CountryCode` AS `CountryCode` from
guMessage:
`world`.`countrylanguage` where (`world`.`countrylanguage`.`Language` =
_latin1'English')
1 row in set (#.## sec)

The filtered column indicates an estimated percentage of table rows that will be filtered by the table
condition. In this case, the query optimizer will need to check every row of the index and thus no table
(or index) rows will be filtered.
Issuing the SHOW WARNINGS command after running the EXPLAIN EXTENDED command will
produce information on how the optimizer qualifies tables and column names in the SELECT statement,
what the SELECT looks like after the application of rewriting and optimization rules, and possibly other
notes about the optimization process.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-7

MySQL Developer Techniques

6.4

JOIN's
In the world of databases and SQL, there are three types of JOIN's that are the most prevalent:
Inner join - A join that identifies combinations of matching rows from two tables is called an
inner join. The joining is done by connecting one table, using the primary key, and another table
that references it with a foreign key.

mysql> SELECT L_Name, F_Name, Class_Name, Class_Date


-> FROM Students INNER JOIN Classes ON Students.Student_ID = Classes.Student_ID
-> WHERE Student_ID = '100459';

... or ...
mysql> SELECT L_Name, F_Name, Class_Name, Class_Date FROM Students, Classes
-> WHERE Students.Student_ID = Classes.Student_ID AND
->
Student_ID = '100459';

bl
a
r
fe

an
r
t
n

no
a
se
a
at
h
D
)
nnt_)IDlass_ide
v

,C u
om(StudtemeG
c

21 dessn_Na
s
t
sp s Stu Cla
@
i
n
h
a
t
r
t
e
n | MySQL
Allievo | Sakila
2008-06-09
us for Database Administrators || 2008-06-23
o
o
Allievo | Sakila t|oMySQL Performance Tuning
(m| Sakila
Allievo
| 2008-07-12
se | MySQL High Availability
n
n
a
r
e
lic
et T
D) e
_I m
nt Na
de _
tu , F
( S me
Na
L_

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

159

Thi

Chapter 6: Improving Performance of Joins

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-8

MySQL Developer Techniques

Cross join This type of join is where each row of each table is crossed with each row in
every other table to produce all possible combinations. The result is known as a Cartesian
Product. A Cartesian product is a mathematical (set theory) concept which basically means
combining all elements from the first entity with all elements of the second entity. In the
example below, each student will show up for each class in the classes table thus creating a
roster where every student is "assigned" to every possible class in the university:

Da
te

mysql> SELECT L_Name, F_Name, Class_Name, Class_Date


-> FROM Students CROSS JOIN Classes;

L_ (Stu
Na de
me nt_
, F ID)
_N
am
e

Cla
ss_ (Stude
Na
me nt_ID)
,C
las
s_

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

160

Thi

Chapter 6: Improving Performance of Joins

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n| 2008-06-09
e
v
Allievo | Sakila | MySQL for Datab ...

d
i
m... |G2008-06-09
u
Dude
| Pete
| MySQL for Datab
o
c

t
Hangover | Dee
| MySQL for
...
n | 2008-06-09
2for1 Datab
e
s
Smith
| Joe
| MySQL
Datab
... | 2008-06-09
d
t
p
tu ... | 2008-06-09
Allievo | Sakila | s
MySQL forSDatab
@
s
...
an | eMySQL
thifor Trees | 2008-12-15
r
t
Dude
| Pete
on| Deeto u|sMySQL for Trees | 2008-12-15
Hangover
o
Smith
(m ns| eJoe | MySQL for Trees | 2008-12-15
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-9

MySQL Developer Techniques

Outer join - Whereas an INNER JOIN will find combinations of matching rows from joined
tables, the OUTER JOIN also finds the instances where a row in one table has no match
in another table. For example, the following SQL using a left outer join would look for
students who have a Student_ID but have not attended any classes:

L_ (Stu
Na de
me nt_
, F ID)
_N
am
e

(S
Co tuden
l um
t_
ns ID)
Ou
tpu
te d

mysql> SELECT L_Name, F_Name


-> FROM Students LEFT JOIN Classes ON Students.Student_ID = Classes.Student_ID
-> WHERE Class_Name IS NULL;

bl
a
r
fe

an
r
t
n

Da
te

No

no
a
s
a
h
n| )Dee ide
v

Hangover
m Gu
o
c

twould look for classes that have no


For example, the following SQL using a 1
right outer join
n
2
e
162
s
students enrolled:
pt Stud
s
@ this
mysql> SELECT Class_Name, Class_Date
n
a
r
-> FROM Students RIGHT
e ON Students.Student_ID = Classes.Student_ID
t JOIN Classes
->
WHERE L_Name o
ISnNULL; us
o
to
m
(
e
an icens
r
T
l
t
e
y
gu
N
hi

No (S
Co tud
lum ent
ns _ID
Ou )
tpu
ted

Cla
ss_ (Stude
Na
me nt_ID)
,C
las
s_

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

161

Chapter 6: Improving Performance of Joins

MySQL for Dogs A real howler


| 2008-11-12
MySQL and Trees From leaf to bud | 2008-12-15

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-10

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Inline Lab 6-A


In this lab you will use the country table in the world database to execute a left and right JOIN.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Step 1: Examine the columns that make up the country table

Thi

Open a new terminal window and start the MySQL client. After you have successfully logged into the
MySQL client, utilize the world database by entering the following SQL statement:
mysql> USE world;

In the MySQL client, type the following MySQL DESC statement to list the columns in the country
table:
mysql> DESC Country\G
+----------------+-------------------------------------| Field
| Type
+----------------+-------------------------------------| Name
| char(52)
| Continent
| enum('Asia','Europe','North Ameri ...
| Region
| char(26)
| SurfaceArea
| float(10,2)
| IndepYear
| smallint(6)
| Population
| int(11)
| LifeExpectancy | float(3,1)
| GNP
| float(10,2)
| GNPOld
| float(10,2)
| LocalName
| char(45)
| GovernmentForm | char(45)
| HeadOfState
| char(60)
| Capital
| int(11)
| Code2
| char(2)
+----------------+-------------------------------------16 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
eThe DESC command produces a listing of the columns contained with the country table to include the

uy field name and the field type (other information is produced but not displayed here).
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-11

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 2: Examine the columns that make up the city table

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the same terminal window from step 1, enter the following SQL statement to list the columns in the
city table:

Thi

mysql> DESC City;


+-------------+----------+------+-----+---------+----------------+
| Field
| Type
| Null | Key | Default | Extra
|
+-------------+----------+------+-----+---------+----------------+
| ID
| int(11) | NO
| PRI | NULL
| auto_increment |
| Name
| char(35) | NO
|
|
|
|
| CountryCode | char(3) | NO
|
|
|
|
| District
| char(20) | NO
|
|
|
|
| Population | int(11) | NO
|
| 0
|
|
+-------------+----------+------+-----+---------+----------------+
5 rows in set (#.## sec)

bl
a
r
fe

The DESC command produces a listing of the columns contained with the city table. All of the fields
listed from the DESC command are displayed here.

an
r
t
n

no
a
s
a
h
In the same terminal window from step 2, enter the following SQL statement
to list all the countries and
n) using
eanINNER JOIN:
their corresponding capitals that are located in the 'Eastern Africa'
region
v

d
i
m Gu
o
c

mysql> SELECT Country.Name AS Country, City.Name


nt= City.ID
21 dASeCapital
-> FROM Country INNER JOIN City ON tCountry.Capital
s
-> WHERE Region = 'Eastern Africa';
sp s Stu
+------------+--------------+ @
i
n
h
a
| Country
| Capital
|
t
r
t
e
+------------+--------------+
on t|o us
| Burundi
| Bujumbura
o
| Djibouti
| Djibouti
(m
e ||
| Eritrea an
| Asmarans
r | Addis
ceAbeba |
| Ethiopia
T
i
l
t
e
| Kenya
| Nairobi
|
|
gu| yComoros | Moroni
Step 3: Obtain values from both the city and the country tables using an outer join

| Madagascar | Antananarivo |
| Malawi
| Lilongwe
|
...
| Tanzania
| Dodoma
|
| Uganda
| Kampala
|
| Zimbabwe
| Harare
|
+------------+--------------+
19 rows in set (#.## sec)

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-12

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Enter the following SQL statement to list only the countries and their corresponding capitals that are
located in the 'Eastern Africa' region and have a capital assigned using a LEFT JOIN:

Thi

mysql> SELECT Country.Name AS Country, City.Name AS Capital


-> FROM Country LEFT JOIN City ON Capital = ID
->
WHERE Region = 'Eastern Africa';
+--------------------------------+--------------+
| Country
| Capital
|
+--------------------------------+--------------+
| Burundi
| Bujumbura
|
| Djibouti
| Djibouti
|
| Eritrea
| Asmara
|
| Ethiopia
| Addis Abeba |
| Kenya
| Nairobi
|
| Comoros
| Moroni
|
| Madagascar
| Antananarivo |
| Malawi
| Lilongwe
|
...
| Tanzania
| Dodoma
|
| Uganda
| Kampala
|
| Zimbabwe
| Harare
|
| British Indian Ocean Territory | NULL
|
+--------------------------------+--------------+
20 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-13

MySQL Developer Techniques

6.4.1

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

163

Thi

Chapter 6: Improving Performance of Joins

Nested JOIN's
MySQL uses what is known as a Nested Loop Join Algorithm and is the simplest join-algorithm to
implement. This join-algorithm compares the outer data set (usually a table) row by row with inner data set
rows that match the join-condition. The diagram below represents a visual sampling of what takes place in
the join of the country and city tables:

mysql> SELECT Country.Name, City.Name AS Capital FROM Country JOIN City


-> ON Country.Capital=City.ID;
world.Country
+----------------------+---------+
| Afghanistan
|
1 |
| Netherlands
|
5 |
| Netherlands Antilles |
33 |
| Albania
|
34 |
| Algeria
|
35 |
| American Samoa
|
54 |
| Andorra
|
55 |
| Angola
|
56 |
| Anguilla
|
62 |
| Antigua and Barbuda |
63 |
| United Arab Emirates |
65 |
| Argentina
|
69 |
| Armenia
|
126 |
| Aruba
|
129 |
| Australia
|
135 |
| Azerbaijan
|
144 |
| Bahamas
|
148 |
| Bahrain
|
149 |
| Bangladesh
|
150 |
...
+----------------------+---------+

world.City
+----+------------------+
| 1 | Kabul
|
| 2 | Qandahar
|
| 3 | Herat
|
| 4 | Mazar-e-Sharif
|
| 5 | Amsterdam
|
| 6 | Rotterdam
|
| 7 | Haag
|
| 8 | Utrecht
|
| 9 | Eindhoven
|
| 10 | Tilburg
|
| 11 | Groningen
|
| 12 | Breda
|
| 13 | Apeldoorn
|
| 14 | Nijmegen
|
| 15 | Enschede
|
| 16 | Haarlem
|
| 17 | Almere
|
| 18 | Arnhem
|
| 19 | Zaanstad
|
| 20 | s-Hertogenbosch |
| 21 | Amersfoort
|
| 22 | Maastricht
|
| 23 | Dordrecht
|
| 24 | Leiden
|
| 25 | Haarlemmermeer
|
| 26 | Zoetermeer
|
| 27 | Emmen
|
| 28 | Zwolle
|
| 29 | Ede
|
| 30 | Delft
|
| 31 | Heerlen
|
| 32 | Alkmaar
|
| 33 | Willemstad
|
...
+----+------------------+

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
ethis representation, the world.country table is the outer data set and the world.city table is the
In
guyinner data set. For every record in the country table, the city table is scanned to determine if there is a

match based on the join criteria. From this diagram, it is easy to see that this type of join-algorithm can
have performance issues that must be addressed.
MySQL implements a 'single-sweep multi-join' algorithm which is a variation of the nested loop join
algorithm. In addition, the remainder of this chapter will focus on how to work with this algorithm and the
MySQL server to improve the performance of joins.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-14

MySQL Developer Techniques

6.4.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

164

Thi

Chapter 6: Improving Performance of Joins

Self Joins
One of the more interesting joins is those joins that utilize the same table for both the outer query and the
inner query thus creating a self-join. The most common reason to use a self-join is for evaluating
relationships between different columns of data in the same table. For example, the following SQL uses a
self-join to locate all the countries that gained independence in the same year as Qatar:

mysql> SELECT x1.IndepYear, x1.Name, x2.Name


-> FROM Country AS x1 JOIN Country AS x2
->
ON x1.IndepYear = x2.IndepYear AND x1.Name='Qatar';
+-----------+-------+----------------------+
| IndepYear | Name | Name
|
+-----------+-------+----------------------+
|
1971 | Qatar | United Arab Emirates |
|
1971 | Qatar | Bahrain
|
|
1971 | Qatar | Bangladesh
|
|
1971 | Qatar | Qatar
|
+-----------+-------+----------------------+
4 rows in set (#.## sec)

1971
1971
1971
1971

|
|
|
|

Qatar
Qatar
Qatar
Qatar

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
world.Country
+-----------+----------------------+
| IndepYear | Name
|
+-----------+----------------------+
|
1919 | Afghanistan
|
|
1581 | Netherlands
|
|
1912 | Albania
|
|
1962 | Algeria
|
|
1278 | Andorra
|
|
1975 | Angola
|
|
1981 | Antigua and Barbuda |
|
1971 | United Arab Emirates |
...
|
1991 | Russian Federation
|
|
1945 | Vietnam
|
|
1991 | Estonia
|
|
1776 | United States
|
|
1980 | Zimbabwe
|
+-----------+----------------------+
192 rows in set (#.## sec)

guy

bl
a
r
fe

world.Country
+-----------+----------------------+
| IndepYear | Name
|
+-----------+----------------------+
|
1919 | Afghanistan
|
|
1581 | Netherlands
|
|
1912 | Albania
|
|
1962 | Algeria
|
|
1278 | Andorra
|
|
1975 | Angola
|
|
1981 | Antigua and Barbuda |
|
1971 | United Arab Emirates |
...
|
1991 | Russian Federation
|
|
1945 | Vietnam
|
|
1991 | Estonia
|
|
1776 | United States
|
|
1980 | Zimbabwe
|
+-----------+----------------------+
192 rows in set (#.## sec)

|
|
|
|

United Arab Emirates


Bahrain
Bangladesh
Qatar

The most important aspect to realize from this SQL, is that aliases must be setup (... AS ...) to
distinguish the outer and inner tables from each other. This type of join is very common in databases
where hierarchical data is being stored and retrieved.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-15

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

165

FULL OUTER JOIN

In addition to the left outer join and right outer join, there is a join that combines the results of both left and
right outer joins called a full outer join. In a full outer join, the resulting joined table will contain all records
from both tables, and fill in NULLs where there are holes (missing matches) on either side.

L_
Na
me
,F

Sakila
Pete
Dude
Joe
NULL
NULL
NULL
NULL

|
|
|
|
|
|
|
|

Aviello
Dude
Hangover
Smith
NULL
NULL
NULL
NULL

_N
am
e

|
|
|
|
|
|
|
|

NULL
NULL
NULL
NULL
MySQL
MySQL
MySQL
MySQL

Cla
ss_
Na
me
,C
las
s_D
ate

6.4.3

Chapter 6: Improving Performance of Joins

|
|
|
|
for Database Adm ... |
High Availability
|
for Dogs
|
for Trees
|

NULL
NULL
NULL
NULL
2008-06-09
2008-07-12
2008-11-12
2008-12-15

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) fromidboth
In this example a FULL OUTER JOIN, which displays all the records
etables without any real
v

connecting join column, is represented.


om t Gu
c

n
MySQL does not support FULL OUTER JOIN's21
e
s
d
t
tu
sp s S
166
MySQL does not support the use of a @
FULL OUTER
JOIN (or FULL JOIN) directly; however, there are
n to correctththisi limitation. The most common workaround is to create
workarounds that can be implemented
a
r
t
e query which allows the combination of result sets from two
a UNION
an SQL statement that utilizes
sALL
n
u
o
o
or more SELECT queries:
to
m
(
e
s
n
an* FROM
r
e
mysql> SELECT
Students_noid
AS x LEFT JOIN Classes_noid AS y
c
T
li
->t ON x.F_Name = y.Class_Name
e
uy -> UNION ALL
g
-> SELECT * FROM Students_noid AS x RIGHT JOIN Classes_noid AS y
hi N
-> ON x.F_Name = y.Class_Name;

This SQL statement would handle any duplicates correctly without producing any invalid outputs. It is
necessary to use UNION ALL. If UNION alone was used, duplicates would be eliminated.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-16

MySQL Developer Techniques

6.5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

167

Thi

Chapter 6: Improving Performance of Joins

Improving JOIN performance


A JOIN operation, in any database system, is a complex process that requires database developers,
designers and administrators to work together to ensure that the data, design and system all support the
most optimal environment for executing these operations. With that said, before discussing improving the
performance of JOIN's, the following question needs to be asked: "Is a JOIN operation absolutely
necessary?"
Normalization
Normalization of data in a database is concerned with minimizing the amount of data that is duplicated to
safeguard the database from data incongruity. Ensuring a database is most effectively normalized for the
data that it contains is an important task that database designers must consider. However, those responsible
for designing databases do not usually take into account the performance of the database when the actual
data needs to be retrieved by joining multiple tables.

bl
When a database is over-normalized, there may be a need to denormalize the data down to a level that willera
sf
improve the performance of mission critical queries. In addition, there is a growing number of database
n
a
developers that are using denormalization to simplify complex queries and to reduce the table
-trcount and
n
row accesses. So the question you have to ask yourself is this: "Is optimizing this JOINo
the
best approach
or should I consider denormalizing my data to a level where a JOIN is not necessary
a n or using minimal
s
tables?" Denormalization is especially useful in many web applications orhwhen
a using the Cluster storage
)
engine (NDB).
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
guy
Sometimes the best way to optimize a JOIN is by not doing a JOIN at all!

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-17

MySQL Developer Techniques

6.5.1

Reducing Rows Touched by Joins


The query optimizer will attempt to reduce the number of rows that need to be retrieved in JOIN
operations. The following demonstration shows how this works:

168

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 6: Improving Performance of Joins

mysql> EXPLAIN SELECT Country.Name, City.Name FROM Country JOIN City


-> ON Country.Code = City.CountryCode\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra:
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: world.City.CountryCode
rows: 1
Extra:
2 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
n switchedththei order of the tables (Country and City) to reduce the
In this example, the query optimizer
a
r
t
e outcome is 4079 rows will be touched in the first table while
be touched.
number of rows that would
The
s
n
u
o
only one row will need
o to be touched
to in the second table, thus 4079 x 1 equals only 4079 rows having to
be touched. (m
e
an icens
r
T
l
t
e
y
u

N
Thi

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-18

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

169

Thi

Chapter 6: Improving Performance of Joins

Forcing a specific table order


Forcing the SELECT statement to keep the order entered by using the STRAIGHT_JOIN syntax produces
a different result:
mysql> EXPLAIN SELECT Country.Name, City.Name FROM Country STRAIGHT_JOIN City
->
ON Country.Code = City.CountryCode\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ALL
possible_keys: PRIMARY
key: NULL
key_len: NULL
ref: NULL
rows: 239
Extra:
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
2 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
Using the STRAIGHT_JOIN syntax
thatithe left table is read before the right table. This can be
n ensures
h
a
t
r
used for those (few) cases for
which
the
join
optimizer puts the tables in the wrong order. If the query
t
e

n tables, thisusexample would have been the outcome. The first table would have
optimizer did not reorderothe
o
ttheo second table would have resulted in touching 4079 rows. This would
required touching
239 rows and
m
(
e
s being touched all together. It is clear from this example that the reordering
of resultedainn239 x 4079nrows
r
e
c
of theT
tables
by the
query
optimizer
was in the best interest of performance.
li
t
e
guy

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-19

MySQL Developer Techniques

6.5.2

Covered Searches
One of the dangers in performance is asking for more than is needed. For example, a common approach
seen in executing SELECT statements is to request a complete listing of all columns in the table using the
SELECT * syntax. This can produce poor query execution times and produce a list of other problems
(such as external applications expecting a specific column in a specific position in the output). Even if
the output includes indexed columns, those non-indexed columns require the query to search the data
pages for the output resulting in a performance degradation. The best way to ensure performance of
JOIN's and non-JOIN's is to only use indexed columns both in the output and the WHERE clauses. In this
way, the query only needs to access the indexed pages and is referred to as a "covered search".
The following example demonstrates how this would affect what is sent to the query optimizer to
execute:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

170

Thi

Chapter 6: Improving Performance of Joins

171

mysql> EXPLAIN EXTENDED SELECT Country.*, City.* FROM Country JOIN


-> City ON Country.Code = City.CountryCode AND
-> GovernmentForm='Republic'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
filtered: 100.00
Extra:
172 *************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: world.City.CountryCode
rows: 1
filtered: 100.00
Extra: Using where
2 rows in set, 1 warning (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-20

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

173 mysql> SHOW WARNINGS\G


*************************** 1. row ***************************
Level: Note
Code: 1003
Message: select `world`.`country`.`Code` AS `Code`, `world`.`country`.`Name` AS
`Name`,`world`.`country`.`Continent` AS `Continent`,`world`.`country`.`Region` AS
`Region`, `world`.`country`.`SurfaceArea` AS `SurfaceArea`,
`world`.`country`.`IndepYear` AS `IndepYear`, `world`.`country`.`Population` AS
`Population`, `world`.`country`.`LifeExpectancy` AS `LifeExpectancy`,
`world`.`country`.`GNP` AS `GNP`,`world`.`country`.`GNPOld` AS
`GNPOld`,`world`.`country`.`LocalName` AS `LocalName`,
`world`.`country`.`GovernmentForm` AS `GovernmentForm`,
`world`.`country`.`HeadOfState` AS `HeadOfState`, `world`.`country`.`Capital` AS
`Capital`,`world`.`country`.`Code2` AS `Code2`,`world`.`city`.`ID` AS
`ID`,`world`.`city`.`Name` AS `Name`,`world`.`city`.`CountryCode` AS `CountryCode`,
`world`.`city`.`District` AS `District`,`world`.`city`.`Population` AS `Population`
from `world`.`country` join `world`.`city` where ((`world`.`country`.`GovernmentForm`
= _latin1'Republic') and (`world`.`country`.`Code` = `world`.`city`.`CountryCode`))
1 row in set (#.## sec)

bl
a
r
fe

ns
a
r
t
Not only does the SELECT statement become expanded to handle each column in both the tables,
- but the
n
o
WHERE statement now must search the table data versus being able to execute the query
only
using the
n a performance
a
indexed data. With larger tables with more data and possible more columns, this
can
create
s
a
drain. The better solution is to choose only columns that are actually needed
in the output and to keep the
h
WHERE clauses to indexed fields only.
n) ide
v

Use of GROUP BY and ORDER BY


om t Gu
174
c

nof the query execution. Furthermore,


21performance
e
As stated, accessing the indexes only can improve
s
d
t
p BYSexpressions
tu
limiting indexes to the GROUP BY and/orsORDER
can also improve the performance of
@
queries. Even if it is unrealistic tonlimit
the output
of
the
query
to
indexed columns, limiting the
s
i columns onlyonly
h
a
t
r
GROUP BY and/or ORDER BY
expressions
to
indexed
can
still
have a positive impact on
t
e
s
n
performance.
o to u
o
m
n ( ense
a
r
T
lic
t
e
uy
g
hi N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-21

MySQL Developer Techniques

6.5.3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

175

Thi

Chapter 6: Improving Performance of Joins

Double Checking EXPLAIN


The MySQL server maintains many status variables that provide information about its operation. For the
most part, database administrators have been the end users most interested in these variables; however, as
a developer, many of these variables can provide great insight into improving (and verifying any
improvement) the execution of queries.
In the case of using the EXPLAIN command, server variables can be used to double check that the
EXPLAIN operation is producing the correct query execution plan. The following is a list of server status
variables that can be used in performing this double checking procedure:
Handler_read_first - This server variable displays the number of times the first entry was read from
an index. If this value is high, it suggests that the server is doing a lot of full index scans.
Handler_read_key - This server variable displays the number of requests to read a row based on a
key. If this value is high, it is a good indication that the tables used in query executions are properly
indexed.

176

bl
Handler_read_next - This server variable displays the number of requests to read the next row in ra
fe
key order. This value is incremented if the query is using an index column with a range constraint
or
s
n
if performing an index scan.
tra
n
othe previous row
Handler_read_prev - This server variable displays the number of requests to read
n
a
in key order. This read method is mainly used to optimize ORDER BY ... s
DESC.
a
h
Handler_read_rnd - This server variable displays the number of)requests to read a row based on a
fixed position. This value is high if the server is executing queries
vn thaturequire
ide sorting of the result. A
m
high number here would identify that the server is c
executing
a lotG
of queries that require entire table
o
1 ent
scans or there are joins that don't use keys properly.
2
s
t displays
udthe number of requests to read the next row
Handler_read_rnd_next - This server
pvariable
t
s
S
in the data file. This value will be
lot of table scans. Generally this suggests that the
@high ifththere
is areindexed
n
a
tables in the query executions
are not properly
or that the queries being executed are not
r
tof the indexes.
e

s
written to take advantage
n
o to u
o
m
Select_full_join
This e
server variable displays the number of joins that perform table scans because
( use indexes.
s
n
they
do
not
This value should be 0. If this value is not 0, table indexes should be
n
a
r
e
c
T
i
checked
and
used.
l
et

uy
g
N

Select_full_range_join - This server variable displays the number of joins that used a range search
on a reference table.

Select_range - This server variable displays the number of joins that used ranges on the first table.
This is normally not a critical issue even if the value is quite large.
Select_range_check - This server variable displays the number of joins without keys that check for
key usage after each row. This value should be 0. If this value is not 0, table indexes should be
checked and used.
Select_scan - This server variable displays the number of joins that did a full scan of the first table.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-22

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Inline Lab 6-B

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will verify that EXPLAIN command produces a correct evaluation of the query
execution by looking at a query's effect on SHOW SESSION STATUS.

Thi

Step 1: Review the indexes of the City table


In the MySQL client, type the following MySQL SHOW INDEXES statement to list the indexes in the
city table (located in the world database):
mysql> SHOW INDEX FROM city\G
*************************** 1. row ***************************
Table: city
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: ID
Collation: A
Cardinality: 4079
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

m Gu
o
c

t This index is the primary key and


The city table contains one index which is applied to
1the ID column.
n
e
contains 4079 values in the set (Cardinality). ts2
p Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-23

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 2: Determine how the query optimizer will execute a join using the City and Country tables

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the same terminal window from step 1, enter the following SQL statement to list the columns in the
city table:
mysql> EXPLAIN SELECT SQL_NO_CACHE country.Code, country.Name,
-> city.Name FROM country JOIN city ON country.Code =
-> city.CountryCode WHERE city.CountryCode LIKE 'A%'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: city
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: country
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: world.city.CountryCode
rows: 1
Extra:
2 rows in set (#.## sec)

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
ndeterminesusthat the best way to execute this query is to swap the order of the
The EXPLAIN command
o
o
tables and perform
a table scanto
on the city table and a single row lookup on the country table for each
m
(
e
s
n
matching
city
row.
The
SQL_NO_CACHE
clause is added for verification purposes to ensure that we
npicture of what the server
a an accurate
r
e
c
are T
seeing
is doing.
li
t
e
uy

N
Thi

bl
a
r
fe

Step 3: Verify the EXPLAIN output by performing the steps of the join individually

In the same terminal window from step 2, enter the following SQL statement to count how many records
are contained in the city table:
mysql> SELECT COUNT(*) FROM city;
+----------+
| COUNT(*) |
+----------+
|
4079 |
+----------+
1 row in set (#.## sec)

This output verifies that the EXPLAIN got it right in reference to how many rows would have to be
evaluated in the city table.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-24

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 4: Review the WHERE clause of the earlier JOIN

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the same terminal window from step 3, enter the following SQL statement to determine how many
matches would fit the criteria of the earlier WHERE clause:

Thi

mysql> SELECT COUNT(Code) FROM country WHERE Code LIKE 'A%';


+-------------+
| COUNT(Code) |
+-------------+
|
17 |
+-------------+
1 row in set (#.## sec)

Since "city.CountryCode LIKE 'A%'" filters country codes during the table scan, the server
should only need to touch the Country table when the server knows there is a match. Additionally, if
the city table matches a record in the country table already retrieved in the result set, the server does
not need to retrieve the record twice. This can be seen in the next few steps, but before we evaluate the
SHOW SESSION STATUS, we must first flush the logs with the following command:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
Step 5: Execute and evaluate the SELECT statement
v

u (which will subsequently


omSQLt statements
In the same terminal window from step 4, enter the following
G
c

add fill the status variables):


21 den
s
t
tu
sp scity.Name
S
mysql> SELECT country.Code, country.Name,
@
i
nON country.Code
-> FROM country JOIN city
= city.CountryCode
h
a
t
r
t
->
WHERE city.CountryCode
LIKE
'A%';
e

s
+------+----------------------+--------------------------------+
oon to|uName
| Code | Name (m
|
e
s
+------+----------------------+--------------------------------+
n
a icen
r
| AFG |TAfghanistan
| Kabul
|
l
... et
yAUT | Austria
| Klagenfurt
|
gu|+------+----------------------+--------------------------------+
mysql> FLUSH STATUS;
Query OK, 0 rows affected (#.## sec)

107 rows in set (#.## sec)

mysql> SHOW SESSION STATUS LIKE 'Handler_read%';


+-----------------------+-------+
| Variable_name
| Value |
+-----------------------+-------+
| Handler_read_first
| 0
|
| Handler_read_key
| 15
|
...
| Handler_read_rnd_next | 4080 |
+-----------------------+-------+
6 rows in set (#.## sec)

The values, though slightly off, show that the EXPLAIN provided the correct "guess" as to how the
SELECT statement would perform against the server.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-25

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 6: Determine why session values not completely accurate

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

The first value that we see slightly off is the Handler_read_key variable. In step 4, we determined
that the value should read 17; however, in our output in step 5 it reads 15. In the same terminal window
from step 5, enter the following SQL statement to see why this might be the case:
mysql> SELECT DISTINCT country.Code FROM country
-> WHERE Code NOT IN (SELECT CountryCode FROM city);
+------+
| Code |
+------+
| ATA |
| ATF |
| BVT |
| HMD |
| IOT |
| SGS |
| UMI |
+------+
7 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

m serverGissued
As far as the Handler_read_rnd_next variable, theo
mysql
u a final operation at the end
c

t
of the table scan to detect when no more rows were
available.
This
n 4079).of course caused an additional
21of 4080d(versus
e
operation to be recorded and thus gave us the tvalue
s
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
uye

This SELECT statement shows that there are some countries without cities, including two starting with A
which our query did not need to touch. So when these two country codes are removed, the
Handler_read_key variable is displaying an accurate number.

N
Thi

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-26

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Step 7: Prevent table scan

Thi

The query being executed in this lab was performing a complete table scan which can be disadvantegous
when it comes to performance. This table scan can be eliminated by adding an index to the CountryCode
column in the city table. In the same terminal window used in step 6, enter the following SQL
statements:
mysql> ALTER TABLE city ADD INDEX (CountryCode);
Query OK, 4079 rows affected (0.38 sec)
Records: 4079 Duplicates: 0 Warnings: 0
mysql> EXPLAIN SELECT SQL_NO_CACHE country.Code, country.Name, city.Name
-> FROM country JOIN city ON country.Code = city.CountryCode WHERE
-> city.CountryCode LIKE 'A%'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: city
type: range
possible_keys: CountryCode
key: CountryCode
key_len: 3
ref: NULL
rows: 104
Extra: Using where

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
*************************** 2. row ***************************
c

id: 1
21 den
s
select_type: SIMPLE
t
sp s Stu
table: country
@
i
type: eq_ref
n
h
a
t
r
possible_keys: PRIMARY
t
e
n
key: PRIMARY
us
o
o
o
key_len: 3
t
(m nse
ref:
world.city.CountryCode
n
a
rows:
r 1 e
T
Extra: lic
t
e
gu2 yrows in set (#.## sec)

With the city.CountryCode being indexed, the resulting query will have to only search through this
index; thus eliminating a full table scan.

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-27

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 8: Execute and evaluate the SELECT statement again

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

With the city.CountryCode being indexed, lets execute the SELECT statement against the server to
determine if the EXPLAIN was correct in its evaluation. In the same terminal window used in step 7,
enter the following SQL statements:

Thi

mysql> SELECT COUNT(*) FROM city WHERE CountryCode LIKE 'A%';


+----------+
| COUNT(*) |
+----------+
|
107 |
+----------+
1 row in set (#.## sec)

Our EXPLAIN was slightly off but still in a very close percentage to be considered a very good
"guess" on the number of rows that would need to be touched by the query execution. Before we can
evaluate the SHOW SESSION STATUS, we must again flush the logs with the following command:
mysql> FLUSH STATUS;
Query OK, 0 rows affected (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
It is now time to actually execute the query that we performed an EXPLAIN
s on to verify that the
a
h
EXPLAIN was providing an educated "guess" on the results:
n) ide
v

mysql> SELECT SQL_NO_CACHE country.Code, country.Name,


u WHEREFROM
om tcity.Name
G
c
-> country JOIN city ON country.Code = city.CountryCode

-> city.CountryCode LIKE 'A%';


21 den
s
t
+------+----------------------+--------------------------------+
sp s Stu
| Code | Name
|@
Name
|
i
n
+------+----------------------+--------------------------------+
h
t
tra | Oranjestad
e
| ABW | Aruba
|

s
n
o to u
...
o
| AZE | Azerbaijan
|
(m nse | Mingevir
+------+----------------------+--------------------------------+
n
a
r set l(#.##
107 rowsTin
ce sec)
i
t
ye SHOW SESSION STATUS LIKE 'Handler_read%';
gumysql>
+-----------------------+-------+
| Variable_name
| Value |
+-----------------------+-------+
| Handler_read_first
| 0
|
| Handler_read_key
| 16
|
| Handler_read_next
| 107
|
...
| Handler_read_rnd_next | 0
|
+-----------------------+-------+
6 rows in set (#.## sec)

The Handler_read_next variable (which verify's that an index dataset was used) provides an
accurate count to what was expected and the Handler_read_key is again only slightly over due to
the servers need to verify that the end of the file was reached. In any case, the values are either right on
or close enough to considered an excellent "guess" on the part of the EXPLAIN command.
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-28

MySQL Developer Techniques

Chapter 6: Improving Performance of Joins

Step 9: Other status variables to consider

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

While the Handler% status variables are quite handy, there is another set of server variables that are
also quite useful. In the same terminal window used in step 8, enter the following SQL statement:

Thi

mysql> SHOW SESSION STATUS LIKE 'Select%';


+------------------------+-------+
| Variable_name
| Value |
+------------------------+-------+
| Select_full_join
| 0
|
| Select_full_range_join | 0
|
| Select_range
| 1
|
| Select_range_check
| 0
|
| Select_scan
| 0
|
+------------------------+-------+
5 rows in set (#.## sec)

The Select_range server variable has been incremented to show that the last query execution used
a range on the first table.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-29

MySQL Developer Techniques

6.5.4

JOIN - Best Practices


In addition to the many tools that have been discussed in relation to JOIN's in this chapter, there are
other best practices that should be taken into account when considering joining two or more tables:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

177

Thi

Chapter 6: Improving Performance of Joins

Column types - When performing JOIN operations, the smaller the indexed columns are in size the
better performance will be realized. In addition, numeric column types are considered to be better
JOIN candidates than string column types when considering performance.
Take advantage of memory - If possible, it is best to fit the data sets that are being accessed in the
join into memory. Processing in memory is much faster than accessing hardware and will pay out
large dividends in performance improvement.
Temporary tables - Storing portions of data that is being accessed in temporary tables can limit the
amount of data needing to be accessed and thus improve the performance of JOIN's utilizing that
data.

bl
a
r
fe

Full table scans are not necessarily bad - When it comes to working with large data sets, full table
scans are often faster than range scans.

an
r
t
n

Inner Join preferred - Do not use an outer join if an inner join can be used. An outer join forces the
order, while an inner join allows the optimizer to choose.

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-30

MySQL Developer Techniques

6.5.5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

178

Chapter 6: Improving Performance of Joins

Simulating INTERSECT and MINUS


Users of other database systems may be familiar with the INTERSECT and MINUS (sometimes
implemented as EXCEPT) commands. These commands are not available in MySQL; however, the logic
behind these two commands can easily be simulated in MySQL.
INTERSECT
An INTERSECT is simply an inner join where the data of one table is compared with those of the another
table, and those that match are displayed with any duplicates being weeded out. A typical implementation
of INTERSECT would look something like this:

mysql> SELECT Name, Region FROM country_a


-> INTERSECT
-> SELECT Name, Region FROM country_b;

bl
a
r
fe

Even though MySQL does not specifically have the INTERSECT command, the following SQL statement
will produce the same result:

an
r
t
n

no
a
s
a
h
n) ide
MINUS
v

m taking
u the results from the first SQL
179
ostatements
The outcome of a MINUS operation is to compare two SQL
G
c

t
1second SQLenstatement. If the second SQL statement
statement and removing any duplicates found in the
2
s
t
includes values not found in the first SQL statement,
they
uared simply passed over and not displayed. A
p
t
s
S
typical implementation of MINUS would
@look something
is like this:
n
h
a
t
r
t FROMucountry_a
se
n
mysql> SELECT Name, Region
o
o
-> MINUS
to country_b;
m
(
e
-> SELECT Name,
Region
FROM
an icens
r
T
l does not specifically have the MINUS command, the following SQL statement will
t though MySQL
e
Even
y
gu produce the same result:
N
hi
mysql> SELECT DISTINCT country_a.Name, country_a.Region
-> FROM country_a INNER JOIN country_b
-> USING (Name, Region);

mysql> SELECT DISTINCT country_a.Name, country_a.Region


-> FROM country_a LEFT JOIN country_b
-> USING (Name, Region) WHERE country_b.Name IS NULL;

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-31

MySQL Developer Techniques

6.6

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

180

Thi

Chapter 6: Improving Performance of Joins

Describe how queries are optimized by the MySQL server

Describe the different types of MySQL JOIN's

Describe how MySQL performs JOIN's

Explain how to improve the performance of JOIN's

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

6-32

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 7

bl
a
r
fe

an
r
t
n

no
HIERARCHICAL
a
s
a
h
n) ide
v
DATA

u
om

uy
g
N

c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7. HIERARCHICAL DATA
7.1. Overview
This chapter provides the knowledge and skills necessary to understand and create tables that utilize
tree and hierarchical structures along with the ability to write queries to retrieve the data. At the
completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

182

Thi

Define graphs, trees and hierarchies

Create and modify an adjacency list structure

Create and modify nested set structures

Create and modify path enumeration structures

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-1

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

7.2. Graphs, Trees and Hierarchies

Thi

183

Storing and representing data can take on many forms. One of the more interesting approaches to
storing data is through the use of nested sets. Nested sets are the answer to the branch of mathematics
that deal with graph theory, or abstract structures. The theory behind the mathematics is not as
important as the implementation and thus only a brief introduction of the concept will be presented.
With that said, there are three main graph types:

Graphs - The term "graph", when being used to describe abstract data structures, consists of
nodes (sometimes referred to as vertices) and relationships defining how the nodes are related
to each other (sometimes referred to as connections). The following example represents what
a graph data structure can look like:

10

11

12

bl
a
r
fe

an
r
t
n

no
a
13
14
15
16
17 as18
h
)
n
v uide
m
o
G 24
t
20
19
21 1c
23
22
n
2
e
pts Stud
s
@ thatthareisconnected to each other through paths in the node
This graph contains 24 n
nodes
a
r
structure. If the starting
take a minimum of 5 steps to reach the 24 node
t the node
sisevia9, itthewould
n
(traversing through
graph
following nodes: 10, 16, 22, 23, 24). This graph
u
o
o
to however, if it was a data structure for a maze it would come across
pathway
may look strange;
m
(
e
ns
aasnlogical:
r
e
c
T
li
et
th

184

uy
g
N

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

185

Thi

Chapter 7: Hierarchical Data

Trees - A tree data structure is a specialized graph that works on the principal that there is a
starting point (referred to as root node or parent node) that contains sub-ordinate nodes
(referred to as child nodes). The root node contains data that can be further broken down into
more detailed data in the child node. Child nodes can also have child nodes that can break
down its "parent" node data even further. The following example demonstrates how a tree
data structure could be represented:

Earth

North
America

South
America

Europe

Asia

Africa

Australia

bl
a
r
fe

Canada

Central
America

Island
Nations

no
a
...s
a
h
n) ide
v

om t Gu
c

21 den
s
t
u being the root node) would be a child
From this example, the countries p
t(Earth
s ofIf thethesworld
S
node to the habitable continents.
countries
are smaller in size, they may be grouped
@
i North America).
n
h
together (such as Island
Nations tunder
In addition, there can also be
a
r
tof the data
e

s
further breakdown
to
include
the
states/territories
each country (such as
n
o to uStates). This type of tree data structureof would
Alabama o
under the United
make it easier to
m
(
e
find
the
state
Alabama
by
digging
down
through
the
Earth
North
America
- United States
s one node (or data table) containing all the data.
n enhaving
a
branch versus
r
lic about trees:
et T Generalizations
Alabama

186

an
r
t
n

United
States

uy
g
N

Alaska

Arizona

Arkansas California

Trees contain only one path between any two nodes. Thus Canada (a country
located in North America) can not be connected to both North America and the
United States (another country located in North America) at the same time.
o Every node must be connected to another node in the tree structure. In trees there are
no disconnected nodes.
o When a node is deleted, something must be done to connect any "orphaned" nodes to
the tree or all the sub-ordinate nodes will become trees themselves. For example, if
the North America node was deleted, Canada would become the root node for its
own tree along with Central America, Island Nations and the United States nodes
all becoming root nodes and independent trees themselves no longer being connected
to the Earth root node.
o Each node is unique to the other nodes. Thus the same node can not be connected to
more than one parent node. With that said, a node that would have the same name
such as the country "Georgia" and the state "Georgia" should be uniquely identified
in some means to avoid any confusion.
_________________________________________________________________________________________________
o

_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-3

MySQL Developer Techniques

187

Hierarchical - A hierarchical data structure is a specialized graph structure that contains root
and child nodes but have two advantages over trees:
o

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 7: Hierarchical Data

Inheritance - When a node is removed, sub-ordinate nodes automatically are


promoted to the deleted nodes level. For example, in the case of the leadership of
the United States, if something happened to the President, the Vice President would
be promoted to President, thus inheriting the position.
Multiple-role nodes - In a general tree structure, each node is unique. In a
hierarchical structure, an individual node can take on one or more roles. This is
common in organization charts (where one individual can have many different roles
in the organization) or a parts breakdown (where one part can be used over again).
The following example demonstrates a parts breakdown for a basic birdhouse:

bl
a
r
fe

an
r
t
n

g
N
i
h

no
a
as
Apparel
Electronics
Health
Sports
Fitness
Toys
Electronics
Apparel
Health&&Beauty
Beauty
Sports)&&h
Fitness
Toys
n
e
v

m GuidFitness
o
Portable
c
Baby
Cosmetics
1 ent
Equipment
Music
2
s
d
t
sp s Stu
@
i Hair
n
TV &
Portable
h
a
t
Female
r
t
Video
Music
Care
e

s
n
u
o
o
to
m
(
e
Video
Team
anMaleicens Games
Pharmacy
r
Sports
T
l
yet

Building
Blocks

Board
Games

Video
Games

In this fictious super store, there are 5 different departments (Apparel, Electronics,
Health & Beauty, Sports & Fitness and Toys). Under each department there are
different groups of products that each department is responsible for.
In the case of inheritance, if the apparel department grew in size, each of the groups
could become a department of themselves (Apparel would be removed and the three
groups underneath apparel would be promoted to departments themselves).
In the case of multiple-role nodes, the portable music group underneath the
electronics department is also present underneath the sports & fitness department
(along with video games being located under both electronics and toys).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-4

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.3. Adjacency List Structures


188

For databases (or any computer program) to recognize and manage graphs, the system must be able to
identify how the individual nodes relate to each other. There are multiple ways to accomplish this.
The first way is to use what is known as an adjacency list.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Undirected graphs

Thi

Undirected graphs are those sets of nodes that either have straight lines signifying that there is no
directional flow or those graphs that have arrows on both ends of the connector showing that the flow
can be either way. The following table would represent an adjacency list for an undirected graph with
five nodes:
Node

Connection

B,C

A,D

A,E

B,E

C,D

B
A

D
C

bl
a
r
fe

an
r
t
n

no
a
Creating a database table to resemble the above adjacency list, requires a tablesthat is denormalized:
189
a
h
)
n ide
mysql> CREATE TABLE ud_graph (Node CHAR(1), Connect CHAR(1));
v
u
om t G('B','A'),
c

mysql> INSERT INTO ud_graph VALUES ('A','B'),


('A','C'),
1
n
2('D','B'),
e
-> ('B','D'), ('C','A'), ('C','E'),
('D','E'),
s
d
t
-> ('E','C'), ('E','D');
sp s Stu
@
i
n
h
a
t
mysql> SELECT Node, Connect
FROM ud_graph;
r
t
e
+------+---------+
on to us
| Node | Connect | o
(m nse
+------+---------+
| A
| Bran
| e
i
| A t |TC
|c
l
| y
Be | A
|
u| B | D
|
g
N |C |A
|
| C
| E
|
| D
| B
|
| D
| E
|
| E
| C
|
| E
| D
|
+------+---------+
10 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-5

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

190

Thi

Chapter 7: Hierarchical Data

Directed graph
The directed graph resembles the undirected graphs with the exception that the
flow is from one node to any other node in a one-way directional pattern (the
node the connection goes to will not return via the same path). This adjacency
list requires a little more detail:
Node

Enter

Exit

an
r
t
n

no
a
srequires a table
a
Creating a database table to resemble the above adjacency list, also
191
h
denormalized:
n) ide
v

u
om
G
c
mysql> CREATE TABLE d_graph (Node CHAR(1), Enter
CHAR(1),
Exit CHAR(1));

t
21 den
s
t
mysql> INSERT INTO d_graph VALUES ('A','B','D'),
tu ('B','E','C'),
sp('D','A','F'),
S
-> ('B','E','A'), ('C','B','F'),
('E','D','B'),
@
s
i
n
h
-> ('E','F','B'), ('F','C','E');
a
t
r
t use
n
o
mysql> SELECT * FROM
o d_graph;to
m
(
+------+-------+------+
se
n | Exit
n
| Node | Enter
|
a
r
e
+------+-------+------+
lic |
tT
| y
Ae | B
| D
u
Ng|| BB || EE || CA ||
F

bl
a
r
fe

that is

| C
| B
| F
|
| D
| A
| F
|
| E
| D
| B
|
| E
| F
| B
|
| F
| C
| E
|
+------+-------+------+
7 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

192

Thi

Chapter 7: Hierarchical Data

Trees

Trees resemble the directed graph but with the direction flowing
down from the root node. With trees appears the idea of parent
nodes. To create an adjacency list for a tree, the idea of a parent
node must be incorporated:
Node

Parent

A
B

bl
a
r
fe

an
r
t
n

no
a
s
a
In this case, the root node (the node on the top of the tree) is A and hashno
parent node. This will be
taken into account when creating the table in MySQL by allowingn
into the
eaNULL value
v and) placing
dnormalized
193
i
parent field associated with the root node. In addition, the m
treetable canube
by having the
G
node column as a primary key with no null's allowed. co
t
1
n
2
s tuKEYdeNOT NULL, Parent CHAR(1));
ptPRIMARY
mysql> CREATE TABLE tree (Node CHAR(1)
s
S
@
s
i
n
h
t ('B','A'), ('C','A'),
mysql> INSERT INTO tree VALUES
tra ('A',NULL),
e

s
n
->
('D','A'), ('E','B'),
('F','B'),
('G','C'), ('H','C');
o to u
o
m se
mysql> SELECTn*(FROM tree;
n
a
+------+--------+
r
e
c
T
i
l
| Nodet | Parent |
ye
+------+--------+
u
g
N | A | NULL |
H

| B
| A
|
| C
| A
|
| D
| A
|
| E
| B
|
| F
| B
|
| G
| C
|
| H
| C
|
+------+--------+
8 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-7

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

194

Thi

Chapter 7: Hierarchical Data

Hierarchy

Hierarchy data sets resemble trees but with the two additional features;
inheritance and multiple-role nodes. The following adjacency list
demonstrates how a node (such as a part used more than once in a
design) can be used in more than one branch:
Node

Parent
E

A
B

bl
a
r
fe

an
r
t
n

no
a
s limitation of hierarchy
a
Now of course, this hierarchy is not in normalized form, but that is an inherent
h
row is NULL. Note
models. Node A is the root node for this hierarchy, thus the parent
this
195
n)field for
econtains
v

d
i
that node E is used in two different branches of the hierarchy
and
thus
the
table
two rows with
m Gu
o
the same node ID (similar to the graph data sets discussed
earlier).
c
1 ent
2
s
pt SParent
tud CHAR(1));
mysql> CREATE TABLE hierarchy (NodesCHAR(1),
@ this
n
a
mysql> INSERT INTO hierarchy
VALUES
('A',NULL), ('B','A'), ('C','A'),
r
t ('F','B'),
e

s
n
-> ('D','A'), ('E','B'),
('E','C'), ('G','C');
o to u
o
m se
mysql> SELECTn*(FROM hierarchy;
n
a
+------+--------+
r
e
c
T
i
l
| Nodet | Parent |
ye
+------+--------+
u
g
N | A | NULL |
G

| B
| A
|
| C
| A
|
| D
| A
|
| E
| B
|
| F
| B
|
| E
| C
|
| G
| C
|
+------+--------+
8 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-8

MySQL Developer Techniques

7.3.1.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

196

Thi

Chapter 7: Hierarchical Data

Search Tree Depths

Once the hierarchical data has been stored in the database, it is time to
develop a means to query the data. There are multiple activities that
may be necessary for the end administrator to accomplish concerning
the data, with the most likely being focused on producing a full tree of
the data.
Displaying the complete hierarchy
Using the hierarchy table produced in the last section, it is possible to
use a self-join query against the table to produce an indented display of
the entire tree:

mysql> SELECT t1.Node AS level1, t2.Node AS level2, t3.Node AS level3


-> FROM hierarchy AS t1
-> LEFT JOIN hierarchy AS t2 ON t2.Parent = t1.Node
-> LEFT JOIN hierarchy AS t3 ON t3.Parent = t2.Node;
+--------+--------+--------+
| level1 | level2 | level3 |
+--------+--------+--------+
| A
| B
| E
|
| A
| B
| F
|
| A
| C
| E
|
| A
| C
| G
|
| A
| D
| NULL
|
| B
| E
| NULL
|
| B
| F
| NULL
|
| C
| E
| NULL
|
| C
| G
| NULL
|
| D
| NULL
| NULL
|
| E
| NULL
| NULL
|
| F
| NULL
| NULL
|
| E
| NULL
| NULL
|
| G
| NULL
| NULL
|
+--------+--------+--------+
14 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ceit is possible to see each branch of the hierarchy. The following describe what the
T
i
l
t
From
this
query,
e

uy
g
N

rows are showing in detail:

First through fourth rows - The first row, shows a branch of the tree which consists of
starting with the root node (A), traversing down through node B into the last node of the
branch (E). The last node of the branch (containing no child nodes) is called a leaf node. The
next row follows this same pattern but this time traversing the A, B and F nodes. The third
and fourth rows traverse the A and C branch to get to the E and G leaf nodes.

Fifth through ninth rows - These rows provide some interesting information about the
branches but with a twist. Level2 of these rows display all the leaf nodes for the hierarchy,
thus node D (which has no children) is associated with the leaf nodes located in the third level
of the hierarchy. So in essence, these rows show the branches that contain a leaf node and
their parent node: node D is the leaf node connected to node A, Node E is the leaf node for
both node B and C, node F is the leaf node connected to node B and node G is the leaf node
connected to node C.

Remaining rows (tenth through fourteenth) - These rows display all the leaf nodes for the
hierarchy in the first column.
_________________________________________________________________________________________________

_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-9

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

197

Chapter 7: Hierarchical Data

Leaf nodes only


There are times when it is beneficial to only view the leaf nodes of the hierarchy. This can be
accomplished through the following self-join query:
mysql> SELECT DISTINCT t1.Node
-> FROM hierarchy AS t1
-> LEFT JOIN hierarchy AS t2
-> ON t1.Node = t2.Parent
-> WHERE t2.Parent IS NULL;
+------+
| Node |
+------+
| D
|
| E
|
| F
|
| G
|
+------+
4 rows in set (#.## sec)

bl
a
r
fe

ns
a
r
t
Similar to the output of the last five rows in the previous query (4 rows are due to n
the- SELECT
o
DISTINCT option that eliminated duplication), this query displays only those nodes
n that are leaf
a
nodes (they have no child nodes connected to them).
s
a
h
198
Another way to accomplish this is by using a subquery to obtain) all the parent
and then
n idisenot alsonodes
comparing those parent nodes against the child nodes. If there isva node that
a parent node,
than it is a leaf node and will be displayed:
om t Gu
c

21 den
s
t
mysql> SELECT DISTINCT Node FROM Hierarchy
sp s Stu
-> WHERE Node NOT IN
@
i
n FROM Hierarchy
-> (SELECT DISTINCT Parent
WHERE Parent IS NOT NULL);
h
a
t
r
t
e
+------+

on to us
| Node |
o
+------+
(m nse
| D
|
n
a
| E
|Tr
lice
| F et |
yG |
u|+------+
g
hi N 4 rows in set (#.## sec)

Subqueries are less performance friendly, but the code and logic in creating them can be
advantageous when creating more complex queries to mine the data.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-10

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Lab 7-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this hands on lab, you will query the adjacency list hierarchical data of the world database.

Thi

Lab Preparation - The contents of the world database will be used to demonstrate the
capabilities of using hierarchical data throughout this chapter. However, the data of the
world database is not in a hierarchical format. The following steps will need to be
accomplished to create hierarchical tables based on the data in the world database:
1. Create a new database called hierarchy:
mysql> CREATE DATABASE Hierarchy;

2. In the Hierarchy database just created, source the location_hierarchy.sql file


(Note: Your instructor will identify the correct location to find this file, for this example the
/tmp directory is used):

bl
a
r
fe

an
r
t
n

no
a
seach type of hierarchical
a
This will create (and populate with data) the following tables. Note,
h
structure associated with the tables will be discussed in detail
n)throughout
ethe remainder of this
v

d
i
chapter:
m Gu
couses

LocationAdjacencyList - This1table
thet tree approach to hierarchies with
n
2
e
each node being a child node containing
a
reference
its parent node (except the root
s
da uniqueto identifier
u
pthet table includes
t
node itself). The content of
called LocationID,
s
S
@
s
i
the reference to thenparent
called
ParentLocationID
and
two
supporting columns
h
a
t
r
t
called LocationName
and
LocationType.
e
on to us
o
LocationMaterializedPath
- This table uses the path enumeration approach to
m
(
e
hierarchies
which
stores
the
path
from
the root node to the individuals node being
sThis approach is possible because
n referenced.
n
a
every node will have a unique path to the
r
e
lic
tT

mysql> SOURCE /tmp/location_hierarchy.sql;

e
guy

root and thus the path becomes a unique identifier. This table also has the same unique
identifier called LocationID along with the two supporting columns called
LocationName and LocationType; however, instead of a parent column
referencing the unique indentifier a string column called LocationPath stores the path
of each node from the root node. This column will be used to identify where each node
fits in the hierarchy structure.

LocationNestedSet - This table uses the nested set hierarchical structure which
stores the left and right edges for each individual node with the root node containing the
furthest most left edge (usually identified by the numeric value of 1) and the furthest
most right edge. This table also has the same unique identifier called LocationID
along with the two supporting columns called LocationName and LocationType.
However, with this table there is a left (Lnode) and right (Rnode) column which
identifies the edges which will be used to parse the hierarchy.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-11

MySQL Developer Techniques


1.

Querying the adjacency list hierarchical data of the world database


Determining the root node in an adjacency list hierarchical data can be accomplished by querying the
data for the row that does not contain a parent identification:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

1.

Thi

Chapter 7: Hierarchical Data

mysql> SELECT * FROM LocationAdjacencyList


-> WHERE ParentLocationID IS NULL;
+------------+------------------+--------------+--------------+
| LocationID | ParentLocationID | LocationName | LocationType |
+------------+------------------+--------------+--------------+
|
1 |
NULL | Earth
| Planet
|
+------------+------------------+--------------+--------------+
1 row in set (#.## sec)

The output identifies that the root node for this table is the planet Earth with a location identifier of 1
and no parent connection.
2. Locate all the children nodes that are directly connected to the root node with the following SQL
statement:
mysql> SELECT children.* FROM LocationAdjacencyList root
-> INNER JOIN LocationAdjacencyList children
->
ON root.LocationID = children.ParentLocationID
->
WHERE root.ParentLocationID IS NULL;
+------------+------------------+---------------+--------------+
| LocationID | ParentLocationID | LocationName | LocationType |
+------------+------------------+---------------+--------------+
|
5 |
1 | Africa
| Continent
|
|
7 |
1 | Antarctica
| Continent
|
|
2 |
1 | Asia
| Continent
|
|
3 |
1 | Europe
| Continent
|
|
4 |
1 | North America | Continent
|
|
6 |
1 | Oceania
| Continent
|
|
8 |
1 | South America | Continent
|
+------------+------------------+---------------+--------------+
7 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
routput identifies
ce the seven continents that are directly connected to the root node (planet Earth).
T
i
l
The
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-12

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

3. Locate all the leaf nodes (those children nodes without any children nodes connected to them) using
the following SQL statement:

Thi

mysql> SELECT * FROM LocationAdjacencyList leaves


-> WHERE NOT EXISTS (
->
SELECT NULL
->
FROM LocationAdjacencyList children
->
WHERE children.ParentLocationID = leaves.LocationID
->
);
+------------+------------------+--------------------+--------------+
| LocationID | ParentLocationID | LocationName
| LocationType |
+------------+------------------+--------------------+--------------+
|
182 |
23 | British Indian ... | Country
|
|
243 |
30 | United States ... | Country
|
|
254 |
32 | Antarctica
| Country
|
|
255 |
32 | French Southern ...| Country
|
|
256 |
32 | Bouvet Island
| Country
|
|
257 |
32 | Heard Island ...
| Country
|
...
|
6269 |
1682 | Valera
| City
|
|
6270 |
1683 | San Felipe
| City
|
|
6271 |
1684 | Cabimas
| City
|
|
6272 |
1684 | Ciudad Ojeda
| City
|
|
6273 |
1684 | Maraca+bo
| City
|
+------------+------------------+--------------------+--------------+
4085 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

This statement is one of multiple ways to accomplish


the e
task
n of querying for the 4,000 plus leaf
21 table.
s
d
t
nodes contained in the LocationAdjacencyList
p Stu Come up with at least two additional
s
ways to accomplish this same task.
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-13

MySQL Developer Techniques

Chapter 7: Hierarchical Data

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4. Use the following query to produce an output that displays the complete hierarchy. Note: This query
will produce 24,000 + records and the LIMIT function has been added to prevent the complete
record set from being displayed.

Thi

mysql> SELECT t1.LocationName AS Level1,


-> t2.LocationName AS level2, t3.LocationName AS level3,
-> t4.LocationName AS level4, t5.LocationName AS level5,
-> t6.LocationName AS level6
-> FROM LocationAdjacencyList AS t1
-> LEFT JOIN LocationAdjacencyList AS t2
->
ON t2.ParentLocationID = t1.LocationID
-> LEFT JOIN LocationAdjacencyList AS t3
->
ON t3.ParentLocationID = t2.LocationID
-> LEFT JOIN LocationAdjacencyList AS t4
->
ON t4.ParentLocationID = t3.LocationID
-> LEFT JOIN LocationAdjacencyList AS t5
->
ON t5.ParentLocationID = t4.LocationID
-> LEFT JOIN LocationAdjacencyList AS t6
->
ON t6.ParentLocationID = t5.LocationID
-> LIMIT 45;

bl
a
r
fe

an
r
t
n

+--------+--------+----------------+--------------------------------+--------------+-------------+
| Level1 | level2 | level3
| level4
| level5
| level6
|
+--------+--------+----------------+--------------------------------+--------------+-------------+
| Earth | Africa | Central Africa | Angola
| Benguela
| Benguela
|
| Earth | Africa | Central Africa | Angola
| Benguela
| Lobito
|
| Earth | Africa | Central Africa | Angola
| Huambo
| Huambo
|
| Earth | Africa | Central Africa | Angola
| Luanda
| Luanda
|
| Earth | Africa | Central Africa | Angola
| Namibe
| Namibe
|
| Earth | Africa | Central Africa | Cameroon
| Centre
| YaoundT
|
| Earth | Africa | Central Africa | Cameroon
| ExtrOme-Nord | Maroua
|
...
| Earth | Africa | Central Africa | Sao Tome and Principe
| Aqua Grande | Spo TomT
|
| Earth | Africa | Eastern Africa | British Indian Ocean Territory | NULL
| NULL
|
| Earth | Africa | Eastern Africa | Burundi
| Bujumbura
| Bujumbura
|
| Earth | Africa | Eastern Africa | Comoros
| Njazidja
| Moroni
|
| Earth | Africa | Eastern Africa | Djibouti
| Djibouti
| Djibouti
|
| Earth | Africa | Eastern Africa | Eritrea
| Maekel
| Asmara
|
| Earth | Africa | Eastern Africa | Ethiopia
| Addis Abeba | Addis Abeba |
| Earth | Africa | Eastern Africa | Ethiopia
| Amhara
| Bahir Dar
|
+--------+--------+----------------+--------------------------------+--------------+-------------+
45 rows in set (#.## sec)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

The first 4,084 records will contain the planet (Earth) in the level1 column with the continents
taking the level2 column, the regions taking the level3 column, and so on. The next set of
records will switch over with the continents taking the level1 column , the regions taking the
level2 column and so on. This process will continue until the cities themselves take the level1
column with NULL's filling in the remaining columns. Use the LIMIT [column #],
[# of columns to display] to search different levels of the output remembering that the
output will produce 24,000 + records.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-14

MySQL Developer Techniques

7.3.2.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

199

Thi

Chapter 7: Hierarchical Data

Modify Tree Values


Using the adjacency list approach, there are issues that must be addressed and dealt with. One of this
issues is modifying the data in the list. For example, if there is a need to remove a parent node, such as
node B in the diagram, then something has to be done to the child nodes. In this case, node D will take
over the "parenting" of the child nodes. This is accomplished in a two step process. First, reassign any
of the "parenting" responsibilities from node B to node D:

mysql> UPDATE hierarchy SET Parent='D'


->
WHERE Parent='B';
mysql> SELECT * FROM hierarchy;
+------+--------+
| Node | Parent |
+------+--------+
| A
| NULL
|
| B
| A
|
| C
| A
|
| D
| A
|
| E
| D
|
| F
| D
|
| E
| C
|
| G
| C
|
+------+--------+
8 rows in set (#.## sec)

ns
a
r
t
- G
n
E

no
a
s
a
h
n) ide
v

m Gu
o
c

t list:
The second step would then be to remove the node from
1 the adjacency
n
2
e
s
pt
tud
mysql> DELETE FROM hierarchy WHERE s
Node='B';S
@ this
n
-> SELECT * FROM hierarchy;
a
r
+------+--------+
A
t use
n
| Node | Parent |
o
o
to
+------+--------+
m
(
e
| A
| NULLn |
ns
| C
|TAra
| ce
C
l|i
| D et | A
|
gu|| yEF || DD
|

| E
| C
|
| G
| C
|
+------+--------+
7 rows in set (#.## sec)

bl
a
r
fe

For the most part, modifying the data of an adjacency list is a manual multi-step process. However,
the most limiting issue associated with adjacency lists is that with pure SQL (no stored procedures or
external applications) the end user must determine the level of the hierarchy prior to designing their
queries. This limitation can be resolved using other methods of storing data (discussed later), but by
far the adjacency list is the least complicated approach to hierarchical data in an SQL environment.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-15

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.4. Nested Set Structures

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

200

The adjacency list structure is a simple approach to hierarchical data with built-in limitations and
performance issues. The deeper the tree, the more issues present themselves. The alternative is the
nested set structure. The nested set structure turns the tree inside itself and produces what could be
described as a container-like approach to hierarchal data.

A
E

bl
a
r
fe

ns
a
r
t
-within this
In this example, the top of the tree is the A "container". All paths of the tree are contained
n
o
container. The second level of the tree has three "containers": B, C and D. The B and
aof nthe Ctree."containers"
each have two containers within them, consisting of what would be the third level
s
a
h
201
)
The means by which the nested set structure is stored within a table
involves
is known as a left
n idewhat
v

and right edge approach.


om t Gu
c

Node
Lnode Rnode
1
n A
2
e
s
d
t
A
1
16
sp s Stu
@
iB
D
C
B
2
7
n
h
a
t
r
t
e
C
8
13
on to1 u2s3 E 4 5 F 6 7 8 9 G 10 11 H 12 13 14
o
D
14 (m 15 e
n
ns
a
E Tr 3 ice4
l
t
e
y
F
5
6
u
g
G
9
10
hi N
H

11

15 16

12

The left and right edges identify the edge of the container. For example, starting from left to right, the
left edge of the A "container" is the first edge and is given the identifier 1. The next left edge is in the
B "container" and is given the identifier 2. The third left edge is in the E "container" and is given the
identifier 3. The next edge reached is the right edge of the E "container" and thus, the next identifier,
4, will be used with that edge. This continues until the farthest right edge is reached (the D "container"
is of course taken into account also).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-16

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

202

Thi

Chapter 7: Hierarchical Data

Even though "containers" provide a better visual when describing nested sets, this process of
numbering the left and right edges could have been accomplished using the original tree structure that
was seen earlier:

16

13

14

15

10 11

12

an
r
t
n

an
s
a
h
mysql> CREATE TABLE nested (Node CHAR(1), Lnode INT, Rnode
INT);
)
vn uide
m
mysql> INSERT INTO nested VALUES ('A',1,16), ('B',2,7),
('C',8,13),
G
co nt('H',11,12);

-> ('D',14,15), ('E',3,4), ('F',5,6), 1


('G',9,10),
2
ts tude
pnested;
mysql> SELECT Node, Lnode, Rnode FROM
s
S
@
s
+------+-------+-------+
i
n
h
t
| Node | Lnode | Rnode | tra
e

s
+------+-------+-------+
n
o6 | to u
o
| A
|
1 |
m
( n
| B
|
2 |
7 s
|e
| C
| ran
8 |
13 |
e
| D t |T
14 |lic 15 |
| y
Ee |
3 |
4 |
gu| F | 5 | 6 |

203

bl
a
r
fe

| G
|
9 |
10 |
| H
|
11 |
12 |
+------+-------+-------+
8 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-17

MySQL Developer Techniques

7.4.1.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

204

Chapter 7: Hierarchical Data

Search Tree Depths


Once the hierarchical data has been stored in the database using a nested set, it is time to develop a
means to query the data. Again, there are multiple activities that may be necessary for the end
administrator to accomplish concerning the data.
Leaf nodes only
There are times when it is beneficial to only view the leaf nodes of the hierarchy. This can be
accomplished for nested structures using the following basic query:

mysql> SELECT Node FROM nested


-> WHERE Rnode = Lnode + 1;
+------+
| Node |
+------+
| D
|
| E
|
| F
|
| G
|
| H
|
+------+
5 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
This query displays only those nodes that are leaf nodes (they haven
no) child nodes
connected to them).
eof the
d
The means to accomplish this is by evaluating the left valuem
andv
the right value
node. If there is
i
u
othus mustt G
a difference of one value, then there are no child nodesc
and
be a leaf node.
21 den
Retrieve a single path
s
t
tu
spto display
S
With the nested set structure, the ability
a single path is simplistic in comparison to the
@
s
i the query used to display the complete hierarchy
n below resembles
h
adjacency list structure. The rquery
a
205
t
t
e
with the following changes:
parent
snode is displayed (rather than the child node) until the query
u
onfor the
o
reaches the node selected
the
path
to
terminate.
o
t
m
(
e
ns
anparent_id.Node
r
e
mysql> SELECT
c
T
li
->t FROM nested AS node_id,
e
y
nested AS parent_id
gu ->
-> WHERE node_id.Lnode BETWEEN
N
hi
->
parent_id.Lnode AND parent_id.Rnode
-> AND node_id.Node = 'E'
-> ORDER BY node_id.Lnode;
+------+
| Node |
+------+
| A
|
| B
|
| E
|
+------+
3 rows in set (#.## sec)

16

13

14

15

10 11

12

In this query, the path terminates at the 'E' leaf node due to it being the node identified in the WHERE
clause.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-18

MySQL Developer Techniques

7.4.2.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

206

Chapter 7: Hierarchical Data

Display Depth
With the adjacency list model, it was important for the developer to know in advance the depths of the
hierarchy. This of course is not the most convenient process especially with data that may be changing
on a constant basis. With the nested set model, the query can be designed to determine the depths
without the developer having foreknowledge of the data. The following SQL demonstrates how this
can be accomplished:

mysql> SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS level


-> FROM nested AS node_id,
->
nested AS parent_id
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> GROUP BY node_id.Node
-> ORDER BY node_id.Lnode;
+------+-------+
Level 0
| Node | level |
A
+------+-------+
| A
|
0 |
Level 1
| B
|
1 |
C
B
| E
|
2 |
| F
|
2 |
| C
|
1 |
Level 2
| G
|
2 |
G
F
E
| H
|
2 |
| D
|
1 |
+------+-------+
8 rows in set (#.## sec)

bl
a
r
fe

aD n
r
t
n

no
a
s
a
H
h
)

vn uide
m
co nt G

1
s2 tude
t
p
Indenting levels
s sS
@
iREPEAT, indentation can be added to give a visual
nCONCAT tand
Using SQL string functions like
207
h
a
r
t
e
representation of the depth
s
onleveltojustucreated:
o
(m nse
mysql> SELECT CONCAT(REPEAT('>',
COUNT(parent_id.Node) - 1), node_id.Node)
n
a
-> ASr 'Node Level'
e
lic AS node_id,
->t T
FROM nested
e
nested AS parent_id
uy ->
g
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> GROUP BY node_id.Node
hi N
-> ORDER BY node_id.Lnode;
+------------+
| Node Level |
+------------+
| A
|
| >B
|
| >>E
|
| >>F
|
| >C
|
| >>G
|
| >>H
|
| >D
|
+------------+
8 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-19

MySQL Developer Techniques

7.4.3.

Displaying a specific branch

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

208

Thi

Chapter 7: Hierarchical Data

There are times when it is necessary to display a specific branch of


the hierarchical tree. For example, it may be necessary to only
display the branch that begins with the 'B' node. The limitation to
making this happen with the existing SQL that has been
demonstrated is that any restrictions applied will cause a
corruption of the results. Adding a sub-query can prevent the
corruption of data when restrictions are placed on the output of the
query. The following SQL demonstrates how this can work:

mysql> SELECT node_id.Node,


-> (COUNT(parent_id.Node) - (branch.depth + 1)) AS depth
-> FROM nested AS node_id, nested AS parent_id, nested AS branch_parent,
-> ( SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS depth
->
FROM nested AS node_id, nested AS parent_id
->
WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
->
AND node_id.Node = 'B'
->
GROUP BY node_id.Node
->
ORDER BY node_id.Lnode
-> ) AS branch
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> AND node_id.Lnode BETWEEN branch_parent.Lnode AND branch_parent.Rnode
-> AND branch_parent.Node = branch.Node
-> GROUP BY node_id.Node
209
-> ORDER BY node_id.Lnode;
+------+-------+
| Node | depth |
+------+-------+
| B
|
0 |
| E
|
1 |
| F
|
1 |
+------+-------+
3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-20

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Displaying a specific sub-level


Similar to displaying a specific branch in a hierarchical tree, there
may be times that only the values in a specific sub-level need to
be displayed. For example, it may be necessary to only display
the nodes directly connected to node 'B'. Using the query that was
developed to show a single branch, a HAVING clause can be
added to accomplish this task:

210

Thi

Chapter 7: Hierarchical Data

mysql> SELECT node_id.Node,


-> (COUNT(parent_id.Node) - (branch.depth + 1)) AS depth
-> FROM nested AS node_id, nested AS parent_id, nested AS branch_parent,
-> ( SELECT node_id.Node, (COUNT(parent_id.Node) - 1) AS depth
->
FROM nested AS node_id, nested AS parent_id
->
WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
->
AND node_id.Node = 'B'
->
GROUP BY node_id.Node
->
ORDER BY node_id.Lnode
-> ) AS branch
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> AND node_id.Lnode BETWEEN branch_parent.Lnode AND branch_parent.Rnode
-> AND branch_parent.Node = branch.Node
-> GROUP BY node_id.Node
-> HAVING depth = 1
211
-> ORDER BY node_id.Lnode;
+------+-------+
| Node | depth |
+------+-------+
| E
|
1 |
| F
|
1 |
+------+-------+
2 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-21

MySQL Developer Techniques

7.4.4.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

212

Thi

Chapter 7: Hierarchical Data

Inserting Nodes
Adding nodes to a nested tree hierarchy is an action that
must be addressed due to the likelihood of it happening in a
real-world scenario. There are specific steps that must be
taken to ensure that the hierarchy is maintained properly.
These steps could be turned into a stored procedure;
however, for the purpose of this class the steps will be
presented individually. The following SQL demonstrates
how a node, in this case node 'I', will be added to the first
sub-level directly connected the root node 'A' and to the
right of node 'B':

mysql> LOCK TABLE nested WRITE;


mysql> SELECT @Current_right := Rnode FROM nested WHERE Node = 'B';
+------------------------+
| @Current_right := Rnode |
+------------------------+
|
7 |
+------------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
mysql> UPDATE nested SET Rnode = Rnode + 2 WHERE Rnode n
> )@Current_right;
e
v

d
i
u
om
mysql> UPDATE nested SET Lnode = Lnode + 2 WHERE
Lnode >G
@Current_right;
c

t
21 den
s
t
mysql> INSERT INTO nested(Node, Lnode,
Rnode)
p Stu
-> VALUES ('I', @Current_rights+ 1, @Current_right
+ 2);
@
s
i
n
h
tra se t
mysql> UNLOCK TABLES;

n
o to u
o
m
n ( ense
a
r
T
lic
t
e
guy

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-22

MySQL Developer Techniques

Of course, it is best to check to make sure that these steps actually performed the required action. This
can be accomplished using the indented query presented earlier:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

213

Thi

Chapter 7: Hierarchical Data

mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node)


-> AS 'Node Level' FROM nested AS node_id, nested AS parent_id
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> GROUP BY node_id.Node ORDER BY node_id.Lnode;
+------------+
| Node Level |
A
+------------+
| A
|
| >B
|
| >>E
|
| >>F
|
B
I
C
| >I
|
| >C
|
| >>G
|
| >>H
|
E
F
G
| >D
|
+------------+
9 rows in set (#.## sec)

bl
a
r
fe

ns
a
r
t
nH

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-23

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

214

Thi

Chapter 7: Hierarchical Data

Inserting an only child node


There are times when a child node gets promoted to a
parent node by having a child node of its own added to the
hierarchy. For example, our newly added node 'I' from the
previous step has been given the responsibility of parenting
node 'J'. The previous update steps would not address this
issue, but with a simple modification of those steps it can
easily be accomplished. The following SQL demonstrates
how this would be handled:

mysql> LOCK TABLE nested WRITE;


mysql> SELECT @Current_left := Lnode FROM nested WHERE Node = 'I';
+------------------------+
| @Current_left := Lnode |
+------------------------+
|
8 |
+------------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
as
h
mysql> UPDATE nested SET Lnode = Lnode + 2 WHERE Lnode > )@Current_left;
n ide
v

u
mysql> INSERT INTO nested(Node, Lnode, Rnode) om
G
c

t
VALUES('J', @Current_left + 1, @Current_left
21 + d2);en
s
t
mysql>UNLOCK TABLES;
sp s Stu
@
i
n
h
a
t
r
t
e
Again, these steps accomplishing
215
s should be verified with the following SQL:
on totheutask
o
(m nse
mysql> SELECTnCONCAT(REPEAT('>',
COUNT(parent_id.Node) - 1), node_id.Node)
a
e
-> ASr 'Node Level'
FROM nested AS node_id, nested AS parent_id
c
T
li
->t WHERE node_id.Lnode
BETWEEN parent_id.Lnode AND parent_id.Rnode
e
y
->
GROUP
BY
node_id.Node ORDER BY node_id.Lnode;
u
Ng+------------+
mysql> UPDATE nested SET Rnode = Rnode + 2 WHERE Rnode > @Current_left;

| Node Level |
+------------+
| A
|
| >B
|
| >>E
|
| >>F
|
| >I
|
| >>J
|
| >C
|
| >>G
|
| >>H
|
| >D
|
+------------+
10 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-24

MySQL Developer Techniques

7.4.5.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

216

Thi

Chapter 7: Hierarchical Data

Deleting Nodes
Once a node is in place with the nested set structure, deleting that node can have negative effects on
the hierarchy if the other nodes (there left and right values) are not modified as well. The node
function (parent or child) will determine the process that
is necessary for deleting the node.
Deleting a branch
Deleting a complete branch (or a single child node) is
probably the easiest form of deletion associated with the
nest set structure. For example, node 'C' is a child of
node 'A' and can be deleted from the hierarchy along
with any child nodes. The SQL associated with making
this happen is the exact opposite of an insert of a single
node (utilizing the same basic SQL though):

bl
a
r
fe

ns
a
r
t
mysql> SELECT @Current_left := Lnode AS CL, @Current_right := Rnode AS CR,
n
o
-> @Current_width := Rnode - Lnode + 1 AS CW FROM nested WHERE n
a Node = 'C';
+------+------+------+
s
a
| CL
| CR
| CW
|
h
)
+------+------+------+
n ide
v

|
12 |
17 |
6 |
m Gu
+------+------+------+
o
c

t
1 row in set (#.## sec)
1
n
2
e
s
t
ud
pBETWEEN
t@Current_left
s
mysql> DELETE FROM nested WHERE Lnode
AND @Current_right;
S
@
s
i
an= Rnode
th- @Current_width WHERE Rnode >
r
mysql> UPDATE nested SET Rnode
t
e
@Current_right;
on to us
o
(m SETseLnode = Lnode - @Current_width WHERE Lnode >
mysql> UPDATEnnested n
a
r
@Current_right;
ce
T
i
l
t
ye UNLOCK TABLES;
gumysql>
mysql> LOCK TABLE nested WRITE;

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-25

MySQL Developer Techniques


The SQL steps presented handled the deletion of the entire branch. Like
updates, it is important to review the data to see if the delete actually
took place. The standard indentation SQL that has been used previously
can be used again:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

217

Thi

Chapter 7: Hierarchical Data

mysql> SELECT CONCAT(REPEAT('>', COUNT(parent_id.Node) - 1), node_id.Node)


-> AS 'Node Level' FROM nested AS node_id, nested AS parent_id
-> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
-> GROUP BY node_id.Node ORDER BY node_id.Lnode;
+------------+
| Node Level |
+------------+
| A
|
| >B
|
| >>E
|
| >>F
|
| >I
|
| >>J
|
| >D
|
+------------+
7 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-26

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

218

Thi

Chapter 7: Hierarchical Data


A

Deleting a parent without deleting the children


The more complex deletion process is when a parent node is to be
deleted, but the children should remain in the hierarchy. Now of course,
the easiest way to handle a deletion of a parent is simply keeping the
parent in place but identifying it as an empty parent until a new parent
can take its place (such as a management position being vacated and
kept in the hierarchy until a new manager can be placed in that node).

mysql> LOCK TABLE nested WRITE;


mysql> SELECT @Current_left := Lnode, @Current_right := Rnode FROM nested
-> WHERE Node='I';
+------------------------+-------------------------+
| @Current_left := Lnode | @Current_right := Rnode |
+------------------------+-------------------------+
|
8 |
11 |
+------------------------+-------------------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
as
mysql> UPDATE nested SET Rnode = Rnode - 1, Lnode = Lnode) -h1
n ide
-> WHERE Lnode BETWEEN @Current_left AND @Current_right;
v

u
G
comRnode

t
mysql> UPDATE nested SET Rnode = Rnode - 21WHERE
>
@Current_right;
n
2
e
s
d
t
tu Lnode > @Current_right;
mysql> UPDATE nested SET Lnode = Lnode
sp - s2 SWHERE
@
i
n
h
a
t
mysql> UNLOCK TABLES;
r
t
e
on to us
o
219
Again, these steps
(maccomplishing
e the task should be verified with the following SQL:
s
n
n
a
r
ce
T
i
l
t
mysql>
SELECT
CONCAT(REPEAT('>',
COUNT(parent_id.Node) - 1), node_id.Node)
e-> AS 'Node Level' FROM nested
y
AS node_id, nested AS parent_id
u
Ng -> WHERE node_id.Lnode BETWEEN parent_id.Lnode AND parent_id.Rnode
mysql> DELETE FROM nested WHERE Lnode = @Current_left;

-> GROUP BY node_id.Node ORDER BY node_id.Lnode;


+------------+
| Node Level |
+------------+
| A
|
| >B
|
| >>E
|
| >>F
|
| >J
|
| >D
|
+------------+
6 rows in set (0.00 sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-27

MySQL Developer Techniques

7.5.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

220

Thi

Chapter 7: Hierarchical Data

Path Enumeration
A path enumeration hierarchy model stores the unique path that is created from the root node to every
node in the tree. This path is stored as a string by concatenating either the edges (as in the nested set
hierarchy model) or the keys (as in the adjacency list hierarchy model) of the nodes in the path.
Searches are completed using string functions that parse the stored path values. The following table
represents an example of using the keys to identify the path
A
from root to the node that is represented:
Node

Path

A/B

A/C

A/D

A/B/E

A/B/F

A/C/G

bl
a
r
fe

an
r
t
n

o
n
a
H
A/C/H
s
a
h
The following SQL statements represents how the associated database
for this table would be
n) table
221
e
v

d
i
created and loaded with the hierarchical values:
om t Gu
c

mysql> CREATE TABLE hier_path (Node CHAR(1),


n
21 PathdVARCHAR(15));
e
s
t
p Stu('B','A/B'), ('C','A/C'),
mysql> INSERT INTO hier_path VALUESs('A','A'),
@
-> ('D','A/D'), ('E','A/B/E'),
('F','A/B/F'),
('G','A/C/G'), ('H','A/C/H');
is
n
h
a
t
r
t
e
mysql> SELECT * FROM hier_path;
on to us
o
+------+-------+
(m nse
| Node | Path |
n
+------+-------+
a
r |lice
| A t |TA
|
u|| yBC e || A/B
A/C
g
N | D | A/D ||
| E
| A/B/E |
| F
| A/B/F |
| G
| A/C/G |
| H
| A/C/H |
+------+-------+

The string column (in this case Path) can not be greater than the number of nodes in the given table
and thus is given a variable character identification that can not exceed 15 characters (8 for the node ID
and 7 for the separator character) in this table.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-28

MySQL Developer Techniques

7.5.1.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

222

Chapter 7: Hierarchical Data

Search the depths of the path

Level 1

The path enumeration model is probably the simplest


approach to storing hierarchical data when it is time
to search the depths of the hierarchical tree.

Level 2

Determining the depth of the tree


The maximum depth of the tree can be obtained by
evaluating the longest path string to determine how
many depth levels are contained in the hierarchy. For
example, using the hier_path table, the following
SQL statement determines the maximum levels of
depth that the hierarchy contains:

Level 3

mysql> SELECT MAX(LENGTH(REPLACE(Path,'/',''))/LENGTH(Node)) AS Depth


-> FROM hier_path;
+------------+
| Depth
|
+------------+
| 3.00000000 |
+------------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
This output displays the maximum depth that the hierarchical tree created
by taking the length
n)Thiscontains
ecan
v

d
i
value.
value
then be used in
of the Path value divided by the fixed length of the Node
m reports
umost often required.
o
G
conjunction with other SQL functions to produce the hierarchical
c
ntthe first level being identified as 1
21 startsdewith
s
Note: that the depths for a path enumerationthierarchy
tu the first level as 0.
sp which
which is different than the nested tree approach
identified
S
@
s
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
uye

N
Thi

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-29

MySQL Developer Techniques


Display the depth for each node

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

223

Thi

Chapter 7: Hierarchical Data

Similar to determining the maximum depth of a tree, each nodes depth level can be displayed by
utilizing the same formula:
mysql> SELECT Node, LENGTH(REPLACE(Path,'/',''))/LENGTH(Node)
-> AS Depth_Level FROM hier_path;
+------+-------------+
| Node | Depth_Level |
+------+-------------+
| A
|
1.0000 |
| B
|
2.0000 |
| C
|
2.0000 |
| D
|
2.0000 |
| E
|
3.0000 |
| F
|
3.0000 |
| G
|
3.0000 |
| H
|
3.0000 |
+------+-------------+
8 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v
Level 1

A
om t Gu
c

1
n
s2 tude
t
p
s Level
S2
@
s
i
B
C
n
h
a
t
r
t
e
on to us
o
Level 3
(m nse
n
E
F
G
a
r
e
c
T
i
l
et

Each node and their location in the depth levels of the hierarchy is displayed. The root node will be
the only node on the first level.

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-30

MySQL Developer Techniques

7.5.2.
224

Chapter 7: Hierarchical Data

Retrieving hierarchical data from the paths


In addition to determining the levels of the hierarchy, the ability to locate and display specific nodes
based on a set criteria can be crucial to the success of a hierarchical model chosen.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Retrieve a single path

Thi

Due to the path being stored in the path enumeration hierarchy, retrieving a path for a single node is as easy
as producing a SELECT statement that simply searches for the row associated with the node requested:
mysql> SELECT * FROM hier_path WHERE Node = 'H';
+------+--------+
| Node | Path
|
+------+--------+
| H
| A/C/H |
+------+--------+
1 row in set (#.## sec)

225

Locating children of a node

bl
a
r
fe

s
tran

n
o
n

There are times when it is necessary to report only the children of a specific node. The following SQL
statement will search for all children associated with the B node:

a
s
a
mysql> SELECT * FROM hier_path WHERE Path LIKE '%/B/%'; ) h
n ide
+------+--------+
v

| Node | Path
|
om t Gu
+------+--------+
c

| E
| A/B/E |
21 den
s
t
| F
| A/B/F |
sp s Stu
+------+--------+
@
i
2 rows in set (#.## sec)
n
h
a
t
r
t
E
e

s
n
u
o
o
to
m
(
e
an icens
r
T
l
t
e
y
gu

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-31

MySQL Developer Techniques

Chapter 7: Hierarchical Data

7.6. Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

226

Thi

Define graphs, trees and hierarchies

Create and modify an adjacency list structure

Create and modify nested set structures

Create and modify path enumeration structures

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

7-32

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 8

bl
a
r
fe

an
r
t
n

no
ADVANCED INDEX
a
s
a
h
n) ide
v
STRUCTURES

u
om

uy
g
N

c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8. ADVANCED INDEX STRUCTURES


8.1. Overview

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

228

Thi

This chapter provides the knowledge and skills necessary to utilize advanced index structures. At the
completion of this chapter, you will be able to:

Describe MySQL index types

Optimize end of field searches

Create FULLTEXT indexes for faster text searches

Simulate function based indexes in MySQL

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-1

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

8.2. MySQL Index Types

Thi

229

MySQL offers a number of indexing types to use with the server; however, some storage engines will limit the
type of indexing that can be used and can provide additional indexing types for use with the specific storage
engine. The following are the most common index types used by the MySQL server:
B-Tree
This widely used data structure emulates a tree structure with a set of linked nodes. There are four
common terms for nodes that tree indexes can contain:

Root node This is the starting node for the tree, all nodes within a tree will be connected to
this node, either directly or indirectly.

Parent node This is a node that is connected to a node above it and contains nodes below it.
This is sometimes referred to as an internal node.

Child node This a node that is connected to a parent node. A child node must have at least
one parent node.

bl
a
r
fe

Leaf node This is a node that could be referred to as a child node but with one additional
feature; leaf nodes have no child nodes connected to them.
Due to each node having zero or more child nodes, the graphical representation looks like a tree growing down,
not up. The B-tree indexing type adds an additional requirement that keeps the tree balanced (there is some belief
that the B in B-Tree stands for balanced). A B-tree is kept balanced by requiring that all leaf nodes are at the
same depth. This depth will increase slowly as elements are added to the tree, but an increase in the overall depth
is infrequent, and results in all leaf nodes being one more hop further removed from the root. By maximizing the
number of child nodes for each parent node, the depth of the tree decreases, balancing occurs less often, and
efficiency increases. All nodes have the same size (ie the "block" structure of the index). This is important as BTrees are optimized for minimizing disk reads.

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
Root node
sp s Stu
(non-leaf)
@
i
n
Pointers to
h
tra se t

child leaf
n
o 20to u
nodes
o
40
60
m
( nse
n
a ice
r
T
l
et

uy
g
N

Keys
2-19

19

21

Keys
22-39

39

41

Keys
42-59

Leaf
nodes

59

Pointers to
data

Key
values

Actual data

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

230

Thi

Chapter 8: Advanced Index Structures

B+TREE
Index nodes
The B+Tree index is a data
(i-nodes)
structure to store vast
amounts of information and
3
5
is the next level B-Tree.
Typically B+Trees are used
Leaf
Key
nodes
to store amounts of data that
values
will not fit in main system
memory.
To do this,
3
2
1
5
4
7
6
secondary storage (usually
disk) is used to store the
Pointers to
leaf nodes of the tree. Only
data
the internal nodes of the tree
Links allow for
rapid
in-order
are stored in computer
traversal
Actual data
memory.
In a B+Tree the leaf nodes
are the only ones that
actually store data items.
All other nodes are called index nodes or i-nodes and simply store ''guide'' values which allow us to traverse the
tree structure from the root down and arrive at the leaf node containing the data item we seek. Because disk I/O
is very slow in comparison to memory access these leaf nodes store more than one data item each. In fact, the
data structure will perform best when the size of the leaf nodes closely approximates the size of a disk sector
under most operating systems. Thus, when a search is performed against a B+Tree (by traversing from the root
node down to the proper data node) a read must take place against the data node from the disk to search its
contents. Another way to improve the speed of a query operation is to keep a memory cache of recently read leaf
nodes.
A B+Tree is a rooted search tree satisfying the following properties:
All paths from root to leaf are of the same length.
The index node has ...
o at least 2 children if not a leaf.
o between 0 and (n1) values (index entries), if a leaf.
All other nodes are at least half-full, i.e. they have ...
o between [n/2] and n children, if not a leaf.
o between [(n1)/2] and n1 values (index entries) if a leaf.
The fanout n (width of the index) depends on the size of the search-key and the size of disk
blocks.
Advantages of B Tree and B+Tree indexes:
Automatically reorganizes itself with small, local, changes, in the face of insertions and deletions.
Reorganization of entire file is not required to maintain performance.
Disadvantage of B Tree and B+Tree indexes:
Extra insertion and deletion overhead, space overhead.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-3

MySQL Developer Techniques

HASH
A hash is simply a key/value pair and a hash index is a collection of those key/value pairs. The hash
index works efficiently by taking a lookup value (the value portion of the key/value pair), obtaining the
specific key associated with the lookup value, then transforming the key using a hash function into a
hash, and then using the hash number with the associated hash table to locate the desired value.

231

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 8: Advanced Index Structures

Keys

Indexes

Records
(Rows)

0000
Robert Freed

0001

Pierre Debok

+1-213-555-4567

0002
B20A

bl
a
r
Robert Freed +1-913-555-6786
B20C
fe
s
n
ra
t
n
FAB8
o
n
Gunter Greese
a
FAB9
s
a
h
FABA
n) ide
v

om t Gu
c
Hash indexes have the following characteristics:

n
21 that use
e
s
They are used only for equality comparisons
the = or <=> operators (but are very fast).
d
t
tuas < that find a range of values.
spoperators
They are not used for comparison
such
S
@
s
i to speed up ORDER BY operations (this type of index
na hash index
h
The optimizer cannot
use
a
t
r
t
e
cannot be used n
to search for the
s next entry in order).
u
o
o
o
t approximately how many rows there are between two values (this
MySQL
determine
(miscannot
e
s
statistic
sometimes
n
n used by the optimizer to decide which index to use).
a
r
e
c
T
Only whole
li keys can be used to search for a row. (With a B-tree index, any leftmost prefix of
t the
e
key can be used to find rows).
y
u
Pierre Debok

N
Thi

Gunter Greese +1-315-555-2314

B20B

A HASH index works within a small space and is extremely fast; however, can be very slow
with many non-unique values.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-4

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8.3. FULLTEXT Index

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

232

Thi

The MySQL server supports full-text indexing and searching which is useful for word searches in text and
searches on several columns. A full-text index in MySQL is an index of type FULLTEXT and can be used
only with MyISAM tables.
Creating a FULLTEXT Index
Full-text indexes can be created for CHAR, VARCHAR, or TEXT columns with CREATE TABLE. A
FULLTEXT index may also be added later using ALTER TABLE or CREATE INDEX. In large datasets,
it is much faster to load the data into a table that has no FULLTEXT index and then create the index
subsequent to the load.
The following syntax is an example of a FULLTEXT index being created at table creation time:
mysql>
->
->
->
->
->
->
->

CREATE TABLE film_text (


film_id int(10) NOT NULL default '0',
title varchar(255) NOT NULL default '',
description text,
category_id int(10) NOT NULL default '0',
PRIMARY KEY (film_id),
FULLTEXT KEY title (title,description)
) ENGINE=MyISAM;

bl
a
r
fe

an
r
t
n

no
a
sthe table has already been
a
The following syntax is an example of a FULLTEXT index being addedhafter
created:
n) ide
v

om t Gu
c

mysql> ALTER TABLE film_text


21 den
-> ADD FULLTEXT KEY title (title,description),
s
t
-> engine=MyISAM;
sp s Stu
@
i
n
h
a
t
r
t
e
on to usFULLTEXT is a B-Tree
o
(m nse
n
a
The FULLTEXT
as a kind of normalized, condensed version of the actual text and created as a B-Tree
r indexlisicstored
eof column
T
index (words
are used instead
values). Words that appear frequently in the data losing their effectiveness as
t
e
y
search
terms
are
removed
and
other
words
grouped together. In addition, stopwords (words that are pre-defined by
userver as common) are removed. Whenarea record
g
the
is inserted into a MyISAM table containing a FULLTEXT index,
N
the data for the indexed fields are analyzed and split into words.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-5

MySQL Developer Techniques

8.3.1.

FULLTEXT Search Functions


Full-text searching is performed using MATCH() ... AGAINST syntax. MATCH() takes a commaseparated list that names the columns to be searched. AGAINST takes a string to search for, and an
optional modifier that indicates what type of search to perform. The search string must be a literal string,
not a variable or a column name. There are three types of full-text searches:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

233

Thi

Chapter 8: Advanced Index Structures

234

Boolean - A boolean search interprets the search string using the rules of a special query
language. The string contains the words to search for. It can also contain operators that specify
requirements such that a word must be present or absent in matching rows, or that it should be
weighted higher or lower than usual. Common words such as some or then are stopwords
and do not match if present in the search string. The IN BOOLEAN MODE modifier specifies a
boolean search. The following example demonstrates a FULLTEXT boolean search:

235
mysql> SELECT id, title, body FROM articles WHERE MATCH (title,body)
-> AGAINST ('+MySQL -YourSQL' IN BOOLEAN MODE);
+----+-----------------------+-------------------------------------+
| id | title
| body
|
+----+-----------------------+-------------------------------------+
| 1 | MySQL Tutorial
| DBMS stands for DataBase ...
|
| 2 | How To Use MySQL Well | After you went through a ...
|
| 3 | Optimizing MySQL
| In this tutorial we will show ...
|
| 4 | 1001 MySQL Tricks
| 1. Never run mysqld as root. 2. ... |
| 6 | MySQL Security
| When configured properly, MySQL ... |
+----+-----------------------+-------------------------------------+

bl
a
r
fe

an
r
t
n

236

no
a
s
a
h
n) ide
v

The + and - operators indicate that a word is required


present
respectively, for a
mto be contain
u ortheabsent,
G
match to occur. Thus, this query retrieves allc
theorows that
word MySQL but that
t
n
do not contain the word YourSQL. s21
e
d
t
p Stu
o ft_boolean_syntax - The s
ft_boolean_syntax
system variable displays the list of
@
s
i
operators supportedn
by booleanh
full-text
searches performed using IN BOOLEAN MODE.
t is + -><()~*:""&| and for the most part would
trafor thissevariable
The default value

n
u If there is a need to alter this setting, review the documentation to
o to be altered.
never o
need
o
t
m
understand
the
requirements
necessary to change this variable.
( nse
n

The
plus
(+)
and
minus
(-) operators identify that a word is required to be present or
a
r
e
c
T
i
l
absent,
repectively,
for
a
match to occur.
et

uy
g
N

The greater than (>) and less than (<) operators are used to change the word's
contribution to the relevance value that is assigned to a row. The > operator increases
the contribution and the < operator decreases it.
The Parentheses (()) are used to group words into subexpressions. The most common
use of the parentheses is with the greater than and less than operators.
The tilde (~) acts as a negation operator, causing the word's contribution to the row's
relevance to be negative. A row containing that matches is rated lower than others, but
is not excluded altogether, as it would be with the minus (-) operator.
The asterisk (*) serves as the truncation (or wildcard) operator. Unlike the other
operators, it should be appended to the word to be affected. Words match if they begin
with the word preceding the * operator. In addition, use of this operator will prevent a
word from being stripped from a boolean operation (such as too short of a word or a
stopword).
A phrase that is enclosed within double quote () characters matches only rows that
contain the phrase literally, as it was typed.
_________________________________________________________________________________________________

_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

237

Thi

Chapter 8: Advanced Index Structures

Natural language (default) - A natural language search interprets the search string as a phrase
in natural human language (a phrase in free text). There are no special operators. The stopword
list applies. In addition, words that are present in 50% or more of the rows are considered
common and do not match. Full-text searches are natural language searches if the IN
NATURAL LANGUAGE MODE modifier is given or if no modifier is given. The following
example demonstrates a FULLTEXT natural language search:

mysql> SELECT id, title, body FROM articles WHERE MATCH (title,body)
-> AGAINST ('database' IN NATURAL LANGUAGE MODE);
+----+-------------------+------------------------------------------+
| id | title
| body
|
+----+-------------------+------------------------------------------+
| 5 | MySQL vs. YourSQL | In the following database comparison ... |
| 1 | MySQL Tutorial
| DBMS stands for DataBase ...
|
+----+-------------------+------------------------------------------+

238

bl
a
r
fe

Query expansion - A query expansion search is a modification of a natural language search.


The search string is used to perform a natural language search. Then words from the most
relevant rows returned by the search are added to the search string and the search is done again.
The query returns the rows from the second search. The IN NATURAL LANGUAGE MODE
WITH QUERY EXPANSION or WITH QUERY EXPANSION modifier specifies a query
expansion search. The following example demonstrates a FULLTEXT natural language with
query expansion search:

an
r
t
n

no
a
s
a
h
n) ide
v

mysql> SELECT id, title, body FROM articles


WHERE
u (title,body)
om t GMATCH
-> AGAINST ('database' WITH QUERYc
EXPANSION);
+----+-------------------+------------------------------------------+
21 den
| id | title
| bodyts
|
p Stu
s
+----+-------------------+------------------------------------------+
is for DataBase ...
| 1 | MySQL Tutorial n@
| DBMShstands
|
a
t
r
| 5 | MySQL vs. YourSQL
| e
In the following database comparison ... |
t
| 3 | Optimizing
|
u|s In this tutorial we will show ...
on MySQL
o
o
+----+-------------------+------------------------------------------+
t
(m nse
n
a
ce documents contains the word databases and the word MySQL, the second
t Tr One oflithese

e
guy

search finds the documents that contain the word MySQL even if they do not contain the
word database.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-7

MySQL Developer Techniques

8.4.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

239

Thi

Simulating Function Based Indexes


Function based indexes allow for the creation of indexes on functions or on even expressions. This type of
indexing allows for case insensitive sorts or searches, complex equations, and efficiently extends the SQL
language. MySQL does not naturally contain function based indexes; however, there are steps that can be
completed to simulate these types of indexes.

8.4.1.
240

Chapter 8: Advanced Index Structures

Searching for valid words


There are times when it may be necessary to test a string value against a list of known words. Such as
testing the strength of passwords (is the password too weak by using dictionary words), to solve a puzzle
(such as wordfind or a word search game) or for any other reason that would result in searching one string
against a list of other string values. This of course could be accomplished with a function based index, but
when that is not available, there are other ways to accomplish this with SQL.
Step 1: Populate a table with a list of words

bl
a
r
fe

The first step is to populate a table within the database itself with a list of strings that will be our source
words to search for. This could come from a standard dictionary file or from the user names in your
system (which would be useful when checking password strength). For training purposes, a table of 997
common English words was created from wiktionary.org (part of the Wikipedia.org family of websites)
and dumped to a file called words.sql. The following is a sampling of those words:

an
r
t
n

an
mysql> SELECT * FROM words LIMIT 20;
s
a
+-------------+--------------+
h
)
| word
| reverse_word |
n ide
v

+-------------+--------------+
om t Gu
| above
| evoba
|
c

| abuse
| esuba
|
21 den
| accent
| tnecca
| ts
sp| s Stu
| access
| ssecca
@
| accident
| tnedicca
| hi
n
a
r
| accommodate | etadommocca
|t
t
e

s
| according o|ngnidrocca
u ||
o
| accurate o | etarucca
t
(m n|sdica
e
| acid
|
n
a
| radequate e | etauqeda
|
|
t T| adjustlic | tsujda

uy
g
N

| adopt
| tpoda
|
| adult
| tluda
|
| advance
| ecnavda
|
| advantage
| egatnavda
|
| advice
| ecivda
|
| advise
| esivda
|
| affair
| riaffa
|
| afraid
| diarfa
|
| Africa
| acirfA
|
+-------------+--------------+
20 rows in set (#.## sec)

The words table includes not only the word in proper order, but also the word reversed (which could be
useful in testing passwords or in crossword puzzles where a word could be backwards).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-8

MySQL Developer Techniques


241

Chapter 8: Advanced Index Structures

Step 2: Populate a test table of users and their passwords

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

The following table is created from an sql file called passwords.sql that includes a user_name and
password column:
mysql> SELECT * FROM passwords;
+-----------+-----------+
| user_name | password |
+-----------+-----------+
| dakin
| ssob
|
| dthompson | gwe876pwe |
| jjohnson | pr45tyz
|
| kgreer
| homework |
| llawson
| enots
|
| mhoward
| th1rsty
|
| rspeer
| button
|
+-----------+-----------+
7 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

This table will be used with the words table to evaluate if there are passwords that have a connection to a
dictionary word in our words table.

no
a
s
242
a
h
The following SQL contains two SELECT statements that are joined together
the UNION command
) with

nconnected
e
v

that is designed to identify those users that have passwords that are
to
the
id words in the words
m
u
o
table:
c nt G
1
2
e
tsword tASud'password
mysql> SELECT user_name AS user,
entered',
p
s
S
> word as 'dictionary word','normal'
as 'Output'
@
s
i
n JOIN passwords
> FROM words INNER
ON password=word
h
a
t
r
t
-> UNION
e

n
uASs user, REVERSE(word), word,'reverse' AS 'Output'
-> SELECTouser_name
o
o
t
-> FROM
words
INNER
(m nse JOIN passwords ON password=reverse_word;
+---------+------------------+-----------------+---------+
n
a ic| epassword entered | dictionary word | Output |
| ruser
T
l
t +---------+------------------+-----------------+---------+
e
y
| kgreer | homework
| homework
| normal |
gu
| rspeer | button
| button
| normal |
N
| dakin
| ssob
| boss
| reverse |
hi
Step 3: Create the query to test the passwords

| llawson | enots
| stone
| reverse |
+---------+------------------+-----------------+---------+
4 rows in set (#.## sec)

The first SELECT will test to see if the password matches exactly a word in the words table. If there is a
match the user name is displayed, along with their password entered, the dictionary word that was matched
and an output describing it as a normal match. The second select, which is joined to the first SELECT with
the UNION command. again displays the user name, but this time checks to see if the password is a
dictionary word reversed. If so, the password entered is displayed, the "normal" dictionary word is
displayed and the output reflects that it is a reverse password of a dictionary word.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-9

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

8.5. Optimizing End of Field Searches

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

243

243

Indexes are built with the assumption that words go from left to right. For the majority of fields indexed,
this is true. However, there are times when this is not true and using standard index practice will not
improve searches that need to search the right most portion of a field. For example, the following list
contains e-mail addresses for customers in a typical database mailing list:
andrew.zimmer@yahoo.com
brandon.heath@sun.com
bob.hart@funny.com
david.green@savetheplanet.org
donald.sutherland@actors.com
frank.sinatra@dancers.com
iggy.fresh@unique.com
jane.doe@sun.com
...
If there was a need to contact all customers that have an account on the sun.com mail server, a typical
search would involve the following SQL statement:

bl
a
r
fe

an
r
t
n

no
a
s this specific search is
In this case, even if the email column is indexed, that index is not useful a
because
blocked from using the index lookup with the % wild card at the front)ofh
the search.
n ide
v

Mirror image
om totwork
Gufrom right to left. Until such a
245
This of course could be resolved if the index could1
bec
designedn
feature can be added by an industrius contributor
s2to thetuMySQL
de code, there is a very easy work around
t
p
to correct this problem: the reverse function.
s
S
@
s
i
n
h
The reverse function takes a string
creates
t a mirror image of that string. By creating a new column
tra and
e

in our table (let's call it n


rev_email)
and
then
s
o to u running the following SQL we fill this new column with a
mirror image of theo
e-mail field:
(m customers
se SET rev_email=REVERSE(email);
n
mysql>
UPDATE n
a
r
e
T the data shown
lic above, this UPDATE query will fill the following data into the rev_email column:
t
Using
e
uy
g
moc.oohay@remmiz.werdna
N
i
moc.nus@htaeh.nodnarb
h
mysql> SELECT email FROM customers WHERE email LIKE '%sun.com;

moc.ynnuf@trah.bob
gro.tenalpehtevas@neerg.divad
moc.srotca@dnalrehtus.dlanod
moc.srecnad@artanis.knarf
moc.euqinu@hserf.yggi
moc.nus@eod.enaj
...

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-10

MySQL Developer Techniques

Once the data has been reversed, an index can be applied to this column and the original search could
then be reissued:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

246

Thi

Chapter 8: Advanced Index Structures

mysql> SELECT email FROM customers WHERE rev_email


-> LIKE 'moc.nus%'
+-----------------------+
| email
|
+-----------------------+
| moc.sun@htaeh.nodnarb |
| moc.nus@eod.enaj
|
+-----------------------+

Final thoughts
247

After the rev_email column has been added to the table, keeping this column updated is a job for
triggers. Both an update trigger (to catch changes to e-mail addresses) and an insert trigger (for new data
being added to the table) will be required:

bl
a
r
mysql> CREATE TRIGGER Customers_BU_EmailReverse BEFORE UPDATE ON customersfe
-> FOR EACH ROW SET NEW.rev_email = REVERSE(NEW.email);
ns
a
r
t
n
mysql> CREATE TRIGGER Customers_BI_EmailReverse BEFORE INSERT o
ON customers
-> FOR EACH ROW SET NEW.rev_email = REVERSE(NEW.email);
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-11

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Further Practice Lab 8-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will use the knowledge you have learned in this chapter to create an SQL statement that uses
reverse logic on the Country table in the world database.

Thi

Step 1: Examine the indexes already present in the Country table


Utilizing the world database, list the indexes currently in place for the Country table.
Step 2: Create and test an index on the Name column in the Country table
Create an index on the Name column of the Country table.
Obtain information about how MySQL would execute the following SELECT statement:

bl
a
r
fe

mysql> SELECT Name, Population FROM Country WHERE Name LIKE '%land';
Obtain information about how MySQL would execute the following SELECT statement:

ns
a
r
t
mysql> SELECT Name, Population FROM Country WHERE Name LIKE- 'S%';
on
n
a
Which SELECT statement would perform better? Why?
s
a
h
)
__________________________________________________________________________________
n
v uide
__________________________________________________________________________________
m
o
c nt G
1
__________________________________________________________________________________
2
e
pts Stud
s
Step 3: Create and update a reverse
based
@column
is on the Name column of the world database
n
h
a
t
r
tto the Country
Add a new indexed column
se table called NameReverse that allows for 52 characters
n
u
o
and is located after o
the Name column.
to
m
(
e
n columnewith
Fill this
new
nsa mirror image of the Name column using the REVERSE function on all the
a
r
c
T
t records lini the table.
ecurrent

uy Step 4: Search for a string at the "end" of a column


g
N

Obtain information about how MySQL would execute the following SELECT statement (similar to the
poor performing SELECT statement presented earlier):
mysql> SELECT Name, Population FROM Country
-> WHERE NameReverse LIKE 'dnal%';
The server is now able to utilize the index due the lead % being removed and the result is 14 rows
having to be evaluated versus a fill table scan (293 rows).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-12

MySQL Developer Techniques

Chapter 8: Advanced Index Structures

Step 5: Create triggers to support long-term maintenance of the new column

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

After the NameReverse column has been added to the Country table, keeping this column updated is
a job for triggers. Create both an update trigger (to catch changes to e-mail addresses) and an insert
trigger (for new data being added to the table).

Thi

Step 6: Test the alteration and addition of new rows


With the triggers in place to handle both updates and additions to the Country table, it is best to test
it to make sure that the triggers will work as expected. Test the alteration of existing data with the
following SQL statements:
mysql> UPDATE Country SET Name='Beigeland' WHERE Name='Greenland';
mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%';

Now test the triggers that manage new records being inserted by entering the following SQL
statements into the mysql client:

bl
a
r
fe

an
r
t
n

mysql> INSERT INTO Country (Code, Name) VALUES ('SUN','Sunland');


mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%';

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-13

MySQL Developer Techniques

8.6.

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

248

Thi

Chapter 8: Advanced Index Structures

Describe MySQL index types

Optimize end of field searches

Create FULLTEXT indexes for faster text searches

Simulate function based indexes in MySQL

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

8-14

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 9

bl
a
r
fe

an
r
t
n

no
LOCKING LEVELS
a
as

uy
g
N

h
)
n
v uide
m
o
c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

9
9.1

LOCKING LEVELS
Overview
This chapter provides the knowledge and skills necessary to explain locking levels and create queries that
compensate for the locking level used. At the completion of this chapter, you will be able to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

250

Thi

Chapter 9:Locking Levels

Explain locking fundamentals

Utilize explicit table locks

Explain the different locking techniques of the storage engines

Explain how InnoDB's row-locking and next-key-locking work

Describe the most common issues associated with locking

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-1

MySQL Developer Techniques

9.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

251

Chapter 9:Locking Levels

Locks
Locking is a mechanism that prevents problems from occurring with simultaneous data access by
multiple clients. Locks are managed by the server: It places a lock on data on behalf of one client to
restrict access by other clients to the data until the lock has been released. The lock allows access to data
by the client that holds the lock, but places limitations on what operations can be done by other clients
that are contending for access. The effect of the locking mechanism is to serialize access to data so that
when multiple clients want to perform conflicting operations, each must wait its turn.
When discussing locks it is important to see them as a synchronization mechanism for enforcing limits
on access to a resource in an environment where there are many threads of execution. Locks are one way
of enforcing concurrency control policies and maintaining data integrity. There are two types of locks:

252

Shared locks A shared lock means that many clients can lock the same data (for reading
only), as the lock is shared between the clients. This could be the case when the client wants to
read data and ensure that the data is not changed by anyone else.

Exclusive locks An exclusive lock means that only the client locking the data can access it.
This type of lock is often used when the client is changing data.

Acquiring locks

bl
a
r
fe

an
r
t
n

no
a
Implicit locks For a client that does nothing special to acquireslocks, the MySQL server
a safely. For example,
implicitly acquires locks as necessary to process the client's h
statements
)
n engineidwill
when a client reads or writes rows from a table, the storage
e implicitly use locks to
v

avoid conflicts.
om t Gu
c

n
Explicit locks In some cases, if implicit
iseinsufficient
for a client's purposes, it can
21atlocking
s
d
t
manage locks explicitly by acquiring
them
the
beginning
of an operation and releasing them at
u
p
t
sbe necessary
Swhen a client needs to perform an operation that
the end. Explicit locking may
@
s
i
n
h
spans multiple statements
and
that
must
not
be interrupted by other clients.
a et
r
t

Releasing locks
on to us
253
o
A session can(m
or
serelease locks only for itself. One session cannot acquire locks for another
n acquire
n
sessionra
or release locks
held by another session. Just like acquiring locks, there are explicit and implicit
e
Tto releasellocks
ic that have been acquired:
t
ways
e
uy Explicit release - The UNLOCK TABLES statement explicitly releases any table locks held by
g
the current thread.
hi N
When discussing locks, there are two locking acquisitional types that also need to be understood:

Implicit release - There are multiple events that cause the MySQL server to release a lock that
has been created:
o
o
o

LOCK TABLES releases any table locks currently held by the thread before acquiring new
locks.
Beginning a transaction (for example, with START TRANSACTION) implicitly performs
an UNLOCK TABLES.
If a client connection drops, the server releases table locks held by the client. If the client
reconnects, the locks will no longer be in effect.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-2

MySQL Developer Techniques


Locks granularity

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

254

Thi

Chapter 9:Locking Levels

In MySQL, there are two different levels of locking granularity implemented by different storage
engines:

Table locks This is when a table as a whole is locked on an all-or-nothing basis.

Row locks This is when an individual row is locked from the other clients. Due to the
complexity of performing this locking granularity it is the most difficult one to implement;
however, it provides a high level of concurrency as only a minimal resource (a row) is locked.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-3

MySQL Developer Techniques

9.3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

255

Thi

Chapter 9:Locking Levels

Explicit Locks
When discussing explicit locks, there are two specific types of locks that need to be reviewed: Table locks
and Row locks. Acquisition of explicit locks can be advantageous in certain situations:
Multiple statement update - An implicit lock lasts for the duration of a single query only,
which is unsuitable should you want to perform a multiple-statement update that requires no
interference by other clients. To handle this, explicit locks can be acquired, which remains in
effect until released. Other clients cannot modify tables or rows that are locked.

Grouped statements - Explicit locking can improve performance for multiple statements
executed as a group while the lock is in effect.
o First, less work is required by the server to acquire and release locks because it need not do so
for each statement. It simply acquires all needed locks at the beginning of the operation, and
releases them at the end.
o

Second, for statements that modify data, index flushing is reduced. For example, if multiple
INSERT statements are executed using implicit locking, index flushing occurs following each
statement. If the table is locked explicitly and then all the inserts are performed, index flushing
occurs only once when the lock is released. This results in less disk activity.

bl
a
r
fe

9.3.1

Explicit Table Locks

an
r
t
n

no
a
s
a
h
9.3.1.1 LOCK TABLES statement
n) ide
v

The LOCK TABLES statement names each table to be locked


and
the type
of lock to be acquired. The
m
u
o
G
c
following statement acquires a read lock on the Country
1 tableeandnta write lock on the City table:
2
s
t
pWRITE;
mysql> LOCK TABLES Country READ, City
tud
s
S
@ this
n
a
The following are a list of issues
that
be addressed when using the LOCK TABLES statement:
r
t must
e

s
n
All locks must
u- If any of the tables to be locked are already in use, LOCK TABLES
obe acquired
o
o
t
does not
return
until
it
has
acquired
all of the requested locks.
m
( nse
n
a
r Locking multiple
tables - If multiple tables need to be used while holding an explicit lock, all of
ceto be locked
T
i
l
them need
at the same time because no unlocked tables can be used while the explicit
t
e
y
lock
is
held.
Also,
all
the
tables must be locked with a single LOCK TABLES statement. LOCK
u
g
TABLES
releases
any
locks
that are already held, so it cannot be issued multiple times to acquire
N
256

Clients manage explicit table locks with two statements. LOCK TABLES acquires table locks and UNLOCK
TABLES releases them.

multiple locks.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-4

MySQL Developer Techniques


257

LOCK TABLES types


The following list describes the available LOCK TABLES types and their effects:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 9:Locking Levels

READ - Locks a table for reading.


o A READ lock locks a table for read queries such as SELECT that retrieve data from the table. It
does not allow write operations such as INSERT, DELETE, or UPDATE that modify the table,
even by the client that holds the lock. When a table is locked for reading, other clients can read
from the table at the same time, but no client can write to it. A client that wants to write to a
table that is read-locked must wait until all clients currently reading from it have finished and
released their locks.
WRITE - Locks a table for writing.
o A WRITE lock is an exclusive lock. It can be acquired only when a table is not being used. Once
acquired, only the client holding the write lock can read from or write to the table. Other clients
can neither read from nor write to it. No other client can lock the table for either reading or
writing.
READ LOCAL - Locks a table for reading, but allows concurrent inserts.
o A concurrent insert is an exception to the readers block writers principle. It applies only to
MyISAM tables. If a MyISAM table has no holes in the middle resulting from deleted or
updated records, inserts always take place at the end of the table. In that case, a client that is
reading from a table can lock it with a READ LOCAL lock to allow other clients to insert into
the table while the client holding the read lock reads from it. If a MyISAM table does have
holes, they can be removed by using OPTIMIZE TABLE to defragment the table.
o A READ LOCAL lock can be acquired for a fragmented MyISAM table, or for a non-MyISAM
table, but in such cases, concurrent inserts are not allowed. The lock acts like a regular READ
lock.
LOW_PRIORITY WRITE - Locks a table for writing, but acquires the lock with a lower priority.
o With this lock type, if the client must wait for the lock, other clients that request read locks
during the wait are allowed to get their locks first. A normal write lock request is satisfied when
no other clients are using the table. If other clients are using the table when the request is made,
it waits until those clients have finished. A LOW_PRIORITY WRITE lock request also waits
for any new read requests that arrive while the lock request is pending.

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

1
n
2
e
s
d
t

sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r TABLE
ce statements
T
9.3.1.2 tUNLOCK
i
l
release explicit locks, the client holding the lock must issue an UNLOCK TABLES statement. This
uyeTo
258
g
statement
names no tables, because it releases all explicit locks held by the issuing client. Other ways that
hi N

explicit locks are released:


LOCK TABLES reissued - Explicit locks held by a client also are released if the client issues
another LOCK TABLES statement.
Client termination - Locks cannot be maintained across connections; if a client has any
unreleased locks when its connection to the server terminates, the server implicitly releases its
locks.
Administrator KILL - An administrator with the SUPER privilege can terminate a client
connection with the KILL statement, which causes release of locks held by the client.
START TRANSACTION - Table locks may be affected by transactions and vice versa.
Beginning a transaction with START TRANSACTION causes an implicit UNLOCK TABLES.
Issuing a LOCK TABLES statement will implicitly commit any pending transaction. If any tables
are already locked, issuing an UNLOCK TABLES statement will implicitly commit any pending
transaction.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-5

MySQL Developer Techniques

9.3.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

259

Thi

Chapter 9:Locking Levels

Explicit Row Locks


MySQL (specifically the InnoDB storage engine) implements standard row-level locking utilizing two
types of locks:
A shared lock allows a transaction to read a row.
An exclusive lock allows a transaction to update or delete a row.
Shared row locks
Performing a read in share mode means that latest available data is read, and then a shared mode lock is
set on the rows that were read. This is implemented by using the following SELECT syntax:

mysql> SELECT CountryCode FROM City WHERE Code LIKE 'A%' LOCK IN SHARE MODE;

260

A shared mode lock prevents others from updating or deleting the rows that have been read. Also, if the
latest data belongs to a yet uncommitted transaction of another client connection, the server will wait
until that transaction commits.
Exclusive Row Locks
Performing a read in exclusive mode means that the latest available data is read, and then an exclusive
lock is set on the rows that were read. This is implemented by using the following SELECT syntax:

bl
a
r
fe

an
r
t
n

no
a
s or deleting the rows
Similar to shared mode locks, an exclusive mode lock prevents others fromaupdating
h
that have been read. However, the exclusive mode lock is different from
shared
mode
locks in that other
)

nthat
e
v
users can not perform a lock (either shared or exclusive) on the rows
have
been
read.

m areGreleased
uid when the transaction is
o
Locks set by IN SHARE MODE and FOR UPDATE
reads
c
1 ent
committed or rolled back.
2
s
pt Stud
s
SELECT
@... tFOR
isUPDATE not working?
n
h
a
r
Locking of rows for update using SELECT
only applies when autocommit is disabled (either by
t u...
seFORor byUPDATE
n
o
beginning transaction with START
TRANSACTION
setting
AUTOCOMMIT to 0). If autocommit is enabled, the
o
o
t
m
rows matching the specification
are
not
locked.
n ( ense
a
r
T
lic
t
e
uy
g
N
mysql> SELECT CountryCode FROM City WHERE Code LIKE 'A%' FOR UPDATE;

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-6

MySQL Developer Techniques

9.4

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

261

9.4.1

Chapter 9:Locking Levels

Storage Engine Locking Techniques


The algorithms used by MySQL concerning locking methods are completely dependent upon the storage
engines that are being utilized. Each storage engine is very specific in its implementation of locks. The
two most popular storage engines supported by MySQL, MyISAM and InnoDB, will be discussed in
detail.

MyISAM Locking
The MyISAM storage engine protects against corruption of data by different threads updating in
conflicting ways by using table-level locking. The MyISAM storage engine utilizes one of three types of
locks depending on the request issued by the connecting thread:

READ LOCAL (SHARED) This type of lock allows an INSERT to append only to the end of
the data file. If the INSERT requires a push of data anywhere else but to the end of the data file
(beginning or middle of the data file), the INSERT statement is delayed until the READ LOCAL
LOCK is released. This is the default lock type for SELECT statements.

bl
a
r
READ (SHARED) When this lock is activated, all UPDATE, INSERT and DELETE
fe
s
n
statements are delayed until the lock is released. READ locks are shared locks.
tra
n
WRITE (EXCLUSIVE) This type of lock is activated whenever an UPDATE
no or DELETE
a
request is submitted, or an INSERT is received that would require adding
the data anywhere but
s until the lock is released.
a
to the end of the data file. This lock prevents any activity on the
table
h
WRITE locks are exclusive locks.
n) ide
v

om t Gu
Tables designed toc
Log Activity
n
21INSERT
e
s
d
With the MyISAM storage engine's capability to perform
operations
SELECT queries are running
t
p Stanu excellent choicewhile
against the data (through the READ LOCAL lock s
type), it makes
for applications that are responsible
@end tusers
isthe ability to run reports or other query operations on the data
n
for logging continual activities while also
giving
h
a
r
eMyISAM knows that all new records will always occur at the end of
contained. This is made possiblen
bytthe fact that
s
u
o
the data file and thus there iso
no need to hold
to up SELECT statements that request data from anywhere else in the table.
m
(
e
ns
an Inserts
Concurrent
r
e
c
T
li
t
262
e
A concurrent insert is an INSERT that happens at the same time as SELECT's on the same table. With
y
gu the MyISAM storage engine, it is possible to force SELECT statements to use the READ LOCAL lock
N
regardless if there is room in the middle of the data file for the INSERT to fill. Whether an INSERT will
hi
be a concurrent insert is determined by the concurrent_insert global status variable. This variable
can be assigned by setting the mysqld --concurrent_insert or SET GLOBAL
concurrent_insert parameter. The concurrent_insert parameter can take any one of the
following values:

0 - (Off) This tells the server to not allow any concurrent inserts.

1 - (Default) Enables concurrent insert for MyISAM tables that don't have holes.

2 - Enables concurrent inserts for all MyISAM tables. If the table has a holes and is in use by
another thread the new row will be inserted at the end of the table. If the table is not in use,
MySQL obtains a normal WRITE lock and inserts the new row into the holes.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-7

MySQL Developer Techniques

Priorities
When running a SQL statement that interacts with a MyISAM table, an individual user can determine the
importance level that their individual statements have in regards to priority execution on the MySQL server.
There are four levels of importance in relation to database actions:
1. Highest priority - The database actions that this level of priority (or importance) include SELECT or
INSERT commands that the user or application interfacing with the server has identified as needing
to run before any other SQL statements waiting to be executed by the server.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

263

Thi

Chapter 9:Locking Levels

2.

Normal writes - If there are no high priority SQL statements waiting to be executed, the MySQL
server will allow INSERT, DELETE and UPDATE commands to run first prior to any SELECT
statements that are waiting to be executed by the server.

3.

Normal reads - As stated, SELECT statements are at the bottom of the execution list (unless
identified as high priority) when waiting for execution by the server.

4.

Lowest priority - The database actions at this level of priority will wait for all other SQL statements
to execute prior to actually executing themselves. The end user of the application interfacing with the
server must specifically tell the server that these SQL statements are considered low priority
otherwise the normal writes and reads order will be implemented.

bl
a
r
fe

The following is a list of the MySQL priority commands that can be used:

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

an
r
t
n

HIGH_PRIORITY This command gives the SQL statement being executed the ability to move to
the top of the list of any waiting SQL statements (from all threads) and can be used with the
following SQL statements:
o SELECT - When SELECT HIGH_PRIORITY ... is utilized, MySQL gives the
SELECT a higher priority than a statement that updates a table. This should only be used
for queries that are very fast and must be done at once. A SELECT HIGH_PRIORITY
query that is issued while the table is locked for reading runs even if there is an update
statement waiting for the table to be free. HIGH_PRIORITY cannot be used with SELECT
statements that are part of a UNION.
o INSERT When INSERT HIGH_PRIORITY ... is utilized, MySQL stops any
concurrent insert capabilities and gives the INSERT statement complete control of the table
until the statement has executed completely.

uy
g
N

Priority levels
When using the INSERT HIGH_PRIORITY statement, the server automatically overrides any existing
low_priority_updates setting. The low_priority_updates setting, when set to 1, forces all INSERT, UPDATE,
DELETE, and LOCK TABLE WRITE statements to wait until there is no pending SELECT or LOCK TABLE READ on the
affected table. This variable previously was named sql_low_priority_updates. In addition, by default writing
queries have a higher priority than reading queries.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-8

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

Chapter 9:Locking Levels

LOW_PRIORITY This command places the SQL statement being executed at the bottom of the
list of any waiting and new SQL statements (from all threads) and will only run when all waiting and
new SQL statements have completed. This command can be used with the following SQL
statements:
o INSERT When INSERT LOW_PRIORITY ... is utilized, MySQL delays the
execution of the statement (and stops any other work from the particular client) until no
other clients are reading from the table. This includes any other clients that may access the
table while the INSERT LOW_PRIORITY statement is waiting to execute. In an
environment where there is a large number of clients that can be reading from the server at
any one time this could cause the client to wait a very long time. This should normally not
be used with MyISAM tables because doing so disables concurrent inserts.
o UPDATE The UPDATE LOW_PRIORITY ... works identical to the INSERT
LOW_PRIORITY statement and should also be avoided unless absolutely necessary.
o DELETE The DELETE LOW_PRIORITY ... also works the same as the
INSERT LOW_PRIORITY by not executing until all clients are done reading from the
table. This can cause integrity issues in that another client can be viewing data that
should be deleted immediately.

bl
a
r
fe

Lock starvation

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Using LOW_PRIORITY with UPDATES, INSERTS and DELETES can cause the queries to wait in the queue forever, if
there is no time of day when clients aren't reading from the table.

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-9

MySQL Developer Techniques

9.4.2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

264

Chapter 9:Locking Levels

InnoDB Locking
The InnoDB storage engine supports row-level locking, providing the finest level of lock granularity,
where only the row that is read or updated is locked; and table-level locking, which by default is only used
when there are changes to the table structure itself (as is the case with ALTER TABLE).
Row-level locking - With row-level locking, InnoDB allows other concurrent transactions to
access other rows in the table. These locks are set using the primary key index structure. Any
query that utilizes locks and does not have restrictions on any index will lock all examined rows.

Next-Key Locking
In row level locking InnoDB uses next-key locking. This means that besides index records, InnoDB can also lock the 'gaps'
before or after an index record to block insertions by other users immediately before the index record. A next-key lock
means a lock which locks an index record and the gaps before it. A gap lock means a lock which only locks a gap before
some index record.

Table-level locking - InnoDB uses table locks implicity all the time. The following describes the
different ways that this works:
o

bl
a
r
fe

AUTO_INCREMENT columns - When an InnoDB table contains an


AUTO_INCREMENT column, a table level lock is consistently used. The purpose of this
is to guarantee the uniqueness of auto-inc values, and make them properly sequential
with no gaps. In MySQL 5.1, there are ways to override this feature; however, there
remain situations where auto-inc statements require table level locks to avoid problems
with concurrency.

an
r
t
n

no
a
s
a
h
n) ide
v

o Intention locks - InnoDB sets "intention"m


locks on tables
that it intends to place row
uto the
oremain
G
level locks on. Intention locks usually
invisible
application, but they are
c

t
1 andemaking
n
critical to things like deadlocksprevention
MySQL's explicit table level locks
2
work.
pt Stud
s
@ (--transaction-isolation)
TRANSACTION ISOLATION n
LEVEL
is
h
a
t
r
265
t storagesengine
e that affects locking is the transaction locking level that is
Another aspect of the InnoDB
u
on levels
used. Transaction o
isolation
specify
what data is visible to statements within a transaction. These
to
m
levels directly(impact
the e
level of concurrent access by defining what interaction is possible between
nssame target data source. InnoDB offers all four transaction isolation levels
transactions
the
an against
r
e
c
T
described
by theliSQL standard:
t
e
y
READ UNCOMMITTED (weak protection) - SELECT statements are performed in a nongu
N
locking fashion, but a possible earlier version of a record might be used. Thus, using this
hi

isolation level, such reads are not consistent. The bottom line problem is that open transactions
see changes seen by other open transactions, and if the other open transactions rollback, the
transactions end up using data that was never committed. Thus in essence these transactions
never really "existed". This is also called a dirty read. This setting is rarely used as a default
setting.
READ UNCOMMITTED is the lowest transaction isolation level and can lead to many problems
with the data if used in a production environment with multiple users. However, there are times
when a developer would want to use this transaction isolation level:
o

Changes not finished - The database developer/administrator may want to view some
work that is on-going, such as a batch job that is currently running. By using READ
UNCOMMITTED for a specific session, the database administrator could see changes that
are not yet finished.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-10

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Chapter 9:Locking Levels

Reducing InnoDB locking overhead - When a developer knows their application's


INSERT/UPDATE patterns are very specific and will not result in dirty reads, then READ
UNCOMMITTED is fine; this setting provides greater concurrency because it reduces
InnoDB locking overhead.

SELECT's only - When the developer does not care about dirty reads for some of their
clients. Clients that simply read data, and have no need to update it, are prime candidates
for considering READ UNCOMMITTED at the session level.

READ COMMITTED (better protection) - All SELECT ... FOR UPDATE and SELECT
... LOCK IN SHARE MODE statements lock only the index records, not the gaps before
them, and thus allow the free insertion of new records next to locked records. UPDATE and
DELETE statements using a unique index with a unique search condition lock only the index
record found, not the gaps before it. With this level selected, the changes made by all
COMMITTED transactions are made as soon as they are committed.

REPEATABLE READ (good protection) - This is the default isolation level of InnoDB.
SELECT ... FOR UPDATE, SELECT ... LOCK IN SHARE MODE, UPDATE, and
DELETE statements that use a unique index with a unique search condition lock only the index
record found, not the gaps before it. With other search conditions, these operations employ nextkey locking, locking the index range scanned with next-key or gap locks, and block new
insertions by other users. With this level selected, data reads within the same transaction always
see tables in the same consistent state even if there are transactions committed between the
reads.

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

SERIALIZABLE (zero anomalies) - This levelm


like REPEATABLE
READ, but InnoDB
u
oto isSELECT
G
c
implicitly converts all plain SELECT statements
... LOCK IN SHARE MODE.

t
1
n
2significant
e
This level is rarely used as it can incur
performance penalties. The point with the
s
d
t
p transactions
tu could be replayed serially and it would
name is that in this mode two s
interlacing
S
give the same result.
@
s
i
n
h
a
t
r
t
e
on to usNon-locking SELECT's
o
In each of the InnoDB transaction
levels (except SERIALIZABLE), normal SELECT statements are non-locking.
(m nsisolation
ecompleted
n
Non-locking SELECT
statements
are
InnoDB through the use of Multiversion Concurrency Control (MVCC).
a
r
ce for improving indatabase
i
MVCC is tanT
advanced technique
performance in a multiuser environment. The InnoDB tablespace
l
e
y
contains
many
versions
of
the
same
rows
in
order
to
maintain
between transactions. Different transactions see
u versions of the same rows. Ultimately, each SELECT isolation
g
different
statement
works from a snapshot of the data taken at the
hi Ntime of reading.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-11

MySQL Developer Techniques

Chapter 9:Locking Levels

9.4.2.1 Deadlocks in InnoDB


Deadlocks are a specific condition when two or more processes are each waiting for another to release
a resource, or more than two processes are waiting for resources in a circular chain.
InnoDB automatically detects a deadlock of transactions and rolls back a transaction or transactions to
break the deadlock. InnoDB tries to pick small transactions to roll back, where the size of a transaction
is determined by the number of rows inserted, updated, or deleted.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

266

System variable settings - InnoDB is aware of table locks only if innodb_table_locks=1


(the default) and AUTOCOMMIT=0, and the MySQL layer above it knows about row-level
locks. Otherwise, InnoDB cannot detect deadlocks where a table lock set by a MySQL LOCK
TABLES statement or a lock set by a storage engine other than InnoDB is involved. These
situations must be resolved by setting the value of the innodb_lock_wait_timeout
system variable.
Rollback releases locks, sometimes - When InnoDB performs a complete rollback of a
transaction, all locks set by the transaction are released. However, if just a single SQL statement
is rolled back as a result of an error, some of the locks set by the statement may be preserved.
This happens because InnoDB stores row locks in a format such that it cannot know afterward
which lock was set by which statement.
Avoiding Deadlocks
There are some steps that database developers can follow to minimize the possibility of deadlocking.

bl
a
r
fe

267

no
a
s
Commit often - Small transactions are less prone to collision.
a
h
Lowest isolation level possible - Having an isolation level
is greater
then what is needed
n)that
epossible
v
d
can add greater complexity to the transaction process. Using
the lowest
level,
i
m Guthe possibility isolation
o
while still ensuring that the data is safe, can
help minimize
of a deadlock
c
1 ent
condition.
2
s
udtransactions are set up to access tables and
Access tables and rows in a fixed
- When
ptorder S
t
s
rows in a fixed order, the possibility
ofis
deadlocks occurring is greatly reduced. This is due to
nor@row being
h
a
t
the fact that each ttable
locked in the same order will ensure that no other
r
e

s
transaction running
can goucounter
to the order thus ensuring a one way direction for updates
n
o tupdates
o
o A and then B, whereas another transaction updates B and then A,
(e.g.. If a transaction
m
(
e
the
system
would
be
prone
to deadlocks).
s
n
n
a
r
e
Hope
worst
Tfor the best,
licprepare forof the
t
e
Even
with
an
understanding
deadlocks
and ways to prevent them from occuring, a developer must
uy

N
Thi

an
r
t
n

prepare their applications to handle deadlock conditions. Due to the fact that a deadlock condition will
rollback a transaction, the application must be designed to recognize that a rollback occurred and reissue
the transaction automatically to prevent data loss or lack of data integrity.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-12

MySQL Developer Techniques

9.4.3

Locking with Other Storage Engines


MyISAM and the InnoDB storage engine are currently the most popular and robust storage engines
available for the MySQL server. However, there are other storage engines that address locking also and
are listed below:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

268

Thi

Chapter 9:Locking Levels

MEMORY - The MEMORY storage engine utilizes table-level locking and acts similar to the
MyISAM storage engine in its implementation.

Falcon - The Falcon storage engine uses MVCC (multi-version concurrency control) to provide
record-level concurrency. MVCC is designed to enable records and tables to be updated without
the overhead associated with row-level locking mechanisms. The MVCC implementation
virtually eliminates the need to lock tables or rows during the update process.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-13

MySQL Developer Techniques

9.5

Locking Issues
A few locking issues that arise with applications using MySQL are:

269

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 9:Locking Levels

Blocked operation This takes places when an operation is prevented from completing due to
a lock on the data in question.

Hold lock - The execution time of statements that create a lock on the data accessed must be
considered to ensure that the performance of other statements are not degraded.

Deadlocks - This is a specific condition when two or more processes are each waiting for
another to release a resource, or more than two processes are waiting for resources in a circular
chain. This type of condition can not occur on storage engines that only support table level
locking.

Lock starvation - This condition is similar to the deadlock issue discussed, but with lock
starvation a transaction is prevented from proceeding for an indefinite amount of time while
other transactions continue to run as expected. Lock starvation is the result of other transactions
that are running fine hogging the server thus preventing the transaction in question being
"starved" from server resources.

bl
a
r
fe

ns
a
r
t
Lock timeouts - Within MySQL, there is a setting called table_lock_wait_timeout
that
n
o
determines how long a lock a statement will wait to acquire a lock before
timing
out
and
n
returning an error. The default setting for this variable is 50 seconds. a
s
a
h
Many of the issues can be minimized or completely eliminated by using
smaller

n) idetransactions.
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
uye

N
Thi

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-14

MySQL Developer Techniques

Chapter 9:Locking Levels

Inline Lab 9-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will initiate queries for testing and reviewing of the way the MyISAM and InnoDB storage
engines handle implicit locking.

Thi

Step 1: Verify that you are using MyISAM tables


In the world database, type the following MySQL SHOW statement to determine if the are using the MyISAM
storage engine:
mysql> SHOW TABLE STATUS\G
+-----------------+--------+---------+------------+------+---| Name
| Engine | Version | Row_format | Rows | ...
+-----------------+--------+---------+------------+------+---| City
| MyISAM |
10 | Fixed
| 4079 |
| country
| MyISAM |
10 | Fixed
| 239 |
| countrylanguage | MyISAM |
10 | Fixed
| 984 |
+-----------------+--------+---------+------------+------+---3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
MyISAM storage
nis)not using
ethe
v

We will be using the City table in this lab. If the City table
d
i
omengine
engine, use the following SQL statement to change the c
storage
to u
MyISAM:
G

t
1
n
2
ts tude
mysql> ALTER TABLE City ENGINE=MyISAM;
p
s
S
Query OK, 4079 rows affected (#.## sec)
@
Records: 4079 Duplicates: 0 n
Warnings:
0is
h
tra se t

n
o to u
o
Step 2: Create
an
SQL
m
( nsstatement
e that will lock the City table
n
a
rsame terminal
In the
cewindow from step 1, enter the following SQL statement which will produce a
T
i
l
t
every lengthy running query:
guy
The SHOW TABLE STATUS command will produce a large amount of information concerning the
tables in the database. In this case, the tables listed include all those tables in the world database
(yours may look different based on the status of your world database).

mysql> SELECT c1.Name FROM City AS c1 JOIN City AS c2


->
JOIN City AS c3 ORDER BY RAND() LIMIT 1;

This query will take some time to run and will give you a chance to test the implicit locking capability
of the MyISAM storage engine.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-15

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 3: Test SELECT, INSERT and UPDATE statements against the City table

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Now that the SQL statement from step 2 is continuing to run, open up a second terminal window and
follow the steps from step 1 to log in and use the world database.

Thi

Once you have a second mysql client running and are using the world database, type the following
SQL statement to see if you can retrieve data from the City table:
mysql> SELECT Name FROM City LIMIT 1;
+-------+
| Name |
+-------+
| Kabul |
+-------+
1 row in set (#.## sec)

From this response, it can be determined that the City table is not locked in reference to other clients
being able to retrieve data from it. Now lets try to perform an INSERT against the City table while it
is being read by another client. Type the following SQL statement to see if you can add a record to the
City table:

bl
a
r
fe

an
r
t
n

no
a
mysql> INSERT INTO City VALUES (NULL,'Sakila','AUS','',1); as
h
Query OK, 1 row affected (#.## sec)
)
n
v uide
m
o
Due to the fact that the record could be added to the end
of
the
City
(versus somewhere in the
Gbetableadded.
c the record
t
1
n
middle of the data), the MyISAM storage engine2
allowed
to
However, let's try an
e
s
d
t
update by entering the following SQL statement:
sp s Stu
@
iName='Sakila';
n WHERE
mysql> UPDATE City SET Name='Test'
h
a
t
r
t
e
s
on to auresponse
This SQL statementowill not return
(and will freeze the activity of this client) due to the fact
m
(
e
that the SQL
statement
from
Step
2
is
still
executing.
update can not take place until that SQL
sthe fact that the record wouldThe
nfinishes due
n
a
statement
to
need
to alter records that are involved in the
r
e
c
T
i
l
t
query
from
step
2.
e
guy

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-16

MySQL Developer Techniques

Chapter 9:Locking Levels

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Step 4: View the City table status

Thi

Due to the fact that the first mysql client is running the SQL SELECT statement from step 2 and the
second mysql client is waiting to execute the UPDATE statement from step 3, it will be necessary to
open up a third mysql client in a third terminal window and follow the steps from Step 1 to log in to
the MySQL server.
Once you have a third mysql client running, type the following SQL statement to view the status of the
running processes on the server:
mysql> SHOW PROCESSLIST\G
*************************** 1. row ***************************
Id: 1
User: root
Host: localhost
db: world
Command: Query
Time: 22
State: Copying to tmp table on disk
Info: SELECT c1.Name FROM City AS c1 JOIN City AS c2
JOIN City AS c3 JOIN City AS c4 JOIN City AS c5
ORDER
*************************** 2. row ***************************
Id: 2
User: root
Host: localhost
db: world
Command: Query
Time: 7
State: Locked
Info: UPDATE City SET Name='Test' WHERE Name='Kabul'
*************************** 3. row ***************************
Id: 3
User: root
Host: localhost
db: NULL
Command: Query
Time: 0
State: NULL
Info: SHOW PROCESSLIST
3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

The status of the first client (with an id of 1) is showing that it is still executing and creating a
temporary table to handle the query. The status of the second client (with an id of 2) is showing that it
is locked due to the first client still running. As far as the third client, that is our current connection
and the NULL in the State column identifies that the query has completed successfully.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-17

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 5: Terminate the queries still executing

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In the current mysql client (the third client), type the following commands (in order to prevent the
update from actually executing):

Thi

mysql> KILL QUERY 2;


Query OK, 0 rows affected (#.## sec)

This will kill the query the UPDATE query that is running in the mysql client that is identified by id 2
(you may have to replace the client id with the id that you were given in Step 4). Using KILL QUERY
instead of KILL or KILL CONNECTION allows for the client to continue running after the query it
was executing is terminated.
Type the following command to terminatate the SELECT query that is running in the mysql client that
is identified by id 1 (you may have to replace the client id with the id that you were given in Step 4):
mysql> KILL QUERY 1;
Query OK, 0 rows affected (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
In the first mysql client, type the following command to change the
n)storageidengine
e of the City table
v

to the InnoDB storage engine:


om t Gu
c

mysql> ALTER TABLE City ENGINE=InnoDB;


21 den
s
Query OK, 4079 rows affected (#.## sec)
t
sp 0s Stu
Records: 4079 Duplicates: 0 Warnings:
@
i
n
h
a
t
r
t
e
the ability
This will provide you with
the same steps that were used against the City table
sto runstorage
u
onusing
when the City table
was
the
MyISAM
engine.
o
o
t
m
(
e
n eSQL
nsstatement that will lock the City table
StepT7:ra
Execute the
c
li
t
e
y
gu In the same terminal window from step 6, enter the SQL statement that was run against the City table
Step 6: Alter the City table to use the InnoDB storage engine

in step 2:

mysql> SELECT c1.Name FROM City AS c1 JOIN City AS c2


->
JOIN City AS c3 ORDER BY RAND() LIMIT 1;

This query will take some time to run and will give you a chance to test how the InnoDB storage
engine deals with the possibity of implicit locking.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-18

MySQL Developer Techniques

Chapter 9:Locking Levels

Step 8: Test SELECT, INSERT and UPDATE statements against the City table (using InnoDB)

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

With the SQL statement from step 7 continuing to run, go back to the second terminal window (that
was opened in step 3) and type the following SQL statement to see if you can retrieve data from the
City table:

Thi

mysql> SELECT Name FROM City LIMIT 1;


+-------+
| Name |
+-------+
| Kabul |
+-------+
1 row in set (#.## sec)

From this response, it can be determined that the City table is not locked in reference to other clients
being able to retrieve data from it. Now lets try to perform an INSERT against the City table while it
is being read by another client. Type the following SQL statement to see if you can add a record to the
City table:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
) allowsethe record to be added.
Similar to the MyISAM storage engine, the InnoDB storage enginen
also
v

id running SQL statement;


Both a SELECT and INSERT worked while the table was tied
up
with
au
long
m
o
G
c
however, will the InnoDB storage engine allow an update
to
take
place
in
the data that is currently

t
1
n
being read:
2
e
pts Stud
s
mysql> UPDATE City SET Name='Test'
WHERE is
Name='Sakila';
@
n
Query OK, 2 rows affected (#.##
sec) th
a
r
t2 Warnings:
Rows matched: 2 Changed:
se 0
n
u
o
o
to
m
(
e
Unlike thenUPDATE of s
the City table (when using the MyISAM storage engine) in step 3, this
nthrough
aactuallyicgoes
r
e
UPDATE
while the SELECT statement is running from step 7. This is due to the
T
l storage engine
t
fact
that
the
InnoDB
utilizes row-level locking and thus will allow data to be altered
e
guy while there are searches going on against any other row except the current row that is locked.
mysql> INSERT INTO City VALUES (NULL,'Sakila','AUS','',1);
Query OK, 1 row affected (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-19

MySQL Developer Techniques

9.6

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

270

Thi

Chapter 9:Locking Levels

Explain locking fundamentals

Utilize explicit table locks

Explain the different locking techniques of the storage engines

Explain how InnoDB's row-locking and next-key-locking work

Describe the most common issues associated with locking

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

9-20

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 10

bl
a
r
fe

an
r
t
n

no
CREATING REPORTS
a
as

uy
g
N

h
)
n
v uide
m
o
c nt G
1
2
e
pts Stud
s
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Chapter 10:Creating Reports

10. CREATING REPORTS

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

10.1.

Thi

Overview
This chapter provides the knowledge and skills to build advanced queries used for reporting management
information to end users. At the completion of this chapter, you will be able to:

272

Use functions to calculate multiple conditions

Create calculated reports

Create quarterly reports

Create cross tab reports

Create a bar chart with SQL

Create a decision table to avoid hard wiring of logic in the application code

Create materialized views

an
r
t
n

an
s
a
h
)
Produce sequential or missing data
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-1

MySQL Developer Techniques

Chapter 10:Creating Reports

10.2. Calculate Multiple Conditions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

273

Aggregate functions in SQL is one of the major features and assets of database systems. For the most
part, calculating aggregate data is straightforward and simple enough to create detailed data with little to
no creative SQL having to be implemented.
Data such as how many countries are represented in the dataset is as simple as performing the following
aggregate SQL statement:
mysql> SELECT COUNT(Code) FROM Country;
+-------------+
| COUNT(Code) |
+-------------+
|
239 |
+-------------+
1 row in set (#.## sec)

bl
This can be useful for some situations, but for most situations a little bit more detail about the data isera
sf
needed.
n
a
-tr
Conditional counting
n
274
o
n that breaks
Of course, taking this counting to the next step might result in creating an SQL statement
a this
down the countries based on the continent that they are located in. In MySQL
much
s
aa conditionisinactually
h
simpler than it sounds. In most database systems, a developer can)place
an aggregate
n of theidcondition
function but MySQL makes it even simpler by reducing the length
e that needs to be
v
entered. For example, the following SQL statement demonstrates
how to
conditions in aggregate
Gofuthecreate
comthatnsome

functions that count more specifically based on the 1


continent
countries belong to:
t
s2 tude
t
p
mysql> SELECT SUM(Continent='Asia')sAS Asia,SSUM(Continent='Europe') AS Europe,
@ this
-> FROM Country;
n
a
+------+--------+
r
t use
| Asia | Europe |
n
o
+------+--------+ o
to
m
|
51 |
46(|
e
n ens
+------+--------+
a
r
T
1 row tin set (#.##
licsec)
e
uy
g
This ability to place conditions in an aggregate function eliminates the need to create a WHERE filter with
hi N
multiple subqueries to produce a similar output.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-2

MySQL Developer Techniques


Taking it to the next level

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

275

Thi

Chapter 10:Creating Reports

The resulting data is getting much more useful, but there is still another level that this can be taken. By
adding some creative SQL, the approximate percentages (based on all the data) can be derived:
mysql> SELECT
->
ROUND(100*SUM(Continent='Asia')/COUNT(Code),1) AS Asia,
->
ROUND(100*SUM(Continent='Europe')/COUNT(Code),1) AS Europe
-> FROM Country;
+------+--------+
| Asia | Europe |
+------+--------+
| 21.3 |
19.2 |
+------+--------+
1 row in set (#.## sec)

The ROUND(equation,1) produces a more accurate percentage by bringing the value out 1 decimal
place. Multiple conditions can also be placed into the aggregate functions to get more specific.

276

bl
a
r
fe

an
r
t
n

mysql> SELECT
->
ROUND(100*SUM(Continent='Asia' AND Population > 1000000)/COUNT(Code),1)
->
AS Asia_Large,
->
ROUND(100*SUM(Continent='Asia' AND Population < 1000000)/COUNT(Code),1)
->
AS Asia_Small,
->
ROUND(100*SUM(Continent='Europe' AND Population > 1000000)/COUNT(Code),1)
->
AS Europe_Large,
->
ROUND(100*SUM(Continent='Europe' AND Population < 1000000)/COUNT(Code),1)
->
AS Europe_Small
-> FROM Country;
+------------+------------+--------------+--------------+
| Asia_Large | Asia_Small | Europe_Large | Europe_Small |
+------------+------------+--------------+--------------+
|
18.4 |
2.9 |
14.6 |
4.6 |
+------------+------------+--------------+--------------+
1 row in set (#.## sec)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-3

MySQL Developer Techniques

Chapter 10:Creating Reports

10.3. Create a Calculated Report


MySQL, like other databases, has a built-in function that handles the process of creating subtotals and
totals without having to be creative with SQL. This built-in function is related to aggregate functions
and is the WITH ROLLUP clause. Using the Country table, the following SQL creates a calculated
report for population numbers by the country, averages based on the countries located in a continent and
then finally an average for all countries:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

277

Thi

278

mysql> SELECT Name, Region, ROUND(AVG(Population),0) FROM Country


-> WHERE Continent='North America'
-> GROUP BY Region, Population WITH ROLLUP;
+----------------------------------+-----------------+--------------------------+
| Name
| Region
| ROUND(AVG(Population),0) |
+----------------------------------+-----------------+--------------------------+
| Anguilla
| Caribbean
|
8000 |
| Montserrat
| Caribbean
|
11000 |
| Turks and Caicos Islands
| Caribbean
|
17000 |
| Virgin Islands, British
| Caribbean
|
21000 |
| Saint Kitts and Nevis
| Caribbean
|
38000 |
| Antigua and Barbuda
| Caribbean
|
68000 |
| Dominica
| Caribbean
|
71000 |
| Virgin Islands, U.S.
| Caribbean
|
93000 |
| Grenada
| Caribbean
|
94000 |
| Aruba
| Caribbean
|
103000 |
| Saint Vincent and the Grenadines | Caribbean
|
114000 |
| Saint Lucia
| Caribbean
|
154000 |
| Netherlands Antilles
| Caribbean
|
217000 |
| Barbados
| Caribbean
|
270000 |
| Bahamas
| Caribbean
|
307000 |
| Martinique
| Caribbean
|
395000 |
| Guadeloupe
| Caribbean
|
456000 |
| Trinidad and Tobago
| Caribbean
|
1295000 |
| Jamaica
| Caribbean
|
2583000 |
| Puerto Rico
| Caribbean
|
3869000 |
| Haiti
| Caribbean
|
8222000 |
| Dominican Republic
| Caribbean
|
8495000 |
| Cuba
| Caribbean
|
11201000 |
| Cuba
| Caribbean
|
1589167 |
| Belize
| Central America |
241000 |
| Panama
| Central America |
2856000 |
| Costa Rica
| Central America |
4023000 |
| Nicaragua
| Central America |
5074000 |
| El Salvador
| Central America |
6276000 |
| Honduras
| Central America |
6485000 |
| Guatemala
| Central America |
11385000 |
| Mexico
| Central America |
98881000 |
| Mexico
| Central America |
16902625 |
| Saint Pierre and Miquelon
| North America
|
7000 |
| Greenland
| North America
|
56000 |
| Bermuda
| North America
|
65000 |
| Canada
| North America
|
31147000 |
| United States
| North America
|
278357000 |
| United States
| North America
|
61926400 |
| United States
|
|
13053865 |
+----------------------------------+-----------------+--------------------------+
40 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

The resulting output includes subtotals (The 2nd listing for Cuba, Mexico and United States are the
subtotals for the region) along with a total average for all the countries in North America (the last row in
the set). This of course is the simplest way to create a calculated report, but it can be difficult to decipher
(especially with large data sets).
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-4

MySQL Developer Techniques

Using UNION
The UNION syntax in SQL is used to combine the result from multiple SELECT statements into a single
result set. With this syntax, the following SQL statement creates a similar response to what was seen
with the WITH ROLLUP option:

279

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

280

Thi

Chapter 10:Creating Reports

mysql> SELECT Name, Region, Population, Population FROM Country


-> WHERE Continent='North America'
-> UNION
-> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country
-> WHERE Continent='North America'
->
GROUP BY Region ORDER BY Region;
+----------------------------------+-----------------+------------+------------+
| Name
| Region
| Population | Population |
+----------------------------------+-----------------+------------+------------+
| Grenada
| Caribbean
|
94000 |
94000 |
| Puerto Rico
| Caribbean
|
3869000 |
3869000 |
| Aruba
| Caribbean
|
103000 |
103000 |
| Saint Vincent and the Grenadines | Caribbean
|
114000 |
114000 |
| Jamaica
| Caribbean
|
2583000 |
2583000 |
| Turks and Caicos Islands
| Caribbean
|
17000 |
17000 |
| Cayman Islands
| Caribbean
|
38000 |
38000 |
| Martinique
| Caribbean
|
395000 |
395000 |
| NULL
| Caribbean
|
NULL |
1589167 |
| Dominican Republic
| Caribbean
|
8495000 |
8495000 |
| Anguilla
| Caribbean
|
8000 |
8000 |
| Saint Kitts and Nevis
| Caribbean
|
38000 |
38000 |
| Bahamas
| Caribbean
|
307000 |
307000 |
| Haiti
| Caribbean
|
8222000 |
8222000 |
| Antigua and Barbuda
| Caribbean
|
68000 |
68000 |
| Guadeloupe
| Caribbean
|
456000 |
456000 |
| Saint Lucia
| Caribbean
|
154000 |
154000 |
| Barbados
| Caribbean
|
270000 |
270000 |
| Trinidad and Tobago
| Caribbean
|
1295000 |
1295000 |
| Virgin Islands, British
| Caribbean
|
21000 |
21000 |
| Cuba
| Caribbean
|
11201000 |
11201000 |
| Virgin Islands, U.S.
| Caribbean
|
93000 |
93000 |
| Dominica
| Caribbean
|
71000 |
71000 |
| Montserrat
| Caribbean
|
11000 |
11000 |
| Netherlands Antilles
| Caribbean
|
217000 |
217000 |
| Guatemala
| Central America |
11385000 |
11385000 |
| Belize
| Central America |
241000 |
241000 |
| Nicaragua
| Central America |
5074000 |
5074000 |
| Costa Rica
| Central America |
4023000 |
4023000 |
| Mexico
| Central America |
98881000 |
98881000 |
| NULL
| Central America |
NULL |
16902625 |
| El Salvador
| Central America |
6276000 |
6276000 |
| Panama
| Central America |
2856000 |
2856000 |
| Honduras
| Central America |
6485000 |
6485000 |
| Greenland
| North America
|
56000 |
56000 |
| Saint Pierre and Miquelon
| North America
|
7000 |
7000 |
| Bermuda
| North America
|
65000 |
65000 |
| Canada
| North America
|
31147000 |
31147000 |
| United States
| North America
| 278357000 | 278357000 |
| NULL
| North America
|
NULL |
61926400 |
+----------------------------------+-----------------+------------+------------+
40 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

This SQL produces the subtotals that were found in the earlier SQL using the WITH ROLLUP clause,
but this time it is clear which one is the subtotal row based on the NULL value in the Name and first
Population column.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-5

MySQL Developer Techniques


281

Chapter 10:Creating Reports

Improving UNION

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Now of course, the location of the subtotal row is in the middle of most of the groups versus at the
bottom where they should be. This can be corrected by using the COALESCE function with a number
greater than any of the populations to move the NULL field to the bottom of the groups as shown below:

Thi

282

mysql> SELECT Name, Region, Population, Population FROM Country


-> WHERE Continent='North America'
-> UNION
-> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country
-> WHERE Continent='North America'
->
GROUP BY Region ORDER BY Region, COALESCE(Population,6000000000);
+----------------------------------+-----------------+------------+------------+
| Name
| Region
| Population | Population |
+----------------------------------+-----------------+------------+------------+
| Anguilla
| Caribbean
|
8000 |
8000 |
| Montserrat
| Caribbean
|
11000 |
11000 |
| Turks and Caicos Islands
| Caribbean
|
17000 |
17000 |
| Virgin Islands, British
| Caribbean
|
21000 |
21000 |
| Cayman Islands
| Caribbean
|
38000 |
38000 |
| Saint Kitts and Nevis
| Caribbean
|
38000 |
38000 |
| Antigua and Barbuda
| Caribbean
|
68000 |
68000 |
| Dominica
| Caribbean
|
71000 |
71000 |
| Virgin Islands, U.S.
| Caribbean
|
93000 |
93000 |
| Grenada
| Caribbean
|
94000 |
94000 |
| Aruba
| Caribbean
|
103000 |
103000 |
| Saint Vincent and the Grenadines | Caribbean
|
114000 |
114000 |
| Saint Lucia
| Caribbean
|
154000 |
154000 |
| Netherlands Antilles
| Caribbean
|
217000 |
217000 |
| Barbados
| Caribbean
|
270000 |
270000 |
| Bahamas
| Caribbean
|
307000 |
307000 |
| Martinique
| Caribbean
|
395000 |
395000 |
| Guadeloupe
| Caribbean
|
456000 |
456000 |
| Trinidad and Tobago
| Caribbean
|
1295000 |
1295000 |
| Jamaica
| Caribbean
|
2583000 |
2583000 |
| Puerto Rico
| Caribbean
|
3869000 |
3869000 |
| Haiti
| Caribbean
|
8222000 |
8222000 |
| Dominican Republic
| Caribbean
|
8495000 |
8495000 |
| Cuba
| Caribbean
|
11201000 |
11201000 |
| NULL
| Caribbean
|
NULL |
1589167 |
| Belize
| Central America |
241000 |
241000 |
| Panama
| Central America |
2856000 |
2856000 |
| Costa Rica
| Central America |
4023000 |
4023000 |
| Nicaragua
| Central America |
5074000 |
5074000 |
| El Salvador
| Central America |
6276000 |
6276000 |
| Honduras
| Central America |
6485000 |
6485000 |
| Guatemala
| Central America |
11385000 |
11385000 |
| Mexico
| Central America |
98881000 |
98881000 |
| NULL
| Central America |
NULL |
16902625 |
| Saint Pierre and Miquelon
| North America
|
7000 |
7000 |
| Greenland
| North America
|
56000 |
56000 |
| Bermuda
| North America
|
65000 |
65000 |
| Canada
| North America
|
31147000 |
31147000 |
| United States
| North America
| 278357000 | 278357000 |
| NULL
| North America
|
NULL |
61926400 |
+----------------------------------+-----------------+------------+------------+
40 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-6

MySQL Developer Techniques


283

Chapter 10:Creating Reports

What about the final total

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Even though it may be slightly easier to read, the fact that there is no final total (like with the WITH
ROLLUP clause) may prevent using this method over the built-in function. However, with a little bit
more creative SQL, that can easily be corrected.

Thi

284

mysql> SELECT Name, Region, Population, Population FROM Country


-> WHERE Continent='North America'
-> UNION
-> SELECT NULL, Region, NULL, ROUND(AVG(Population),0) FROM Country
-> WHERE Continent='North America'
->
GROUP BY Region
-> UNION
-> SELECT NULL, 'AVG Total', NULL, ROUND(AVG(Population),0) FROM Country
-> WHERE Continent='North America'
->
ORDER BY Region, COALESCE(Population,6000000000);
+----------------------------------+-----------------+------------+------------+
| Name
| Region
| Population | Population |
+----------------------------------+-----------------+------------+------------+
| Anguilla
| Caribbean
|
8000 |
8000 |
| Montserrat
| Caribbean
|
11000 |
11000 |
| Turks and Caicos Islands
| Caribbean
|
17000 |
17000 |
| Virgin Islands, British
| Caribbean
|
21000 |
21000 |
| Cayman Islands
| Caribbean
|
38000 |
38000 |
| Saint Kitts and Nevis
| Caribbean
|
38000 |
38000 |
| Antigua and Barbuda
| Caribbean
|
68000 |
68000 |
| Dominica
| Caribbean
|
71000 |
71000 |
| Virgin Islands, U.S.
| Caribbean
|
93000 |
93000 |
...
| Jamaica
| Caribbean
|
2583000 |
2583000 |
| Puerto Rico
| Caribbean
|
3869000 |
3869000 |
| Haiti
| Caribbean
|
8222000 |
8222000 |
| Dominican Republic
| Caribbean
|
8495000 |
8495000 |
| Cuba
| Caribbean
|
11201000 |
11201000 |
| NULL
| Caribbean
|
NULL |
1589167 |
| Belize
| Central America |
241000 |
241000 |
| Panama
| Central America |
2856000 |
2856000 |
| Costa Rica
| Central America |
4023000 |
4023000 |
| Nicaragua
| Central America |
5074000 |
5074000 |
| El Salvador
| Central America |
6276000 |
6276000 |
| Honduras
| Central America |
6485000 |
6485000 |
| Guatemala
| Central America |
11385000 |
11385000 |
| Mexico
| Central America |
98881000 |
98881000 |
| NULL
| Central America |
NULL |
16902625 |
| Saint Pierre and Miquelon
| North America
|
7000 |
7000 |
| Greenland
| North America
|
56000 |
56000 |
| Bermuda
| North America
|
65000 |
65000 |
| Canada
| North America
|
31147000 |
31147000 |
| United States
| North America
| 278357000 | 278357000 |
| NULL
| North America
|
NULL |
61926400 |
| NULL
| AVG Total
|
NULL |
13053865 |
+----------------------------------+-----------------+------------+------------+
41 rows in set (0.00 sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

With the addition of another UNION, the total row is created and the result produced is similar to the
WITH ROLLUP clause but slightly easier to read.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-7

MySQL Developer Techniques


285

Chapter 10:Creating Reports

Avoiding all confusion

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

With a little bit more creative SQL, this calculated report idea can be taken to the next level in
readability. By adding two additional columns for subtotals and the grand total, the readability of the
report can be improved.

Thi

286

mysql> SELECT Name, Region, Population, Subtotal, Total FROM (


->
SELECT Name, Region, Population, '' AS Subtotal, '' Total
->
FROM Country WHERE Continent='North America'
->
UNION
->
SELECT NULL, Region, NULL, ROUND(AVG(Population),0), ''
->
FROM Country WHERE Continent='North America'
->
GROUP BY Region
->
UNION
->
SELECT 'AVG Total',NULL, NULL, '', ROUND(AVG(Population),0)
->
FROM Country WHERE Continent='North America'
-> ) AS Country_t ORDER BY Total, Region, COALESCE(Population,6000000000);
+----------------------------------+-----------------+------------+----------+----------+
| Name
| Region
| Population | Subtotal | Total
|
+----------------------------------+-----------------+------------+----------+----------+
| Anguilla
| Caribbean
|
8000 |
|
|
| Montserrat
| Caribbean
|
11000 |
|
|
| Turks and Caicos Islands
| Caribbean
|
17000 |
|
|
| Virgin Islands, British
| Caribbean
|
21000 |
|
|
| Saint Kitts and Nevis
| Caribbean
|
38000 |
|
|
| Cayman Islands
| Caribbean
|
38000 |
|
|
| Antigua and Barbuda
| Caribbean
|
68000 |
|
|
| Dominica
| Caribbean
|
71000 |
|
|
| Virgin Islands, U.S.
| Caribbean
|
93000 |
|
|
...
| Martinique
| Caribbean
|
395000 |
|
|
| Guadeloupe
| Caribbean
|
456000 |
|
|
| Trinidad and Tobago
| Caribbean
|
1295000 |
|
|
| Jamaica
| Caribbean
|
2583000 |
|
|
| Puerto Rico
| Caribbean
|
3869000 |
|
|
| Haiti
| Caribbean
|
8222000 |
|
|
| Dominican Republic
| Caribbean
|
8495000 |
|
|
| Cuba
| Caribbean
|
11201000 |
|
|
| NULL
| Caribbean
|
NULL | 1589167 |
|
| Belize
| Central America |
241000 |
|
|
| Panama
| Central America |
2856000 |
|
|
| Costa Rica
| Central America |
4023000 |
|
|
| Nicaragua
| Central America |
5074000 |
|
|
| El Salvador
| Central America |
6276000 |
|
|
| Honduras
| Central America |
6485000 |
|
|
| Guatemala
| Central America |
11385000 |
|
|
| Mexico
| Central America |
98881000 |
|
|
| NULL
| Central America |
NULL | 16902625 |
|
| Saint Pierre and Miquelon
| North America
|
7000 |
|
|
| Greenland
| North America
|
56000 |
|
|
| Bermuda
| North America
|
65000 |
|
|
| Canada
| North America
|
31147000 |
|
|
| United States
| North America
| 278357000 |
|
|
| NULL
| North America
|
NULL | 61926400 |
|
| AVG Total
| NULL
|
NULL |
| 13053865 |
+----------------------------------+-----------------+------------+----------+----------+
41 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

With specifically placed NULL's and spaces in the SQL, the calculated chart can be created in such a way
to bring out the data in a easy to read format all in one SQL statement.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-8

MySQL Developer Techniques

10.4.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

287

Thi

Chapter 10:Creating Reports

Quarterly Reports
A common report concerning dates revolves around the idea of producing data for quarters. In many
business environments, the quarters are connected to the quarters of the calendar year; January through
March constitutes the first quarter, April through May constitutes the second quarter and so on.
Typically, data is not stored on a quarterly or even monthly basis; rather, business events are recorded
on a daily basis, storing the full date (sometimes including even the time of day) when the event
occurred. For example, sales of items in a supermarket are typically recorded by storing the product
code and time of day.
SaleDate

SalePrice

SaleCost

2007-10-15
2007-11-07

$23,000.00
$17,450.00

$14,300.00
$14,465.00

2007-12-14
2008-01-06
2008-02-22

$21,456.00
$14,525.00
$16,395.00

$16,540.00
$8,780.00
$10,700.00

2008-03-17
2008-04-29

$27,560.00
$18,750.00

$22,420.00
$14,500.00

2008-05-12
2008-05-22

$21,500.00
$19,600.00

$17,300.00
$12,400.00

bl
a
r
fe

an
r
t
n

no
a
s
a
2008-06-13
$14,100.00
$9,500.00
h
n) ide
v

The chart above (a used car sales report) represents a used car
and illustrates
what the underlying
msales
u
odata,
G
c
data for a quarterly report might look like. In the sample
there
are dates for only three quarters

t
n December, the first quarter of
21 October
represented; the last quarter of 2007 which includes
through
e
s
d
t
2008 which includes January through March
and the second
tu quarter of 2008 which includes April to
stopgroup
S
June. The following SQL could be used
the
data
into a quarterly report:
@
s
i
n
h
a
t
r
t asu'Sales
se Year',
n
mysql> SELECT YEAR(SaleDate)
o
o
to SaleDate, NULL)) THEN SalePrice END) AS Q1,
-> SUM(IF(QUARTER(SaleDate)=1,
m
(
e
-> SUM(IF(QUARTER(SaleDate)=2,
SaleDate, NULL)) THEN SalePrice END) AS Q2,
an icens
r
-> SUM(IF(QUARTER(SaleDate)=3,
SaleDate, NULL)) THEN SalePrice END) AS Q3,
T
l
t
->
SUM(IF(QUARTER(SaleDate)=4,
SaleDate, NULL)) THEN SalePrice END) AS Q4
e-> FROM carsales
y
u
g -> GROUP BY 'Sales Year';

This SQL would produce a chart similar to the one below that shows the total sales (sale price) for
each quarter of the year the sale took place. NULL's are placed in those columns where there is no data
for that quarter of the year.
+------------+-------+-------+------+-------+
| Sales Year | Q1
| Q2
| Q3
| Q4
|
+------------+-------+-------+------+-------+
|
2007 | NULL | NULL | NULL | 61906 |
|
2008 | 58480 | 73950 | NULL | NULL |
+------------+-------+-------+------+-------+
2 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-9

MySQL Developer Techniques

Chapter 10:Creating Reports

10.4.1. Cross Tab Reports

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

288

Thi

Even though the output of the quarterly report presented earlier is acceptable, there may be times that
the data itself may need to be presented to highlight attention to the quarters (versus the years). The
following SQL statement provides quarterly data with the quarters highlighted:
mysql> SELECT QUARTER(SaleDate) AS Quarter,
-> SUM(IF(YEAR(SaleDate)=2007, SalePrice, NULL)) AS '2007',
-> SUM(IF(YEAR(SaleDate)=2008, SalePrice, NULL)) AS '2008'
-> FROM carsales GROUP BY Quarter;
+---------+-------+-------+
| Quarter | 2007 | 2008 |
+---------+-------+-------+
|
1 | NULL | 58480 |
|
2 | NULL | 73950 |
|
4 | 61906 | NULL |
+---------+-------+-------+
3 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

This method of creating a cross tab report of this nature is more hard coded (having to actually enter in
the years) than the standard quarterly reports but can add value depending on the requirements of the
end user. In addition, for those quarters where no data was found (such as Q3), the standard quarterly
report still reflects that information versus this type of approach that did not.

no
a
s
a
h
Column totals
289
n)yearlyidtotals
e can be calculated to
v

By providing the WITH ROLLUP clause to the end of the last


query,
om t Gu
go along with the quarters displayed:
c

21 den
s
t
p Stu
mysql> SELECT QUARTER(SaleDate) AS s
Quarter,
-> SUM(IF(YEAR(SaleDate)=2007,
SalePrice,
AS '2007',
@
is NULL))
n
h
-> SUM(IF(YEAR(SaleDate)=2008,
SalePrice,
NULL)) AS '2008'
a
t
r
e WITH ROLLUP;
t BY Quarter
-> FROM carsales GROUP
s
n
u
o
+---------+-------+--------+
o
to
m
| Quarter | 2007
| 2008 e |
(
+---------+-------+--------+
ns |
an ic| e58480
r
|
1 | NULL
T
t 2 | NULLl | 73950 |
| ye
u
|
NULL |
Ng|| NULL4 || 61906
61906 | 132430 |
+---------+-------+--------+
4 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-10

MySQL Developer Techniques

10.5.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

290

Thi

Chapter 10:Creating Reports

SQL Bar Chart


Sometimes, information can be better displayed using a bar chart instead of a listing of the raw data.
For example, the following displays the population values for the regions in the African continent::

mysql> SELECT Region, SUM(Population)


-> FROM Country
-> WHERE Continent='Africa'
-> GROUP BY Region;
+-----------------+-----------------+
| Region
| SUM(Population) |
+-----------------+-----------------+
| Central Africa |
95652000 |
| Eastern Africa |
246999000 |
| Northern Africa |
173266000 |
| Southern Africa |
46886000 |
| Western Africa |
221672000 |
+-----------------+-----------------+
5 rows in set (#.## sec)

291

bl
a
r
fe

an
r
t
n

The following SQL statement would convert these values into a bar chart for a visual representation of
the data:

no
a
mysql> SELECT Region, REPEAT('#',(SUM(Population)/5000000)) ASs'Population Chart'
a
-> FROM Country
h
)
-> WHERE Continent='Africa'
n ide
v

-> GROUP BY Region;


om t Gu
+-----------------+---------------------------------------------------+
c

| Region
| Population Chart
|
21 den
s
+-----------------+---------------------------------------------------+
t
p Stu
| Central Africa | ###################
|
s
@
s
| Eastern Africa | #################################################
|
i
n
h
a
t
| Northern Africa | ###################################
|
r
t
e

s
n
| Southern Africa | #########
|
o to u
| Western Africa |o############################################
|
(m nse
+-----------------+---------------------------------------------------+
n(#.##
5 rows in rset
sec)
a
e
c
T
i
l
et
y
u
g This of course does not replace the more detailed graphs that can be created in other tools; however,

for a quick representation of the data within MySQL itself it can be very useful.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-11

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-A

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this hands on lab, you will create a cross tab report for car sales that focuses on the actual days of
the week that the cars were sold.

Thi

Lab Preparation
To perform the following lab a CarSales table needs to be created in the test database. The
following SQL statement builds the structure for the CarSales table:
mysql>
->
->
->
->

CREATE TABLE CarSales (


SaleDate date
NOT NULL DEFAULT '0000-00-00',
SalePrice decimal(8,2) NOT NULL DEFAULT 0.0,
SaleCost decimal(8,2) NOT NULL DEFAULT 0.0
);

bl
a
r
fe

an
r
t
n

no
a
s
a
h

n) VALUES
mysql> INSERT INTO CarSales (SaleDate, SalePrice, SaleCost)
e
v

d
i
-> ('2007-10-15',23000.00,14300.00),
om t Gu
c
-> ('2007-11-07',17450.00,14465.00),

n
-> ('2007-12-14',21456.00,16540.00), 21
e
s
d
t
-> ('2008-01-06',14525.00,8780.00),
sp s Stu
-> ('2008-02-22',16395.00,10700.00),
@
i
n
-> ('2008-03-17',27560.00,22420.00),
h
a
t
r
t
-> ('2008-04-29',18750.00,14500.00),
e
n
us
-> ('2008-05-12',21500.00,17300.00),
o
o
o
t
-> ('2008-05-22',19600.00,12400.00),
(m nse
-> ('2008-06-13',14100.00,9500.00);
n
a
r
ce
T
i
l
t
1. eSales per week, per day
guy1. It is fairly common that sales are not evenly distributed over all days of the week. Create a query to report
The following SQL statement inserts the data into the CarSales table that will be used in the
following lab:

the number of cars sold as a crosstab, having one row for each year and one column for each weekday.
Hint: Examine the quarterly CarSales report shown previously. Use the WEEKDAY function to obtain the
day for a particular date.
2. The resultant table should resemble the following MySQL output:
+------+------+------+------+------+------+------+------+
| Year | Mon | Tue | Wed | Thu | Fri | Sat | Sun |
+------+------+------+------+------+------+------+------+
| 2007 |
1 |
0 |
1 |
0 |
1 |
0 |
0 |
| 2008 |
2 |
1 |
0 |
1 |
2 |
0 |
1 |
+------+------+------+------+------+------+------+------+

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-12

MySQL Developer Techniques

10.6.

Decision Tables
Decision tables are used to lay out in tabular form all possible situations which a business decision
may encounter and to specify which action to take in each of these situations. The following
demonstrates a decision table for a world wide business focused on selling financial products.

292

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Chapter 10:Creating Reports

Rules

Conditions

Actions

Avg. Life Expectancy >= 65

GNP per citizen >= 25M

GNP per citizen >= 250K

Offer Retirement Options

Offer Life Insurance

Offer Banking Services

bl
a
r
fe

ns
a
r
t
- 65 years
n
Average life expectancy - If the average life expectancy is equal to or greater
than
o
n rules.
of age for a given country, then the financial advisor will review the firstathree
s
a per citizen is equal to or
GNP per citizen (>= 25M) - If the gross national product (GNP)
h
)
the financial officer
n idthen
greater than 25,000,000 dollars (or an equivalent dollarvexchange),
e

will review the first and fourth rule to determinem


products
u should be offered to the
o bywhich
G
c
customer. The GNP per citizen can be determined
dividing
the country's GNP by the total

t
21 den
population of that country.
s
t
p theSfacttuthat the majority of countries have a rather
GNP per citizen (>= 250K)
-sDue to s
@
i must review the first, second, fourth and fifth
nthe financialthadvisor
small GNP per citizen,
a
r
t
e
per citizenusis greater than (or equal to) 250,000 dollars (or an equivalent
rules if the GNP
onand
dollar exchange)
than 25,000,000 dollars. This time the financial advisor will only
o
ocondition
tless
e
need (
tom
review the
first
based on the fact that the second condition could not apply.
n ens
a
r
Based
T on whichlicconditions apply to the country in question, the financial advisor has up to three
t
e
products
that should be recommended to the customer:
uy

293

N
Thi

There are three conditions that the financial advisor will use to determine which products they can
offer to citizens of a country:

Retirement options - These would include packages that are most appropriate where
countries with an average life span equal to or greater than 65 years of age. For countries
with an earlier average life span, the potential for customers living to retirement is not
conducive for retirement products.

Life insurance - These would include packages that are most appropriate in countries where
each individual has a higher possibility (based on the GNP per citizen) of having the resources
to purchase (or the need to purchase) life insurance.

Banking services - These would include packages where the citizens may have some
additional funds that can be used for saving and investing (based on the GNP per citizen). In
countries where the GNP per citizen is less than 250,000 dollars (or an equivalent dollar
exchange), the likelihood of the citizens having funds beyond everyday living for savings or
investment purposes would be improbable.

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-13

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

294

Thi

Implementing with SQL


Now that the decision table has been made, it is time to implement the conditions and actions into SQL
statements. The first step is to work through the conditions:
mysql>
->
->
->

295

Chapter 10:Creating Reports

SELECT Name, IF(LifeExpectancy >= 65,


'Rules 1 thru 3',
'Rules 4 thru 6'
) AS Action FROM Country;

This SELECT statement tests the first condition determining if the average life expectancy is greater
then or equal to 65. If the condition is met, the text will display that 'Rules 1 thru 3' have been met,
and if not, the text will display 'Rules 4 thru 6'.
mysql> SELECT Name, IF(LifeExpectancy >= 65,
-> IF((GNP*10000000000/Population) >= 25000000,
->
'Offer Retirement, Insurance and Banking',
->
'Rules 2 thru 3'
->
),
-> IF((GNP*10000000000/Population) >= 25000000,
->
'Offer Insurance and Banking',
->
'Rules 5 thru 6'
->
)
->) AS Action FROM Country;

bl
a
r
fe

an
r
t
n

no
a
s
a
h
) statement
which handled
nSELECT
This SELECT statement adds the second condition to the original
e
v

d
i
the first condition. These nested IF functions, if met will m
displayu'Offer Retirement, Insurance
oeither
G is greater then or equal to
c

and Banking' (which is rule 1) if the average life expectancy


for
the
country
t
296
1
n
2
e life expectancy for the country is
65 or 'Offer Insurance and Banking' (which is
4) if thedaverage
ptsrule
tu are not met for the first nested IF or
less then 65. 'Rule 2 thru 3' will be displayed
if theS
conditions
s
@met forththeissecond IF function.
'Rule 5 thru 6' if the conditions are
not
n
a
r
t use >= 65,
n
mysql> SELECT Name, IF(LifeExpectancy
o
o
to
-> IF((GNP*10000000000/Population)
>= 25000000,
m
(
e
s
->
'Offer
Retirement,
Insurance
and
Banking',
n
a icen
-> TrIF((GNP*10000000000/Population)
l Insurance and Banking',>= 250000,
->t
'Offer
e
'Offer Insurance'
uy ->
g
->
)
N
->
),
-> IF((GNP*10000000000/Population) >= 25000000,
->
'Offer Insurance and Banking',
->
IF((GNP*10000000000/Population) >= 250000,
->
'Offer Banking',
->
'No offer suggested'
->
)
->
)
-> ) AS Action FROM Country;

This SELECT is the final statement that takes into account all possible conditions for the decision table
and produces an output that would provide the financial advisor with the options they should suggest to
any potential customers based on the country they are citzens of.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-14

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

297

Thi

Chapter 10:Creating Reports

Here is an example of the script executed with two different countries: Monaco and Haiti.
mysql> SELECT Name, IF(LifeExpectancy >= 65,
->
IF((GNP*10000000000/Population) >= 25000000,
->
'Offer Retirement, Insurance and Banking',
->
IF((GNP*10000000000/Population) >= 250000,
->
'Offer Insurance and Banking',
->
'Offer Insurance'
->
)
->
),
->
IF((GNP*10000000000/Population) >= 25000000,
->
'Offer Insurance and Banking',
->
IF((GNP*10000000000/Population) >= 250000,
->
'Offer Banking',
->
'No offer suggested'
->
)
->
)
->
) AS Action FROM Country WHERE Name = 'Monaco';
+--------+-----------------------------------------+
| Name
| Action
|
+--------+-----------------------------------------+
| Monaco | Offer Retirement, Insurance and Banking |
+--------+-----------------------------------------+
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
mysql> SELECT Name, IF(LifeExpectancy >= 65,
v

->
IF((GNP*10000000000/Population) >= 25000000,
om t Gu
c
->
'Offer Retirement, Insurance and Banking',

1>= 250000,
n
->
IF((GNP*10000000000/Population)
2
e
s
d
t
->
'Offer Insurance and Banking',
sp s Stu
->
'Offer Insurance'
@
i
n
->
)
h
a
t
r
t
->
),
e
n
us
->
IF((GNP*10000000000/Population)
>= 25000000,
o
o
o
t
->
'Offer
Insurance
and
Banking',
m
->
IF((GNP*10000000000/Population)
>= 250000,
se
n ( eBanking',
n
-> Tra 'Offer
c
i
->t
'Nol offer suggested'
e
y
->
)
gu -> )

bl
a
r
fe

->
) AS Action FROM Country WHERE Name = 'Haiti';
+-------+---------------+
| Name | Action
|
+-------+---------------+
| Haiti | Offer Banking |
+-------+---------------+
1 row in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-15

MySQL Developer Techniques

10.7.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

298

Chapter 10:Creating Reports

Materialized Views
MySQL does not have materialized views; however, many of the features of materialized views can be
completed with summary tables. The term "summary tables" can be misleading. Summary tables, in
relation to the discussion here, includes those tables that summarize data from other sources (most likely
other tables). The following is an example of query that could be considered a summary table:

mysql>
->
->
->
->

CREATE TABLE MV_Country AS


SELECT Country.Region, MAX(Country.LifeExpectancy) AS MaxLifeExpectancy,
COUNT(DISTINCT CountryLanguage.Language) AS LanguagesSpoken
FROM Country,CountryLanguage WHERE Country.Code=CountryLanguage.CountryCode
GROUP BY Region;

mysql> SELECT Region, MaxLifeExpectancy, LanguagesSpoken FROM MV_Country;

299 +---------------------------+-------------------+-----------------+
| Region
| MaxLifeExpectancy | LanguagesSpoken |
+---------------------------+-------------------+-----------------+
| Australia and New Zealand |
79.8 |
11 |
| Baltic Countries
|
69.5 |
8 |
| British Islands
|
77.7 |
4 |
...
| Southern and Central Asia |
71.8 |
54 |
| Southern Europe
|
83.5 |
22 |
| Western Africa
|
76.8 |
65 |
| Western Europe
|
79.6 |
21 |
+---------------------------+-------------------+-----------------+
24 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

1 for theenaverage life expectancy by region, while


In this example, the world.Country table is 2
squeried
dfor the number of languages spoken in each
t
the joined world.CountryLanguage p
table
is queried
tuquery
s
S
region. The data has been summarized
and
from
this
more specific data can be retrieved. By
@
s
i
n
h
having a summary table of thea
data,
accessing
this
summary
data
can cost less performance wise when
tr se t

user require access to thisndata.


o to u
Different viewsm
of o
the same data
300
seusing a join query, end users may need to use the data in different ways. One
n (summarized
With theadata
n
r
e
c
Tuser may beliconcerned
end
with the average life span and will wish to sort the data showing the shortest
t
e
life span countries first.
of course, the query can be changed to sort on the
y
u AvgLifeExpectancy column,Now
g
but that could cause problems for another user that is more concerned
with the number of languages spoken. The answer to this dilemma is to give each user a table of the
hi N
summarized data that they can perform more specific queries on.
Updating tables
A dilemma associated with "summary tables" is how to keep them accurate and up to date; if that is
necessary. An answer to this dilemma is the use of triggers. When data that makes up the summary table
is updated or changed, triggers can be set up to catch these events and create a new table for the users to
access. Triggers can have an effect on overall performance of the queries that update or alter the tables
which must be taken into account when setting up such processes. Other less performance heavy options
may include nightly batch processes that include deleting and recreating the tables based on the most
current data at that time (if up to date data is not absolutely necessary).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-16

MySQL Developer Techniques

10.8.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

301

Thi

Chapter 10:Creating Reports

Producing Sequential or Missing Data


One of the ways to produce sequential or missing data is by utilizing an integer table. An integer table
is a single column table that contains a sequential list of numbers. The syntax could simply be as
follows for creating the table:

mysql> CREATE TABLE int_table (int_col INT PRIMARY KEY NOT NULL AUTO_INCREMENT);

Filling the table with data


Once the table is created, it is necessary to fill the table with data. The number of rows that should be
inserted, should be much larger than the number of rows in the largest table that is located in the
database that this table will support. For example, in the world database, the table with the greatest
number of rows is the city table with approximately 4,000 records. So in this case, the integer table
should contain somewhere around 8,000 records to ensure that there is room for the city table to
grow. Now of course, it is possible to create the max number of integer rows into the table
(4,294,967,295 unsigned); however, if none of the tables in the database will ever reach that number
there is no reason to waste even this small amount of space with unneeded rows. It would be quite
easy to add additional rows at a later date.
302

bl
a
r
fe

an
r
t
n

Exponential growth - One of the ways to fill the integer table with values is to use the table
itself in an exponential fashion to grow itself. The following SQL shows how this would
work:

no
a
s
a
h
mysql> INSERT INTO int_table VALUES (NULL);
n) ide
v

om t Gu
mysql> SELECT int_col FROM int_table;
c

+---------+
21 den
s
t
| int_col |
sp s Stu
+---------+
@
i
|
1 |
n
h
a
t
r
+---------+
t
e
1 row in set (#.## sec)
on to us
o
(m nse
n
Adding
a NULL
value into the int_col column will result in the next sequential number being
a
r
e
c
T
i
l
added
to
the
table.
In this case, it was the first record created and thus the integer 1 was
t
e
y
added.
gu

mysql> INSERT INTO int_table SELECT NULL FROM int_table;


mysql> INSERT INTO int_table SELECT NULL FROM int_table;
mysql> INSERT INTO int_table SELECT NULL FROM int_table;
mysql> SELECT * FROM int_table;
+---------+
| int_col |
+---------+
|
1 |
...
|
8 |
+---------+
8 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-17

MySQL Developer Techniques

Chapter 10:Creating Reports

By executing the INSERT statement above that inserts a NULL value a number of times
equivalent to the rows that are located in the int_table, the number of records in the int_table
grows exponentially.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

303

Thi

Stored procedure - Another way to fill the integer table is to create a stored procedure. In
the following stored procedure, the user enters the number of records they wish to have added
to the int_table:

mysql> DELIMITER //
mysql> CREATE PROCEDURE fill_int (int_max INT)
-> BEGIN
->
DECLARE x INT DEFAULT 0;
->
WHILE x < int_max DO
->
INSERT INTO int_table VALUE (NULL);
->
SET x = x + 1;
->
END WHILE;
->
SELECT COUNT(*) FROM int_table;
-> END //
Query OK, 0 rows affected (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
mysql> CALL fill_int(8000);
a
h
+----------+
n) ide
| COUNT(*) |
v

+----------+
om t Gu
c
|
8000 |

21 den
+----------+
s
t
1 row in set (#.# sec)
sp s Stu
Query OK, 0 rows affected (#.# sec)
@
i
n
h
a
t
r
t
e
s input provided (in this case 8,000) and executes an INSERT
The stored procedure,
uthe
on ttakes
o
o
statement
against e
the int_table that number of times. After the WHILE loop is completed, the
(m
s
n
stored
procedure
performs a SELECT query against the table to count how many rows the
n
a
r table contains.
ce This process allows for more control over how many rows are added, but the
T
i
l
t
e
cost in time (for the procedure to run) is greatly increased.
guy
mysql> DELIMITER ;

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-18

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-B

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you create an integer table and then use that table to produce non-integer sequential
outputs.

Thi

1.

1. Create and load the integer table


Using the world database, create the following table:

mysql> CREATE TABLE int_table (int_col INT PRIMARY KEY NOT NULL);

2.

This will create the integer table that will be used throughout the remainder of this lab.
Insert the values 0 through 9 into the int_table using the following SQL:

INSERT INTO int_table VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);

This will populate the integer table with the numbers 0 through 9 for use in the views to be created.
1.

2. Create the English alphabet view


Create the following view that will list the 26 letters of the English alphabet:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e

us
onthat
This will create o
a view
will
display the 26 letters of the English alphabet.
o
t
(m
senames by first letter
3. Count
the country
n
n
a
r
e
T the following
1. Perform
lic SQL query to count the names of the countries (using the first letter of the country
t
e
name):
guy
CREATE VIEW abc(int_col) AS
SELECT
SUBSTRING('ABCDEFGHIJKLMNOPQRSTUVWXYZ'
FROM 10*tbl1.int_col+tbl2.int_col FOR 1)
AS Alphabet
FROM int_table AS tbl1
CROSS JOIN int_table AS tbl2
WHERE 10*tbl1.int_col+tbl2.int_col
BETWEEN 1 AND 26
ORDER BY Alphabet;

mysql> SELECT LEFT(Name,1) AS Alpha,


-> COUNT(LEFT(Name,1)) AS Total FROM
-> Country GROUP BY LEFT(Name,1);

Each letter of the English alphabet is represented by the country names with the exception of the letter X:
+-------+-------+
| Alpha | Total |
+-------+-------+
| A
|
15 |
...
| W
|
2 |
| Y
|
2 |
| Z
|
2 |
+-------+-------+
25 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-19

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

2.

Thi

Chapter 10:Creating Reports

Perform the following SQL query to have every letter of the English alphabet displayed (including X)
with the total number of countries that start with that letter:

SELECT abc.int_col AS Alpha,


COUNT(LEFT(Name,1)) AS Total FROM abc
LEFT OUTER JOIN
Country ON abc.int_col = LEFT(Name,1)
GROUP BY abc.int_col;

Each letter of the English alphabet is represented by the country names including the letter X:
+-------+-------+
| Alpha | Total |
+-------+-------+
| A
|
15 |
| B
|
20 |
| C
|
22 |
...
| V
|
5 |
| W
|
2 |
| X
|
0 |
| Y
|
2 |
| Z
|
2 |
+-------+-------+
26 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-20

MySQL Developer Techniques

Chapter 10:Creating Reports

10.8.1. Using the integers table to fill in missing data

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

304

Thi

Now that the integers table has been created, it is time to demonstrate how this can be used to fill in
missing data for another table. In the following example, a copy of the world.city table has been
created (city2) with records 6 through 9 deleted:
mysql> SELECT * FROM city2 LIMIT 10;
+----+----------------+-------------+---------------+------------+
| ID | Name
| CountryCode | District
| Population |
+----+----------------+-------------+---------------+------------+
| 1 | Kabul
| AFG
| Kabol
|
1780000 |
| 2 | Qandahar
| AFG
| Qandahar
|
237500 |
| 3 | Herat
| AFG
| Herat
|
186800 |
| 4 | Mazar-e-Sharif | AFG
| Balkh
|
127800 |
| 5 | Amsterdam
| NLD
| Noord-Holland |
731200 |
| 10 | Tilburg
| NLD
| Noord-Brabant |
193238 |
| 11 | Groningen
| NLD
| Groningen
|
172701 |
| 12 | Breda
| NLD
| Noord-Brabant |
160398 |
| 13 | Apeldoorn
| NLD
| Gelderland
|
153491 |
| 14 | Nijmegen
| NLD
| Gelderland
|
152463 |
+----+----------------+-------------+---------------+------------+
10 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
Using the integer table (in this case, the 100k view created earlier will be used),
s the following OUTER
a
305
h
JOIN query will fill in the missing integers:
n) ide
v

mysql> SELECT 100k.int_col AS ID, city2.Name, city2.CountryCode


om t Gu FROM 100k
c

-> LEFT OUTER JOIN


21 den
->
city2 ON 100k.int_col = city2.ID
s
t
-> LIMIT 1,10
sp s Stu
+----+----------------+-------------+---------------+------------+
@
i
n | tDistrict
h
| ID | Name
| CountryCode
| Population |
a
r
t
e

+----+----------------+-------------+---------------+------------+
o|nAFG to us | Kabol
| 1 | Kabul
|
1780000 |
o
| 2 | Qandahar(m
| AFG
|
Qandahar
|
237500 |
e
n
| 3 | Herat
|sAFG
| Herat
|
186800 |
n
a
r
e
| 4 | Mazar-e-Sharif
| AFG
| Balkh
|
127800 |
c
T
i
l
| 5e|t Amsterdam
| NLD
| Noord-Holland |
731200 |
u| y6 | NULL
| NULL
| NULL
|
NULL |
g
| NULL
| NULL
|
NULL |
N | 7 | NULL
| 8 | NULL
| NULL
| NULL
|
NULL |
| 9 | NULL
| NULL
| NULL
|
NULL |
| 10 | Tilburg
| NLD
| Noord-Brabant |
193238 |
+----+----------------+-------------+---------------+------------+
10 rows in set (#.## sec)

In this OUTER JOIN query, the int_col column from the 100k view takes the place of the ID
column of the city2 table along with displaying the remaining columns of the city2 table (Name,
CountryCode).

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-21

MySQL Developer Techniques

Chapter 10:Creating Reports

Lab 7-C

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

In this lab you will use the knowledge learned up to this point to produce cross tab reports that take
advantage of the integer table idea presented earlier along with creating a SQL bar chart to represent
the data queried.

Thi

1.

2.

1. Independent Governments of the second half of the 20th century


Make a report that produces one row for each year of the last 50 years of the previous century. Make a
comma separated list of all countries that became independent in that particular year, and also list the total
number of countries.
Hint: Use an integer table to deal with missing years.
The resultant table should resemble the following MySQL output:

+------+---------------+-----------------------------+
| year | num_countries | GROUP_CONCAT(country.name) |
+------+---------------+-----------------------------+
| 1951 |
2 | Libyan Arab Jamahiriya,Oman |
| 1952 |
0 | NULL
|
| 1953 |
2 | Laos,Cambodia
|
| 1954 |
0 | NULL
|
| 1955 |
1 | Germany
|
| 1956 |
3 | Sudan,Tunisia,Morocco
|
| 1957 |
2 | Ghana,Malaysia
|
...
| 1998 |
0 | NULL
|
| 1999 |
0 | NULL
|
| 2000 |
0 | NULL
|
+------+---------------+-----------------------------+
50 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
3. Create an SQL bar chart
s of countries that became independent in the last 50 years of the
onto tlistotheunumber
previous century.
o
(mtable nshould
4. The resultant
se resemble the following MySQL output:
n
a
r
e
T
lic
t
+------+--------------------+
e
yyear | num_countries
|
gu|+------+--------------------+

| 1951 | **
|
| 1952 |
|
| 1953 | **
|
| 1954 |
|
| 1955 | *
|
| 1956 | ***
|
...
| 1990 | ***
|
| 1991 | ****************** |
| 1992 | *
|
| 1993 | ***
|
| 1994 | *
|
...
| 2000 |
|
+------+--------------------+
50 rows in set (#.## sec)

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-22

MySQL Developer Techniques

10.9.

Chapter 10:Creating Reports

Summary
In this chapter, you have learned to:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

306

Thi

Use functions to calculate multiple conditions

Create a report with values and subtotals for each group along with final rollup

Create aggregate reports against dates in the data

Create cross tab reports

Create a bar chart with SQL

Create a decision table to avoid hard wiring of logic in the application code

Create summary tables with temporary tables

Produce sequential or missing data

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________

10-23

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

CHAPTER 11

an
r
t
n

CONCLUSIONhas a n
n) ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Chapter 11: Conclusion

11 CONCLUSION

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

11.1 Course Objectives

Thi

308

This course was designed for experienced database developers who wished to learn additional skills required to
create complex queries and efficient structures while improving the performance of the database by learning to
create and write queries that are optimized based on the data in the database.
Now that you have completed this course, you should now be able to:

Utilize the multiple indexing options available to create better response times of query executions

Optimize queries that are responsible for searching text fields

Optimize queries that search date fields

Optimize queries that perform calculations and aggregations

Create summary tables that utilize triggers, event scheduler and table merges

Optimize queries that retrieves data from more than one table

bl
a
r
fe

an
r
t
n

309

o
n
a
Create tables that utilize tree and hierarchical structures
s
a
h
Utilize temporary tables to improve query performance n)
e
v

d
i
om t Gu
Explain the affect locking levels have on querycperformance

n
21 dprecise
e
Create report queries that provide p
thets
end userstwith
and organized data
u
s
S
@ this
n
a
r
t use
n
o
o
to
m
(
e
an icens
r
T
l
et

uy
g
N

11-1

MySQL Developer Techniques

Chapter 11: Conclusion

11.2 Training and Certification Website


View the course catalog, schedule, certification program information, venue information (and more) on the
Training and Certification website: http://www.mysql.com/training/ .

310

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

24H

bl
a
r
fe

Thi

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Topic
Menu

an
r
t
n

uy
g
N

11-2

MySQL Developer Techniques

Chapter 11: Conclusion

Overview The home page for Training information. Links to all other training information.
Certification Description of current certification program. Links to exam registration and more
information.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Curriculum Paths The categories and order in which training courses should be taken.

Thi

Catalog Comprehensive course catalog. Links to more information on courses.


Schedule Schedule of courses, listed by title. Links to course registration and more information.
Delivery Options The available options for training delivery.
Savings and Promotions Information about training bundles and credits.
Training Bundles Available bundles and pricing.
Training Partners List and links of MySQL training partners.
Testimonials Training endorsements.

FAQ Frequently asked questions and links to more other training information.

bl
a
r
fe

an
r
t
n

Additional training information, including Articles, White Papers and Web Seminars, can be found on the
Community web page: http://dev.mysql.com/ .

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

11-3

MySQL Developer Techniques

Chapter 11: Conclusion

11.3 Course Evaluation


We Need Your Evaluation!

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

311

Thi

Please take the time to complete a course evaluation to let us know about your experience. We are continuously
making improvements to our courses and our overall training program. And we count on the feedback of our
students to assist in making the right choices.
Please follow the instructions given by your instructor regarding completion of the evaluation form.

Thank you in advance for taking the time to give us your opinions!

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

11-4

MySQL Developer Techniques

Chapter 11: Conclusion

11.4 THANK YOU!


We appreciate your attendance at this course. Congratulations on your completion. We hope that your training
needs regarding this course topic have been fully met. Dont hesitate to contact us, if you have any training
questions:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

312

Thi

Website: http://www.mysql.com/training/
Email: training@mysql.com
Phone: USA Toll Free: 1-866-697-7522
Worldwide: 1-208-514-4780

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

11-5

MySQL Developer Techniques

Chapter 11: Conclusion

11.5 Q&A Session


313

You may record any questions and answers that are shared at this point in the class, in the space provided below.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

___________________________________________________________________________________________

Thi

___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________

bl
a
r
___________________________________________________________________________________________
fe
s
n
___________________________________________________________________________________________
tra
n
no
___________________________________________________________________________________________
a
s
a
h
___________________________________________________________________________________________
n) ide
v

om t Gu
___________________________________________________________________________________________
c

1
n
2
e
s
d
t
___________________________________________________________________________________________
sp s Stu
@
i
n
h
___________________________________________________________________________________________
a
t
r
t
e
n
us
o
o
o
___________________________________________________________________________________________
t
(m nse
n
a
r
___________________________________________________________________________________________
ce
T
i
l
t
ye
gu___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
__________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________

11-6

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

APPENDIX A

bl
a
r
fe

an
r
t
n

uy
g
N

an
INSTALLATION
s
a
h
)
n ide
v

PROCEDURES
om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Appendix A: Installation Procedures

Appendix A - Installation Procedures


A.1

MySQL Server installation on Microsoft Windows

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

For the purposes of this course, you will download the 5.1 Generally Available release, Essentials Version:

Thi

1.

Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2.

Find section entitled: MySQL Community Server

3.

Find the section called Upcoming Releases and click the link for: MySQL 5.1

4.

Find the Essentials version of the appropriate platform and click on the link called Pick a mirror

5.

The registration at the top of the page is optional, so to skip it select the link at the bottom of the page that reads
No thanks, just take me to the downloads!. You will not perform registration during the course.

6.

Scroll down the page to the section showing the Mirrors in: <country/city>. Find the location closest to you
and click on the link next to it called HTTP:

7.

A window will pop up with the option to Run or Save.

8.

Select Run to download the software for installation. (If a window regarding security appears, click on Run to
allow the download.)

an
r
t
n

no
a
s
10. Select a Typical installation.
a
h
) be as follows:
The wizard is ready to install and indicates that the install directoryn
will
e
v

d
i
C:\Program Files\MySQL\MySQL Server 5.1
om t Gu
c

n
11. Start the install by clicking on the Install button.21
e
s
d
t
u will pop up.
After the files are installed, the MySQL.com
twindow
sp Sign-Up
S
@
s
i
12. Select Skip Sign-Up, and click onn
the Next> h
button.
a
t
r
t
e
13. In the Wizard Completed
make
s sure that check box labeled Configure the MySQL Server now is
u
on window,
selected. Then, clickoon the Finish
button.
o
t
m Instance
(Server
e
s
The MySQL
Configuration
wizard will appear.
n
a icen
r
T
l button to perform the configuration.
14. Click
t on the Next>
e
y
gu15. Select Detailed Configuration, then click the Next> button.
9.

bl
a
r
fe

The Windows Setup wizard will appear. Click on the Next> button to perform the set-up.

16. Select Developer Machine, then click the Next> button.

17. Select Multifunctional Database, then click the Next> button.


18. Do not change any InnoDB Tablespace Settings at this time, and click the Next> button.
19. Select Decision Support, then click the Next> button.
20. Select Enable TCP/IP Networking and Enable Strict Mode, then click the Next> button.
21. Select Standard Character Set, then click the Next> button.
22. Select Install As Windows Service and Include Bin Directory in Windows PATH, then click the Next>
button.
23. Select Modify Security Settings, then enter root as the new password. Then click the Next> button.
Use of an anonymous account is not recommended, due to the lack of security.
24. Click on the Execute button.
The wizard now shows that the configuration file was created successfully.
25. Click on the Finish button to complete.

A-1

MySQL Developer Techniques

A.2

Appendix A: Installation Procedures

MySQL Server installation on Linux

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

For the purposes of this course, you will download the 5.1 Release Candidate release, Essentials Version:

Thi

1.

Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2.

Find section entitled: MySQL Community Server

3.

Find the section called Upcoming Releases and click the link for: MySQL 5.1

4.

Download the latest binary tarball:


(eg. mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz)

5.

Unpack the tarball to /usr/local with (change /tmp/ to the proper path):

cd /usr/local

tar zxvf /tmp/mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz

6.

Add a symbolic link to the directory:

an
r
t
n

an
s
a
h
7. Install the system tables:
)
n ide
v

cd mysql
om t Gu
c

21 den
s
scripts/mysql_install_db
t
sp s Stu
@
i
n
8. Add a mysql system user and a
group:
h
t
r
t
e
on to us
groupadd mysql
o
se
n (m
n
useradd gra
mysql
mysql
e
T
lic
t
e
gu9.y Change the owner of the data directory:
ln s mysql-5.1.26-rc-linux-i686-icc-glibc23.tar.gz mysql

bl
a
r
fe

chown R mysql:mysql data

10. Copy the startup script to /etc/init.d:


cp support-files/mysql.server /etc/init.d/mysql

11. Start the script:


/etc/init.d/mysql start

12. Edit the bashrc file and add this path:


PATH=${PATH}:/usr/local/mysql/bin

A-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

A.3

Thi

Appendix A: Installation Procedures

MySQL 'world' database file installation


1.

Go to the MySQL web site Documentation page: http://dev.mysql.com/doc/

2.

Find section entitled: Example Databases

3.

Select the most appropriate method for download of the world database:
Note: For the purposes of this course, you will not need to download the setup guide.

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

A-3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

APPENDIX B

bl
a
r
fe

an
r
t
n

uy
g
N

MYISAM has a n
n) ide
v

LOCKING
om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Appendix B: MyISAM Locking

Appendix B - MyISAM Locking

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Table lock types


The MyISAM storage engine protects against corruption of data by different threads updating in conflicting ways by
using table-level locking. The MyISAM storage engine utilizes one of three types of locks depending on the request
issued by the connecting thread:
READ LOCAL SHARED

READ SHARED

WRITE EXCLUSIVE

Allows an INSERT to append only to


the end of the data file. If the INSERT
requires a push of data anywhere else
but to the end of the data file
(beginning or middle of the data file),
the INSERT statement is delayed
until the lock is released.

Activated when the actual data file


(*.MYD), rather than the in-memory
cache, is used to get information.
When this lock is activated, all
UPDATE, INSERT and DELETE
statements are delayed until the lock
is released.

This type of lock is activated whenever


an UPDATE or DELETE request is
submitted, or an INSERT is received
that would require adding the data
anywhere but to the end of the data file.
This lock prevents any activity on the
table until the lock is released.

ns
a
r
t
-are running against
With the MyISAM storage engine's capability to perform INSERT operations while SELECT queries
n
o
n that are responsible for
the data (through the READ LOCAL lock type), it makes an excellent choice for applications
aquery
s
logging continual activities while also giving end users the ability to run reports or
other
on the data
a will always operations
h
contained. This is made possible by the fact that MyISAM knows that all new )records
occur at the end of the
else in the table.
ndata from
eanywhere
v
data file and thus there is no need to hold up SELECT statements that request

d
i
om t Gu
c

Concurrent
n
21 Inserts
e
s
d
t
p SELECT
tustatements to use the READ LOCAL lock regardless if
With the MyISAM storage engine, it is possible tosforce
S
@
s
there is room in the middle of the data file
for the INSERT
to fill. Whether an INSERT will be a concurrent insert is
anstatus
thi This
r
determined by the concurrent-inserttglobal
variable.
variable can be assigned by setting the mysqld -e
s
u
onGLOBAL
concurrent_insert or o
SET
concurrent_insert parameter.
to
m
(
e
s take any one of the following values:
The concurrent_insert
ncan
an parameter
r
e
0 - (Off) This
tells the server
to
not
allow any concurrent inserts.
c
T
li
t
e
1 - (Default)
Enables
concurrent
insert
for MyISAM tables that don't have gaps.
y
g
2u
- Enables concurrent inserts for all MyISAM tables. If the table has a hole and is in use by another thread the new
hi Nrow will be inserted at the end of the table. If the table is not in use, MySQL obtains a normal WRITE lock and inserts
Tables designed to Log Activity

bl
a
r
fe

the new row into the hole.

A-1

MySQL Developer Techniques

Appendix B: MyISAM Locking

Priorities

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

When running a SQL statement that interacts with a MyISAM table, an individual user can determine the importance
level that their individual statements have in regards to priority execution on the MySQL server. There are four levels of
importance in relation to database actions:

Thi

Highest priority

Normal writes

Normal reads

Lowest priority

The database actions that this


level
of
priority
(or
importance) include SELECT
or INSERT commands that
the user or application
interfacing with the server
has identified as needing to
run before any other SQL
statements waiting to be
executed by the server.

If there are no high


priority SQL statements
waiting to be executed,
the MySQL server will
allow INSERT, DELETE
and UPDATE commands
to run first prior to any
SELECT statements that
are waiting to be executed
by the server.

As stated, SELECT
statements are at
the bottom of the
execution
list
(unless identified
as high priority)
when waiting for
execution by the
server.

The database actions at this level of


priority will wait for all other SQL
statements to execute prior to
actually executing themselves. The
end user of the application
interfacing with the server must
specifically tell the server that these
SQL statements are considered low
priority otherwise the normal writes
and
reads
order
will
be
implemented.

an
r
t
n

no
a
s to move to the top of the list of
HIGH_PRIORITY This command gives the SQL statement being executed the ability
a
h
any waiting SQL statements (from all threads) and can be used with the following
SQL
n) idstatements:
e
v

u HIGH_PRIORITY
INSERT HIGH_PRIORITY
om t GSELECT
c

1 the INSERT
n MySQL gives the SELECT a higher
2gives
e
MySQL stops any concurrent insert capabilities and
s
d
t
thasu executed priority than a statement that updates a
statement complete control of the table until s
thepstatement
S
@
s
completely.
table. This should only be used for
i
n
h
a
t
r
queries that are very fast and must be
t
e

> Priority levels


s
n
done
at
once.
A
SELECT
u
o
o
o
> When using the INSERT
HIGH_PRIORITY
statement, the server HIGH_PRIORITY query that is issued
t
m
automatically overrides
se low_priority_updates setting. while the table is locked for reading
n ( anyeexisting
n
a
r
The low_priority_updates
setting, when set to 1, forces all INSERT, runs even if there is an update statement
T
ic TABLE
lLOCK
t
e
UPDATE,
DELETE,
and
statements to wait until there waiting for the table to be free.
uypending SELECT or LOCK TABLEWRITE
is
no
READ on the affected table. This HIGH_PRIORITY cannot be used with
g
N
The following is a list of the MySQL priority commands that can be used:

variable previously was named sql_low_priority_updates. In


addition, by default writing queries have a higher priority than reading
queries, so by default there are two priority queues.

bl
a
r
fe

SELECT statements that are part of a


UNION.

A-2

MySQL Developer Techniques

Appendix B: MyISAM Locking

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

LOW_PRIORITY This command places the SQL statement being executed at the bottom of the list of any waiting
and new SQL statements (from all threads) and will only run when all waiting and new SQL statements have completed.
This command can be used with the following SQL statements:

Thi

INSERT LOW_PRIORITY

UPDATE LOW_PRIORITY

DELETE LOW_PRIORITY

MySQL delays the execution of the statement


(and stops any other work from the particular
client) until no other clients are reading from the
table. This includes any other clients that may
access the table while the INSERT
LOW_PRIORITY statement is waiting to execute.
In an environment where there is a large number
of clients that can be reading from the server at
any one time this could cause the client to wait a
very long time. This should normally not be used
with MyISAM tables because doing so disables
concurrent inserts.

Works identical to the


INSERT
LOW_PRIORITY
statement and should also be
avoided unless absolutely
necessary.

Also works the same as the


INSERT LOW_PRIORITY by
not executing until all clients are
done reading from the table. This
can cause integrity issues in that
another client can be viewing data
that
should
be
deleted
immediately.

bl
a
r
fe

ns
a
r
t
- queue forever, if
Using LOW_PRIORITY with UPDATEs, INSERTs and DELETEs can cause the queries to wait innthe
o
there is no time of day when clients aren't reading from the table.
an
s
a
Miscellaneous
h
)
n ide is not directly related to
v

INSERT DELAYED - The use of the INSERT DELAYED statement


in SQL statements
om tinGtheulocking section. When the INSERT
locking issues in MyISAM, but has a bearing on many of thec
discussions
n into a buffer, and the client issuing the
DELAYED statement is executed, the server puts the row2
or1
rows to be
inserted
e
s
d
t
p SIftuthe table is in use, the server holds the rows. When the
INSERT DELAYED statement can then continues
immediately.
table is free, the server begins inserting rows,
checking iperiodically
to see whether there are any new read requests for
@
s
n
h
a
t
the table. If there are, the delayed row
queue
is
suspended
until
the
table
becomes free again. Another major benefit of
r
e clients are bundled together
t usmany
n
using INSERT DELAYED is o
that
inserts from
and written in one block. This is much
o
o
t
faster than performing many
separate
inserts.
INSERT
DELAYED
is
slower
than
a normal INSERT if the table is not
(m
e
otherwise in use. There
is also n
thesadditional overhead for the server to handle a separate thread for each table for which
n
arows. Inicaddition,
r
e if the server crashes before the INSERT's have been actually inserted, they will be
there are delayed
T
l
t
lost. y
u Thise means that INSERT DELAYED should be used only when it is absolutely necessary.
g
N

A-3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

APPENDIX C

bl
a
r
fe

an
r
t
n

uy
g
N

an
QUIZ SOLUTIONS
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Appendix C: Quiz Solutions

Appendix C - Quiz Solutions


Quiz 2-A Solutions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

1. SELECT d FROM t1 WHERE d = 10


mysql> EXPLAIN SELECT d FROM t1 WHERE d = 10\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: d
key: d
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

2. SELECT a FROM t1 WHERE a = 10

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
3. SELECT r
c FROM t1eWHERE c = 10
T
lic
t
e
y EXPLAIN SELECT c FROM t1 WHERE c = 10\G
umysql>
g
***************************
1. row ***************************
id: 1
hi N
mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

bl
a
r
fe

select_type: SIMPLE
table: t1
type: index
possible_keys: NULL
key: a
key_len: 15
ref: NULL
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

C-1

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4.

Appendix C: Quiz Solutions

SELECT d FROM t1 WHERE d = 10 ORDER BY a


mysql> EXPLAIN SELECT d FROM t1 WHERE d = 10 ORDER BY a\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: d
key: d
key_len: 5
ref: const
rows: 1
Extra: Using where; Using filesort
1 row in set (#.## sec)

5.

SELECT a FROM t1 WHERE a = 10 ORDER BY b


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 ORDER BY b\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
6. SELECT a FROM t1 WHEREtraa= 10 ANDth
b = 5
e

us
oanFROM
o
mysql> EXPLAIN SELECT
t1 WHERE a = 10 AND b = 5\G
o
t
(m nse 1. row ***************************
***************************
n
id:
1
a
r SIMPLE
ce
T
i
select_type:
l
t
t1
uye table:
type: ref
g
hi N possible_keys: a

bl
a
r
fe

key: a
key_len: 10
ref: const,const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

C-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

7.

Appendix C: Quiz Solutions

SELECT a FROM t1 WHERE a = 10 AND c = 17


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND c = 17\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

8.

SELECT a FROM t1 WHERE a = 10 AND d = 5


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND d = 5\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a,d
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
9. SELECT a FROM t1 WHEREtraa= 1 AND tbh = 1 AND c = 5
e

us
oanFROM
o
mysql> EXPLAIN SELECT
t1 WHERE a = 1 AND b = 1 AND c = 5\G
o
t
(m nse 1. row ***************************
***************************
n
id:
1
a
r SIMPLE
ce
T
i
select_type:
l
t
t1
uye table:
type: ref
g
hi N possible_keys: a

bl
a
r
fe

key: a
key_len: 15
ref: const,const,const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

C-3

MySQL Developer Techniques

Appendix C: Quiz Solutions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

10. SELECT a FROM t1 WHERE a = 1 AND b = 1 AND c = 5 AND d = 1


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 1 AND b = 1 AND c = 5 AND d = 1\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a,d
key: a
key_len: 15
ref: const,const,const
rows: 1
Extra: Using where
1 row in set (#.## sec)

11. SELECT a FROM t1 WHERE a = 10 GROUP BY b


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 GROUP BY b\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n a = 10thGROUP
a
12. SELECT MAX(c) FROM t1 tWHERE
BY b
r
e

s
n
u t1 WHERE a = 10 GROUP BY b\G
oMAX(c)toFROM
o
mysql> EXPLAIN SELECT
m
***************************
n 1( ense 1. row ***************************
id:
a
r
T
select_type:
SIMPLE
lic
t
e
table:
t1
uy type: ref
g
hi N possible_keys: a

bl
a
r
fe

key: a
key_len: 5
ref: const
rows: 1
Extra: Using where; Using index
1 row in set (#.## sec)

C-4

MySQL Developer Techniques

Appendix C: Quiz Solutions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

13. SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b


mysql> EXPLAIN SELECT a FROM t1 WHERE a = 10 AND d = 5 ORDER BY b\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ref
possible_keys: a,d
key: a
key_len: 5
ref: const
rows: 1
Extra: Using where
1 row in set (#.## sec)

14. SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b


mysql> EXPLAIN SELECT a FROM t1 WHERE a BETWEEN 1 AND 10 ORDER BY b\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: index
possible_keys: a
key: a
key_len: 15
ref: NULL
rows: 1
Extra: Using where; Using index; Using filesort
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
15. SELECT a FROM t1 WHEREtraa= 10 ANDth
b < 3 ORDER BY c
e

us
oanFROM
o
mysql> EXPLAIN SELECT
t1 WHERE a = 10 AND b < 3 ORDER BY c\G
o
t
(m nse 1. row ***************************
***************************
n
id:
1
a
r SIMPLE
ce
T
i
select_type:
l
t
t1
uye table:
type: ref
g
hi N possible_keys: a

bl
a
r
fe

key: a
key_len: 5
ref: const
rows: 10
Extra: Using where; Using index; Using filesort
1 row in set (#.## sec)

C-5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

APPENDIX D

bl
a
r
fe

an
r
t
n

uy
g
N

no
a
FURTHER PRACTICE
s
a
h
n) ide
v

SOLUTIONS
om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

uy
g
N
an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
an
r
t
n
bl
a
r
fe

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Appendix D: Further Practice Solutions


Further Practice Lab 2-C

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

1.

Thi

If necessary, install a fresh copy of the world database to remove any changes that may have
been made to the data or the data structures.

mysql> DROP DATABASE IF EXISTS world;


mysql> CREATE DATABASE world;
mysql> USE world;
mysql> SOURCE C:\world.sql

The location of the world.sql file may be different than the one displayed, please change it
accordingly.
2. Create three new indexes in the world.Country table:
a.

A unique index named uName that indexes the Name column.

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

m the Continent
u
b. A composite index named Cont_Pop that indexes
and Population
o
G
c

t
columns together.
1
n
s2 tude
t
p
s (Continent,
mysql> CREATE INDEX Cont_Pop ON Country
Population);
S
@
s
Query OK, 239 rows affected (#.##
sec)
i
n
th0
Records: 239 Duplicates:t0raWarnings:
e
us that indexes the Capital column.
onnamedtouCapital
o
c. A unique
index
(m nse
n
a
r INDEX
euCapital ON Country (Capital);
mysql> CREATE
c
T
i
l
t
Query
239 rows affected (#.## sec)
ye OK,239
Duplicates: 0 Warnings: 0
guRecords:
mysql> CREATE INDEX uname ON Country (Name);
Query OK, 239 rows affected (#.## sec)
Records: 239 Duplicates: 0 Warnings: 0

3.

Create a new index in the world.City table against the CountryCode column.

mysql> CREATE INDEX uCountryCode ON City (CountryCode);


Query OK, 4079 rows affected (#.## sec)
Records: 4079 Duplicates: 0 Warnings: 0

D-1

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4.

Thi

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the
following SQL statement:

mysql> EXPLAIN SELECT DISTINCT Country.Region FROM Country


-> INNER JOIN CountryLanguage ON CountryLanguage.CountryCode = Country.Code
->
WHERE Language = 'Spanish'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ALL
possible_keys: PRIMARY
key: NULL
key_len: NULL
ref: NULL
rows: 239
Extra: Using temporary
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: CountryLanguage
type: eq_ref
possible_keys: PRIMARY,CountryCode
key: PRIMARY
key_len: 33
ref: world.Country.Code,const
rows: 1
Extra: Using where; Using index; Distinct
2 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

D-2

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

5.

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the
following SQL statement:

mysql> EXPLAIN SELECT Name, Continent FROM Country WHERE Population > 200000000\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 239
Extra: Using where
1 row in set (#.## sec)

bl
a
r
fe

Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer
would consider using now:

ns
a
r
t
-200000000\G
mysql> EXPLAIN SELECT DISTINCT Continent FROM Country WHERE Population >
n
o
*************************** 1. row ***************************
an
id: 1
s
select_type: SIMPLE
a
h
table: Country
)
n ide
v
type: index

possible_keys: NULL
om t Gu
c

key: Cont_Pop
21 den
key_len: 5
s
t
ref: NULL
sp s Stu
rows: 239
@
n indexthi
Extra: Using where;ra
Using
t
1 row in set (#.## sec)n
se
u
o
o
to
m
(
e
an icens
r
T
l
t
e
y
u

N
Thi

D-3

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

6.

Thi

Appendix D: Further Practice Solutions

Determine which indexes (if any) the MySQL query optimizer would consider using in the
following SQL statement:

mysql> EXPLAIN SELECT CountryCode, MAX(Population) FROM City GROUP BY CountryCode\G


*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using temporary; Using filesort
1 row in set (#.## sec)

bl
a
r
fe

Modifying the above SQL statement, determine which indexes (if any) the MySQL query optimizer
would consider using now:

mysql> EXPLAIN SELECT CountryCode, MAX(Population) FROM City


-> WHERE CountryCode LIKE 'C%' GROUP BY CountryCode\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: range
possible_keys: CountryCode,uCountryCode
key: CountryCode
key_len: 3
ref: NULL
rows: 538
Extra: Using where
1 row in set (#.## sec)

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

D-4

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice 3-C


1.

Install a copy of the sakila database into a database with the same name.

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> DROP DATABASE IF EXISTS sakila;

Thi

mysql> CREATE DATABASE sakila;


mysql> USE sakila;
mysql> SOURCE C:\sakila.sql

2.

The location of the sakila.sql file may different than the one displayed, please change it
accordingly.
Using the sakila database, review the structure of the customer, film, inventory and
rental tables.

an
r
t
n

mysql> SHOW COLUMNS FROM customer;


+-------------+------------------+------+-----+---------+----------------+
| Field
| Type
| Null | Key | Default | Extra
|
+-------------+------------------+------+-----+---------+----------------+
| customer_id | int(10) unsigned | NO
| PRI | NULL
| auto_increment |
| store_id
| int(10) unsigned | NO
| MUL |
|
|
| last_name
| varchar(45)
| NO
|
|
|
|
| first_name | varchar(45)
| NO
|
|
|
|
| address
| varchar(255)
| NO
|
|
|
|
| address2
| varchar(255)
| YES |
| NULL
|
|
| city
| varchar(60)
| NO
|
|
|
|
| district
| varchar(60)
| NO
|
|
|
|
| country
| varchar(60)
| NO
|
|
|
|
| phone
| varchar(50)
| NO
|
|
|
|
| postal_code | varchar(20)
| NO
|
|
|
|
| active
| tinyint(1)
| NO
|
| 1
|
|
+-------------+------------------+------+-----+---------+----------------+
12 rows in set (#.## sec)

bl
a
r
fe

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

y SHOW COLUMNS FROM film;


umysql>
g
+------------------+---------------------+------+-----+---------+----------------+
N
| Field
| Type
| Null | Key | Default | Extra
|
+------------------+---------------------+------+-----+---------+----------------+
| film_id
| int(10) unsigned
| NO
| PRI | NULL
| auto_increment |
| category_id
| int(10) unsigned
| NO
|
|
|
|
| title
| varchar(255)
| NO
| MUL |
|
|
| description
| text
| YES |
| NULL
|
|
| rental_duration | tinyint(3) unsigned | NO
|
| 3
|
|
| rental_rate
| decimal(4,2)
| NO
|
| 4.99
|
|
| length
| int(10) unsigned
| YES |
| NULL
|
|
| replacement_cost | decimal(5,2)
| NO
|
|
|
|
| rating
| enum('G','PG', ... | YES |
| G
|
|
+------------------+---------------------+------+-----+---------+----------------+
9 rows in set (#.## sec)
mysql> SHOW COLUMNS FROM inventory;
+--------------+------------------+------+-----+---------+----------------+
| Field
| Type
| Null | Key | Default | Extra
|
+--------------+------------------+------+-----+---------+----------------+
| inventory_id | int(10) unsigned | NO
| PRI | NULL
| auto_increment |
| film_id
| int(10) unsigned | NO
| MUL |
|
|
| store_id
| int(10) unsigned | NO
| MUL |
|
|
+--------------+------------------+------+-----+---------+----------------+
3 rows in set (#.## sec)

D-5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

Appendix D: Further Practice Solutions

mysql> SHOW COLUMNS FROM rental;


+--------------+------------------+------+-----+---------+-------+
| Field
| Type
| Null | Key | Default | Extra |
+--------------+------------------+------+-----+---------+-------+
| rent_date
| datetime
| NO
| PRI |
|
|
| inventory_id | int(10) unsigned | NO
| PRI |
|
|
| customer_id | int(10) unsigned | NO
| PRI |
|
|
| return_date | datetime
| YES |
| NULL
|
|
| staff_id
| int(10) unsigned | NO
| MUL |
|
|
+--------------+------------------+------+-----+---------+-------+
5 rows in set (#.## sec)

3.

Using the rental table, determine the maximum rental days where a customer checked out a
video and returned it. In addition, include the average length of days that customers (who have
returned the videos) have kept the videos.

mysql> SELECT MAX(DATEDIFF(return_date,rent_date)) AS Max_Days,


->
AVG(DATEDIFF(return_date,rent_date)) AS Avg_Days
->
FROM rental WHERE return_date IS NOT NULL;
+----------+----------+
| Max_Days | Avg_Days |
+----------+----------+
|
10 |
5.0263 |
+----------+----------+
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

D-6

MySQL Developer Techniques

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

4.

Thi

Appendix D: Further Practice Solutions

Create a report that identifies videos that are currently out. Include in the report the customers name
and contact info, the video that they have checked out and the number of days they have had the movie
checked out (from the current date). Note: Due to this data being slightly outdated, the days checked
out will be quite lengthy.

mysql> SELECT customer.first_name, customer.last_name, customer.phone, film.title,


-> DATEDIFF(NOW(),rental.rent_date) AS days_checked_out
->
FROM customer,film,inventory,rental
->
WHERE customer.customer_id = rental.customer_id
->
AND rental.inventory_id = inventory.inventory_id
->
AND inventory.film_id = film.film_id
->
AND return_date IS NULL LIMIT 20;
+------------+------------+--------------+---------------------+------------------+
| first_name | last_name | phone
| title
| days_checked_out |
+------------+------------+--------------+---------------------+------------------+
| BRANDON
| HUEY
| 99883471275 | ACE GOLDFINGER
|
1160 |
| CARMEN
| OWENS
| 272234298332 | AFFAIR PREJUDICE
|
1156 |
| SETH
| HANNON
| 864392582257 | AFRICAN EGG
|
1156 |
| TRACY
| COLE
| 371490777743 | ALI FOREVER
|
1157 |
| MARCIA
| DEAN
| 727785483194 | ALONE TRIP
|
1160 |
| CECIL
| VINES
| 879347453467 | AMADEUS HOLY
|
1160 |
| MARIE
| TURNER
| 177727722820 | AMERICAN CIRCUS
|
1160 |
| JOE
| GILLILAND | 53912826864 | AMISTAD MIDSUMMER
|
1157 |
| EDWARD
| BAUGH
| 46568045367 | ARMAGEDDON LOST
|
1161 |
| BETH
| FRANKLIN
| 470884141195 | BAKED CLEOPATRA
|
1159 |
| GINA
| WILLIAMSON | 584316724815 | BANG KWAI
|
1158 |
| MELVIN
| ELLINGTON | 368284120423 | BASIC EASY
|
1156 |
| SONIA
| GREGORY
| 195003555232 | BERETS AGENT
|
1162 |
| FLORENCE
| WOODS
| 330838016880 | BLADE POLISH
|
1156 |
| TYLER
| WREN
| 211256301880 | BLANKET BEVERLY
|
1160 |
| MILDRED
| BAILEY
| 250767749542 | BOOGIE AMELIE
|
1161 |
| ANNA
| HILL
| 911872220378 | BOULEVARD MOB
|
1161 |
| ROBIN
| HAYES
| 942570536750 | BOUND CHEAPER
|
1157 |
| LOIS
| BUTLER
| 345679835036 | BUBBLE GROSSE
|
1160 |
| LISA
| ANDERSON
| 635297277345 | BULL SHAWSHANK
|
1162 |
...
| BECKY
| MILES
| 648482415405 | VIRGINIAN PLUTO
|
1158 |
| JENNY
| CASTRO
| 62781725285 | VOLCANO TEXAS
|
1156 |
| GREG
| ROBINS
| 206060652238 | WANDA CHAMBER
|
1157 |
| REGINA
| BERRY
| 201705577290 | WEDDING APOLLO
|
1156 |
| NAOMI
| JENNINGS
| 959949395183 | WILD APOLLO
|
1159 |
| JEREMY
| HURTADO
| 600264533987 | WINDOW SIDE
|
1156 |
| NATALIE
| MEYER
| 873492228462 | WOMEN DORADO
|
1162 |
| NEIL
| RENNER
| 478380208348 | WORLD LEATHERNECKS |
1158 |
| LOUIS
| LEONE
| 45554316010 | ZHIVAGO CORE
|
1162 |
+------------+-------------+--------------+--------------------+------------------+
184 rows in set (#.## sec)

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

Your output may be slightly different than the output displayed here due to the use of the NOW() function which
will affect the output based on when this statement is run.

D-7

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice Lab 5-A Solution

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Using the divide and conquer technique calculate the total tax, the tax on revenue % and the total % of
the salaries that the employees get to keep when we have the following list of employees:

Thi

Name

Salary

Johan

30,000

Max

55,000

Tobias

90,000

mysql> CREATE TABLE taxrate (


-> base DECIMAL(8,2) NOT NULL,
-> threshold DECIMAL (10,2) NOT NULL,
-> tax DECIMAL (3,2) NOT NULL
-> );

bl
a
r
fe

mysql> INSERT INTO taxrate VALUES


-> (0, 5687, 0),
-> (5687, 11344, 0.055),
-> (11344, 25195, 0.14),
-> (25195, 67546, 0.3),
-> (67546, 9999999, 0.4);

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
mysql> CREATE TABLE salaries (
@
i
n
-> name char(10) NOT NULL,
h
a
t
r
t
-> salary DECIMAL(10,2)
NOT
NULL
e
-> );
on to us
o
(m nse
n
a
r
e
csalaries
T
i
l
t
mysql>
INSERT
INTO
e-> ("Johan", 30000), VALUES
y
u
g -> ("Max", 55000),

->

an
r
t
n

("Tobias", 90000);

mysql> CREATE VIEW v_salaries AS


-> SELECT *, salary*0.815 AS revenue FROM salaries;

D-8

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Developer Techniques

Thi

mysql> SELECT name, salary, taxpart as totaltax,


-> ROUND(taxpart/revenue*100,2) AS taxpct,
-> ROUND((revenue - taxpart)/salary*100,2) AS totalpct
-> FROM (
->
SELECT name, salary, revenue,
->
ROUND(
->
SUM(
->
IF(revenue <= base,
->
0,
->
IF(revenue < threshold,
->
(revenue - base) * tax,
->
(threshold - base) * tax
->
)
->
)
->
), 2) AS taxpart
->
FROM v_salaries CROSS JOIN taxrate GROUP BY name
-> ) AS tmp;
+--------+----------+----------+--------+----------+
| name
| salary
| totaltax | taxpct | totalpct |
+--------+----------+----------+--------+----------+
| Johan | 30000.00 | 2174.26 |
8.89 |
74.25 |
| Max
| 55000.00 | 8167.56 | 18.22 |
66.65 |
| Tobias | 90000.00 | 17305.46 | 23.59 |
62.27 |
+--------+----------+----------+--------+----------+

Appendix D: Further Practice Solutions

bl
a
r
fe

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

uy
g
N

D-9

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Further Practice Lab 8-A


Step 1: Examine the indexes already present in the Country table

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Utilizing the world database, list the indexes currently in place for the Country table:
mysql> SHOW INDEX FROM Country\G
*************************** 1. row ***************************
Table: Country
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: Code
Collation: A
Cardinality: 239
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:
1 row in set (#.## sec)

bl
a
r
fe

an
r
t
n

no
a
sconcerning each of the
a
The SHOW INDEX command will produce a large amount of information
h
on the Code (the
indexes associated with the table requested. In this case, the index
n)listediisdbased
e
v

three digit country code associated with the country) column.


om t Gu
c

1 in theenCountry table
2column
Step 2: Create and test an index on the Name
s
t
p Stud
s
@
Create an index on the Name column
of the Country
is table:
n
h
a
t
r
t Country
se ADD INDEX (Name);
n
u
mysql> ALTERoTABLE
o239 rowstoaffected (#.## sec)
Query OK,
m
(
e
sDuplicates:
n 239
Records:
0 Warnings: 0
n
a
r
e
c
T
li
t
e
y
u

N
Thi

D-10

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Obtain information about how MySQL would execute the following SELECT statement:

Thi

mysql> SELECT Name, Population FROM Country WHERE Name LIKE '%land';
mysql> EXPLAIN SELECT Name, Population FROM Country
-> WHERE Name LIKE '%land'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 239
Extra: Using where
1 row in set (#.## sec)

an
r
t
n

Obtain information about how MySQL would execute the following SELECT statement:

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

bl
a
r
fe

mysql> SELECT Name, Population FROM Country WHERE Name LIKE 'S%';
mysql> EXPLAIN SELECT Name, Population
-> FROM Country WHERE Name LIKE 'S%'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: range
possible_keys: Name
key: Name
key_len: 52
ref: NULL
rows: 32
Extra: Using where
1 row in set (#.## sec)

uy Which SELECT statement would perform better? Why?


g
N

__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
In this first SELECT statement, the index was blocked from being used (due to the leading %) and a
full table scan would need to be performed (all 239 rows). In the second SELECT statement, the
server would only need to evaluate 32 rows (versus having to search all 239 rows of the table) using
the WHERE clause. This is exactly what was to be expected from the search due to the fact that we
were utilizing an indexed column and searching from the left most portion of the field.

D-11

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Step 3: Create and update a reverse column based on the Name column of the world database

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Add a new indexed column to the Country table called NameReverse that allows for 52 characters
and is located after the Name column.
mysql> ALTER TABLE Country ADD NameReverse CHAR(52) AFTER Name, ADD
-> INDEX (NameReverse);
Query OK, 239 rows affected (#.## sec)
Records: 239 Duplicates: 0 Warnings: 0
This command added a new column (NameReverse) to the Country table (right after the Name
column) and then followed up adding an index on that same column.
Fill this new column with a mirror image of the Name column using the REVERSE function on all the
current records in the table.
mysql> UPDATE Country SET NameReverse = REVERSE(Name);
Query OK, 239 rows affected (#.## sec)
Rows matched: 239 Changed: 239 Warnings: 0

ns
a
r
t
- of the
This UPDATE command accomplished the task of updating each record with a mirror n
image
o
Name column of the Country table.
an
s
a
h
Step 4: Search for a string at the "end" of a column
)
n ide
v

m SELECT
u statement (similar to the
Obtain information about how MySQL would execute theofollowing
G
c

t
poor performing SELECT statement presented earlier):
21 den
s
t
tuCountry
mysql> SELECT Name, Population
sp sFROM
S
@
-> WHERE NameReverse
LIKE
i 'dnal%';
an Name,
thPopulation
r
mysql> EXPLAIN SELECT
t
e
s LIKE 'dnal%'\GFROM Country
n
-> WHERE
NameReverse
u
o
o
to
***************************
1. row ***************************
m
(
e
s
id:
1
n
a icen SIMPLE
r
select_type:
T
ltable: Country
t
e
y
u

N
Thi

bl
a
r
fe

type: range
possible_keys: NameReverse
key: NameReverse
key_len: 53
ref: NULL
rows: 14
Extra: Using where
1 row in set (#.## sec)

The server is now able to utilize the index due the lead % being removed and the result is 14 rows
having to be evaluated versus a fill table scan (293 rows).

D-12

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Step 5: Create triggers to support long-term maintenance of the new column

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

After the NameReverse column has been added to the Country table, keeping this column updated is
a job for triggers. Create both an update trigger (to catch changes to e-mail addresses) and an insert
trigger (for new data being added to the table).

Thi

mysql> CREATE TRIGGER Country_BU_NameReverse BEFORE UPDATE ON Country


-> FOR EACH ROW BEGIN SET NEW.NameReverse = REVERSE(NEW.Name);
Query OK, 0 rows affected (#.## sec)
mysql> CREATE TRIGGER Country_BI_NameReverse BEFORE INSERT ON Country
-> FOR EACH ROW BEGIN SET NEW.NameReverse = REVERSE(NEW.Name);
Query OK, 0 rows affected (#.## sec)

Step 6: Test the alteration and addition of new rows


With the triggers in place to handle both updates and additions to the Country table, it is best to test
it to make sure that the triggers will work as expected. Test the alteration of existing data with the
following SQL statements:

bl
a
r
fe

an
r
t
n

no
a
s
a
h
LIKE 'dnal%';
n) NameReverse
mysql> SELECT Name, NameReverse FROM Countryv
WHERE
e
d
i
+------------------+------------------+ m
o t Gu
c
| Name
| NameReverse
|

1
n
+------------------+------------------+
2
e
s
d
t
| New Zealand
| dnalaeZ p
s weN s St||u
| Iceland
| dnalecI
@
i |
n
| Beigeland
|adnalegieB
h
t
r
t
| Ireland
|
dnalerI
|
e
| dnaliahT
s
n
u
| Thailand
|
o
o
o
t
...
m
se | dnalsI samtsirhC |
| Christmas
Island
n ( Island
n
a
r
e
|
Bouvet
| dnalsI tevuoB
|
lic
tT
mysql> UPDATE Country SET Name='Beigeland' WHERE Name='Greenland';
Query OK, 1 row affected (#.## sec)
Rows matched: 1 Changed: 1 Warnings: 0

e
guy

+------------------+------------------+
12 rows in set (#.## sec)

The first SQL statement updated the Country table by changing the name of 'Greenland' to
'Beigeland'. The second SQL statement retrieved all the records that ended in land to see if the
change was also passed onto the NameReverse column. From the example above, the name column
that was altered also changes the NameReverse column from 'dnalneerG' to 'dnaleqieB'; thus
demonstrating that the trigger for updating records worked.

D-13

MySQL Developer Techniques

Appendix D: Further Practice Solutions

Now test the triggers that manage new records being inserted by entering the following SQL
statements into the mysql client:

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

mysql> INSERT INTO Country (Code, Name) VALUES ('SUN','Sunland');


Query OK, 1 row affected (#.## sec)
mysql> SELECT Name, NameReverse FROM Country WHERE NameReverse LIKE 'dnal%';
+------------------+------------------+
| Name
| NameReverse
|
+------------------+------------------+
| New Zealand
| dnalaeZ weN
|
| Iceland
| dnalecI
|
| Beigeland
| dnalegieB
|
| Ireland
| dnalerI
|
| Thailand
| dnaliahT
|
| Swaziland
| dnalizawS
|
| Finland
| dnalniF
|
| Sunland
| dnalnuS
|
| Poland
| dnaloP
|
| Switzerland
| dnalreztiwS
|
| Norfolk Island
| dnalsI klofroN
|
| Christmas Island | dnalsI samtsirhC |
| Bouvet Island
| dnalsI tevuoB
|
+------------------+------------------+
13 rows in set (#.## sec)

an
r
t
n

no
a
s
a
h
) by adding
ntable
The first SQL statement inserted a new record into the Country
e the country code
v

d
i
u in that order.. The
(Code) and country name (Name) with the values 'SUN'
'Sunland'
G
coinmlandand

t
second SQL statement retrieved all the records that1
ended
to
see
if the insert took place and the
n
2
e
s
new country name was also passed onto thetNameReverse
column.
From the example above, the
d
u
p
t
s
column that was inserted also added the mirror image
S ('dnalnuS') of the Name column to the
@
s
i
n
NameReverse column; thus demonstrating
that
the
trigger for inserting records worked also.
h
a et
r
t

on to us
o
(m nse
n
a
r
ce
T
i
l
t
uye

N
Thi

bl
a
r
fe

D-14

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Thi

<Insert Picture Here>

bl
a
r
fe

an
r
t
n

uy
g
N

no
a
asRecovery
h
Better Performance, Scalability, )and
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Server 5.5

Thi

InnoDB default storage engine


InnoDB performance improvements
MySQL performance improvements
Improved availability
Improved usability
Improved scalability

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
tu
sp InnoDB
S
MySQL
y Q 5.5 introduces a re-architected
that includes many
y pperformance
@
s
i
n
h
a
t
r
and scalability featuresnin
t addition
seto what is available in the previous versions.
u
o
to MySQL server performance and scalability across
MySQL Server(m
5.5ooptimizes
e
ns specifically on multi-CPU/core, hyper-threaded
heterogeneous
an platforms,
r
e
c
T
li
t
architecture.
e
y
gu
CONFIDENTIAL HIGHLY RESTRICTED

MySQL Developer Techniques Appendix E - 2

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

InnoDB: Default Storage Engine

Thi

From MySQL 5.5, InnoDB is the default storage


engine.
ACID Transactions
FKs
Crash Recovery

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
tu
sp s ACID-compliant
S
Oracles InnoDB pprovides highly
g
y
efficient
p
transactional
@
i
n
h
a
t
r
capabilities, and includes
elements that assure high
e
t unique
sarchitectural
n
u
o
o
performance and
scalability.
toThis storage engine is structurally designed to handle
m
(
e
transactional
ns that require crash recovery, referential integrity, high
an applications
r
e
c
T
levels
and SLA exceeding response times.
li
t of user concurrency,
e
y
gu
CONFIDENTIAL HIGHLY RESTRICTED

MySQL Developer Techniques Appendix E - 3

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

InnoDB Performance Improvements

Thi

Multiple buffer pool instances


Multiple rollback segments
Extended change buffering
Improved recovery times

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
tu
sp issupdating
S
No more waitingg when
a
thread
p
g the buffer pool.
p
@
i
n
h
a
t
r
tsegments
Multiple rollback
se now support of 128K concurrent transactions
n
u
o
o
to delete buffering and purge buffering.
Change(m
buffering
includes
e
n ens
a
r
T
lic 5.5, there are default optimizations designed to speed up the
et From MySQL
CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

scanning and applying of redo logs, so the next restart is faster. Users who
previously sized redo logs artificially low because of slow recovery times
can now increase the log file size without concern.

MySQL Developer Techniques Appendix E - 4

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL Performance Improvements

Thi

Better metadata locking within transactions


Split LOCK_open mutex
Eliminated LOCK_alarm mutex as bottleneck
Eliminated LOCK_thread_count as bottleneck
Improved performance/scale on Win32, 64

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
p Stu
sthat
Transaction serialization avoids
one
@
is session pperforms a data definition
n
h
a
t
r
language (DDL) statement
e that is used in an incomplete transaction in
t onuastable
n
o
o
another session.
to
m
(
e
s for speeding up and scaling up
Windows-specific
an icenfeatures
r
T
l API calls for much of the I/O done inside MySQL
t Windows
e

y
gu
CONFIDENTIAL HIGHLY RESTRICTED

Ability to build engines and other plug-ins as DLLs on Windows

Network support for autodetecting the MAC address

Much cleanup and simplifying of threading code

MySQL Developer Techniques Appendix E - 5

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Improved Availability

Thi

Semi-synchronous replication
Replication heartbeat
Replication slave fsync options
Automatic relay log recovery

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
p Stu
sbeen
From MySQL
y Q 5.5,, replication
p
has
@
isenhanced to:
n
h
a
t
r
t ubetween
Ensure data consistency
se master and slave
n
o
o
tifo replication is not working
m
(
Immediately
detect
e
n ens
a
r
c
T
t Allow alicrashed
slave to automatically recover from the master relay log
e
y
gu Allow users to filter events for specific servers
CONFIDENTIAL HIGHLY RESTRICTED

Correctly convert data types between masters and slaves

MySQL Developer Techniques Appendix E - 6

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Improved Usability

SIGNAL/RESIGNAL
More partitioning options
Replication server filtering
Replication slave side type conversions
Individual log flushing
Pluggable external authentication, audit interfaces
PERFORMANCE_SCHEMA

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
sp ysofSthetustored objects
MySQL
y Q 5.5 extends the
usability
j
and table/index
@
i
n
h
a
t
r
partitioning features.
t use
n
o
o RESIGNAL
to statements allow you to implement exception SIGNAL
and
m
(
e
nsin your stored procedures, stored functions, triggers, events,
handling
an iclogic
r
e
T
l applications.
t and database
e
y
u
CONFIDENTIAL HIGHLY RESTRICTED

N
Thi

With the RANGE COLUMNS and LIST COLUMNS clauses of the CREATE
TABLE statement,
statement partitioning is more flexible and can optimize queries
better.

Performance schema provides tables that let you see performance data in
real time, or historically.

MySQL Developer Techniques Appendix E - 7

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

Improved Scalability

Thi

Scales on multi-core with InnoDB 1.0 and even


further with InnoDB built-in
Improved log sys mutex
Separate flush list mutex
Improved purge scheduling

bl
a
r
fe

an
r
t
n

no
a
s
a
h
n) ide
v

om t Gu
c

21 den
s
t
tu
sp stoSincrease
Transactions pper second
continue
beyond
y
16 coresThe internal
@
i
n
h
a
t
r
processing happens
e waiting and less blocking of other operations
t withusless
n
o
o buffertopool. You do not need to do any additional.
involving
the
m
(
sliste mutex means reduced contention with buffer pool
n flush
n
Separate
a
r
e
lic and faster InnoDB. You do not need to do any additional.
et Toperations
CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

From MySQL 5.5, purge can run in its own thread, thereby permitting more
concurrency.
concurrency

Generally, improved performance levels at higher transactions per second


and user connection loads, so applications remain responsive even when the
physical server resources are saturated.
(Mutexes are in-memory structures that prevent different threads from
interfering with each others changes to important memory areas such as the
buffer pool.)

MySQL Developer Techniques Appendix E - 8

MySQL 5.5 Scales on Multi Core

Thi

MySQL 5.5.4
MySQL 5.5.3

Transactions/S
Second

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

S B
SysBench
h Read
R d Write
W it

MySQL 5.1

an
r
t
n

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
CONFIDENTIAL HIGHLY RESTRICTED

AMD Opteron 7160 (Magny-Cours) @2100 MHz


64 GB memory
2 x Intel X25E SSD drives
OS is Oracle Enterprise Linux with the Enterprise Kernel
4 sockets with a total of 48 cores.

uy
g
N

MySQL Developer Techniques Appendix E - 9

bl
a
r
fe

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL 5.5 SysBench Benchmarks


Linux

Thi

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50
(InnoDB Plug-in)

bl
a
r
fe

MySQL 5.1.50
(InnoDB built-in)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Intel Xeon X7460 x86_64


4 CPU x 6 Cores/CPU
2.66 GHz, 32GB RAM
Fedora 10

CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

MySQL Developer Techniques Appendix E - 10

an
r
t
n

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL 5.5 SysBench Benchmarks


Linux

Thi

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50
(InnoDB Plug-in)

bl
a
r
fe

MySQL 5.1.50
(InnoDB built-in)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e

Intel Xeon X7460 x86_64


4 CPU x 6 Cores/CPU
2.66 GHz, 32GB RAM
Fedora 10

CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

MySQL Developer Techniques Appendix E - 11

an
r
t
n

MySQL 5.5 Scales on Multi Core

Thi

Sysbench OLTP: Read / Write


Performance

Transactions Per Second

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

S B
SysBench
h Read
R d Write
W it

10000
8000

6000

bl
a
r
fe

InnoDB

4000

MyISAM

2000
0

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
6

12

18

24

CPU Cores

CONFIDENTIAL HIGHLY RESTRICTED

30

36

AMD Opteron 7160 (Magny-Cours) @2100 MHz


64 GB memory
2 x Intel X25E SSD drives
OS is Oracle Enterprise Linux with the Enterprise Kernel
4 sockets with a total of 48 cores.

uy
g
N

MySQL Developer Techniques Appendix E - 12

an
r
t
n

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL 5.5 SysBench Benchmarks


Windows

Thi

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50
(InnoDB Plug-in)

bl
a
r
fe

MySQL 5.1.50
(InnoDB built-in)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
Intel x86
x86_64
64

an
r
t
n

4 CPU x 2 Cores/CPU
3.166 GHz, 8GB RAM
Windows Server 2008

CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

MySQL Developer Techniques Appendix E - 13

13

Unauthorized reproduction or distribution prohibited Copyright 2016, Oracle and/or its affiliates

MySQL 5.5 SysBench Benchmarks


Windows

Thi

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50
(InnoDB Plug-in)

bl
a
r
fe

MySQL 5.1.50
(InnoDB built-in)

an
s
a
h
)
n ide
v

om t Gu
c

21 den
s
t
sp s Stu
@
i
n
h
a
t
r
t
e
on to us
o
(m nse
n
a
r
ce
T
i
l
t
e
Intel x86
x86_64
64

an
r
t
n

4 CPU x 2 Cores/CPU
3.166 GHz, 8GB RAM
Windows Server 2008

CONFIDENTIAL HIGHLY RESTRICTED

uy
g
N

MySQL Developer Techniques Appendix E - 14

14