Creative Programming With Mysql: Giuseppe Maxia, Mysql Community Team Lead Database Group, Sun Microsystems Inc

Creative programming with MySQL
Giuseppe Maxia,
MySQL Community Team Lead
Database Group,
Sun Microsystems Inc
giuseppe.maxia@sun.com
The Data Charmer
Creative programming?
2008 CommunityOne Conference | developers.sun.com/events/communityone | 2
WTF?
WTF?
bright ideas
hacking!
WTF?
bright ideas
About me
22 years in IT
consultant
MySQL community
QA developer
MySQL Community Team Lead
About me
database hacker
creative programmer
Does it look familiar?

SELECT * FROM table_name;

UPDATE table_name SET x=1 WHERE
y=10;

y=10;
DELETE FROM tab_name WHERE z=1;

y=10;
make coffee;

y=10;
make coffee;
play movie;

y=10;
make coffee;
play movie;
??
Agenda
Database programming blues
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
MySQL Proxy
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
MySQL Proxy
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
MySQL Proxy
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
What average programmers dislike

regular expressions

regular expressions
"I (love|hate)\s*([Ff]\w*)\s*REGEXP"

regular expressions
database backend

regular expressions
database backend
SELECT t.id
FROM (SELECT id2 FROM other
WHERE b = 2) AS t
WHERE a > (SELECT MAX(x) FROM z);
Why many programmers hate database

handling?
SQL != Java
SQL != Perl
SQL != PHP
SQL != C++
I want to write in my language of choice
database handling
bring the
logic at
application
level
database handling
database
C/C++
Java
Perl
PHP
.NET
database handling
too many languages
Java class
database
C/C++
Java
Perl
PHP
Perl library
PHP library
.NET
database handling
set the logic

at server
level (stored
routines)
database handling
database
C/C++
stored
routines
library
Java
Perl
PHP
.NET
database handling
set the logic

at protocol
level (proxy)
database handling
database
C/C++
Proxy
library
Java
Perl
PHP
.NET
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Stored routines - the boring stuff

SQL standard
heavy syntax
tricky performance
no debug - limited exception handling
procedures
can return multiple record sets
can execute dynamic code
functions
must return one value
no dynamic code
Stored routines - the boring stuff

SQL statements
IF THEN ELSE .. END IF
WHILE .. DO .. END WHILE
LOOP .. END LOOP
CURSOR
CONDITION
HANDLER
Stored routines - some hacks

enhance the language
arrays
easy loops
global variables
see MySQL general purpose routine library
create routines on-the-fly
undocumented hack!
see http://datacharmer.org
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Triggers - the boring stuff

BEFORE or AFTER event
BEFORE INSERT/UPDATE/DELETE
AFTER INSERT/UPDATE/DELETE
No dynamic code
No operations on the same table
No multiple triggers per event
Triggers - what for

dedicated logging
data aggregation
transfer of data to other tables/databases
calculated fields
Triggers - HACKS!
wait.
Strong combination with FEDERATED and
BLACKHOLE
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Views
no storage
look like tables
can aggregate data
can insert data
can save time
but can also become horribly inefficient
Views - the cool stuff

can run functions
can create sort of triggers on select
Views with functions
CREATE FUNCTION f(V VARCHAR(20))

RETURNS INT
BEGIN
INSERT INTO log_table
VALUES (V, NOW());
RETURN 1;
END
Views with functions

CREATE VIEW v1 AS
SELECT x FROM table_name
WHERE f('v1') = 1;
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Events - the not-so-boring facts

Temporal triggers
Operating System independent
executes SQL command or routine
AT timestamp
EVERY X time_interval
can execute dynamic SQL
Events
CREATE EVENT e1
ON SCHEDULE
AT NOW() + INTERVAL 10 minute
DO
CALL clean_employee_recs();
Events
CREATE EVENT e1
ON SCHEDULE
EVERY 5 minute
DO
CALL import_recs_from_file();
Agenda
Stored routines
Triggers
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Federated storage engine

no storage
connects to a table on a remote server
since version 5.1.22 can connect to a table on the
same server
table t1f
federated
table t1
MyISAM
Federated storage engine - caveats

no drop-in replacement for regular tables
inefficient aggregates
limited push-down conditions

limited pushdown conditions
federated
table
base
table
SELECT ID
FROM table_name
WHERE x = 10
expected
~ 100 records
SELECT ID
FROM table_name
filter
returns
1 million records

workaround pushdown conditions
federated
table
add fake index

on column x
to Federated table
SELECT ID
FROM table_name
WHERE x = 10
expected
~ 100 records
base
table
SELECT ID
FROM table_name
WHERE x = 10
returns
100 records
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
Blackhole
The NON-STORAGE engine
like /dev/null in Unix
used for triggers and replication relay
record
record
record
record
binary log
triggers
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
More exciting triggers

Put them together
triggers
Federated engine
Blackhole engine
cascade triggers
BASE TABLE
before insert
after insert
before update
after update
before delete
after delete
t1
cascade triggers
BASE TABLE
before insert
Federated
TABLE
before insert
after insert
before update
after insert
after update
after update
before delete
before delete
after delete
after delete
t1
before update
t1_f1
triggers on request
Federated
TABLE
before insert
either
BASE TABLE
after update
t1_f1
after delete
Federated
TABLE
or
before insert
after insert
before update
t1
t1_f2
before delete
blackhole triggers
BASE TABLE
before insert
blackhole
TABLE
t1
BASE TABLE
t1_b1
t2
remote execution
blackhole
table
before insert
t1
Federated
TABLE
t1_f1
remote execution
VIEW
function
v1
Federated
TABLE
t1_f1
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
loops
or why do we have to endure this?
loops with CURSORS
dreadful syntax
INEFFICIENT (temporary table created)
loops alternatives ... coming soon
loops with cursors :(
data to
process
crc
flag
cursor
temporary
loop
table
handler
crc
loops with cursors (I) :(

CREATE PROCEDURE p(j INT)
BEGIN
DECLARE done BOOLEAN default 'false';
DECLARE crc VARCHAR(42) default '';
DECLARE cid INT;
DECLARE x CURSOR FOR
SELECT id FROM
table_name WHERE
b < j;
-- to be continued
loops with cursors (II) :((

DECLARE CONTINUE HANDLER
FOR NOT FOUND set done = true;
OPEN CURSOR x;
search: LOOP
FETCH x into cid;
IF done THEN
LEAVE search;
END IF;
-- do something with cid
SET crc = MD5(CONCAT(crc,cid))
END LOOP;
SELECT crc;
loops with blackhole :)
data to
process
crc
blackhole
table
crc
loops with blackhole (I) :)

CREATE TABLE t1 (
id VARCHAR(200)
) ENGINE = BLACKHOLE;
SET @crc = '';
SET @j = 10;
loops with blackhole (II) :)

INSERT INTO t1
SELECT @crc :=
MD5(CONCAT(@crc, id))
FROM table_name
WHERE b < @j;
SELECT @crc;
IT'S FASTER!!!
more loops alternatives

use the General Purpose Routine Library
https://sourceforge.net/projects/mysql-sr-lib/
use events!
(ON SCHEDULE EVERY 1 SECOND
ENDS NOW() + INTERVAL 10 SECOND)
use MySQL Proxy (later)
loops with
MySQL stored routines library
call
for_each_table_value_complete(
'db_name', 'table_name',
'id', null, null, '',
'set @crc=MD5(concat(@crc,$I1))',
'set @crc=""',
'select @crc','',
'once');
Agenda
Stored routines
Triggers
Views
Events
Federated
Blackhole
Creative loops
Revisiting Triggers
MySQL Proxy
MySQL Proxy
So, what about "make coffee" ?
MySQL
Proxy
MySQL Proxy principles - hooks

PROXY
CORE
connection
hook
read query
hook
read result
hook

PROXY
CORE
Lua script
connection
hook
function
function
function
read query
hook
function
read result
hook
function

PROXY
CORE
Lua script
connection
hook
function
function
function
read query
hook
function
read result
hook
function
??
MySQL Proxy principles - Lua
Why not ...
Perl ?
PHP?
Javascript?
[whatever]?
SMALL ( < 200 KB)

DESIGNED for
EMBEDDED systems
Widely used (lighttpd)
SMALL ( < 200 KB)

DESIGNED for
EMBEDDED systems
Widely used (lighttpd)
lighttpd, like MySQL
Proxy, was created by
Jan Kneschke
Very popular among

game writers
Very popular among

game writers
Very popular among

game writers
injecting
queries
injecting
queries
Debugging
server
proxy
client
diagnostics
text
Debugging scripts
server
proxy
proxy
diagnostics
text
diagnostics
text
client
Chained proxies - double features

server
proxy
proxy
pivot tables
loops
client
Testing - customized packages

server
proxy
client
fake packets
e.g.
connectors
MySQL Proxy - loops revisited

# get the script from MySQL Forge
# http://forge.mysql.com/tools/tool.php?id=96
mysql-proxy \
--proxy-lua-script=loop.lua

mysql -h 127.0.0.1 -P 4040
FOR VAR 1 3
CREATE TABLE t_$VAR (id int);
# creates t_1, t_2, t_3

mysql -h 127.0.0.1 -P 4040
FOR VAR ( aa, bb cc)
CREATE TABLE t_$VAR (id int);
# creates t_aa, t_bb, t_cc
Summary
MySQL is open to development creativity
Know your bricks
stored routines, triggers, views, events
dynamic engines (Federated, Blackhole)
MySQL Proxy
Put them together : be creative
For More Information

MySQL Forge ( http://forge.mysql.com )
my blog ( http://datacharmer.blogspot.com )
MySQL blogs ( http://planetmysql.org )
Q&A
Let's talk!
80

Creative Programming With Mysql: Giuseppe Maxia, Mysql Community Team Lead Database Group, Sun Microsystems Inc

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Creative Programming With Mysql: Giuseppe Maxia, Mysql Community Team Lead Database Group, Sun Microsystems Inc

Загружено:

Авторское право:

Доступные форматы

Creative programming with MySQL

The Data Charmer

2008 CommunityOne Conference | developers.sun.com/events/communityone | 2

2008 CommunityOne Conference | developers.sun.com/events/communityone | 2

2008 CommunityOne Conference | developers.sun.com/events/communityone | 2

2008 CommunityOne Conference | developers.sun.com/events/communityone | 2

2008 CommunityOne Conference | developers.sun.com/events/communityone | 3

2008 CommunityOne Conference | developers.sun.com/events/communityone | 4

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

Does it look familiar?

2008 CommunityOne Conference | developers.sun.com/events/communityone | 5

What average programmers dislike

2008 CommunityOne Conference | developers.sun.com/events/communityone | 8

What average programmers dislike

2008 CommunityOne Conference | developers.sun.com/events/communityone | 8

What average programmers dislike

2008 CommunityOne Conference | developers.sun.com/events/communityone | 9

What average programmers dislike

2008 CommunityOne Conference | developers.sun.com/events/communityone | 9

Why many programmers hate database

2008 CommunityOne Conference | developers.sun.com/events/communityone | 10

2008 CommunityOne Conference | developers.sun.com/events/communityone | 11

set the logic

2008 CommunityOne Conference | developers.sun.com/events/communityone | 14

set the logic

2008 CommunityOne Conference | developers.sun.com/events/communityone | 16

2008 CommunityOne Conference | developers.sun.com/events/communityone | 17

Stored routines - the boring stuff

Stored routines - the boring stuff

2008 CommunityOne Conference | developers.sun.com/events/communityone | 20

Stored routines - some hacks

2008 CommunityOne Conference | developers.sun.com/events/communityone | 21

Triggers - the boring stuff

2008 CommunityOne Conference | developers.sun.com/events/communityone | 23

Triggers - what for

2008 CommunityOne Conference | developers.sun.com/events/communityone | 24

2008 CommunityOne Conference | developers.sun.com/events/communityone | 25

2008 CommunityOne Conference | developers.sun.com/events/communityone | 27

Views - the cool stuff

2008 CommunityOne Conference | developers.sun.com/events/communityone | 28

Views with functions

CREATE FUNCTION f(V VARCHAR(20))

2008 CommunityOne Conference | developers.sun.com/events/communityone | 29

Views with functions

2008 CommunityOne Conference | developers.sun.com/events/communityone | 30

Events - the not-so-boring facts

2008 CommunityOne Conference | developers.sun.com/events/communityone | 32

2008 CommunityOne Conference | developers.sun.com/events/communityone | 33

2008 CommunityOne Conference | developers.sun.com/events/communityone | 34

Federated storage engine

Federated storage engine - caveats

2008 CommunityOne Conference | developers.sun.com/events/communityone | 37