Вы находитесь на странице: 1из 63

Introducing Lua for MySQL Proxy programming

Giuseppe Maxia QA Developer December 2007


Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

About me
Giuseppe Maxia MySQL QA Developer a.k.a. The Data Charmer Despite popular belief, not a Lua expert (less than six months experience)
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

Agenda
MySQL Proxy architecture (very quickly) Introducing Lua Language fundamentals Examples Comments

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

References
MySQL Forge
http://forge.mysql.com/wiki/MySQL_Proxy

Getting started article


http://dev.mysql.com/tech-resources/articles/proxygettingstarted.html

Lua
http://www.lua.org/docs.html http://lua-users.org/wiki/TutorialDirectory

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Proxy (< lat. procuratio) = Someone taking care of someone else's interests A server proxy is something acting on behalf of another server
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

Proxy Overview

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Proxy Overview
PROXY CORE connection hook read query hook read result hook Lua script function function function function function

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Lua Overview

??
Why not ...
Copyright 2004-2007 MySQL AB

Perl ? PHP? Javascript? [whatever]?

The Worlds Most Popular Open Source Database

Lua Overview
SMALL ( < 200 KB) DESIGNED for

EMBEDDED systems Widely used (lighttpd) lighttpd, like MySQL Proxy, was created by Jan Kneschke
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

Lua Overview
Very popular among game writers

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

10

Proxy Overview
global context
session session context session context session context session context session context context Lua script

connect_server read_handshake read_auth read_auth_result read_query read_query_result disconnect_client

session context

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

11

Using Lua les


/usr/local/sbin/mysql-proxy \ --proxy-lua-script=/path/name.lua IMPORTANT! THE SCRIPT DOES NOT START UNTIL THE FIRST CLIENT CONNECTION

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

12

Lua types
nil number string table function userdata
a b c t f u = = = = = = nil 1 'abc' { a,b,c } print some_C_struct

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

13

Lua comments
-- simple comment print(1) --[[ print(2) print('hello') --]] print(3)

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

14

Lua comments
-- simple comment --[=[ print(1) --[[ print(2) print('hello') --]] print(3) --]=]
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

15

numbers and strings


nil = no value (false) number = oating point a = nil '5' =/= 5
b = 5; c = '5' print (b == c) false print (b == c +0) true
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

16

numbers and strings

conversion on demand a = 5 ; b = '5' print(type(a), type(b)) number string print(type(b+0)) number print(type(a .. "")) string
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

17

numbers and strings

conversion on demand a = 5 ; b = '5' print(type(tostring(a))) string print(type(tonumber(b))) number

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

18

strings
smart quoting
a b c d e = = = = = 'Hello' "World" "Can't" [[Don't say "Hello"]] [=["d'oh" [[braces]]!]=]

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

19

tables
associative arrays can be used as arrays can create complex
structures t1 = {10, 20, 30 } t2 = { a = 'abc', b = 2, c = { 3, 4} }
The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

20

functions
can be assigned to
variables new functions can override existing ones function f (x) print(x) end g = f g(10)
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

21

userdata
containers to exchange
data between Lua and host language can have "tag methods"

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

22

Statements
normal assignments
a = 3 b = get_num() -- func return multiple assignments a,b = 3,2 multiple return values function x () return 1, 'OK' end a, b, c = x() -- a = 1, b = 'OK', c = nil
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

23

statement blocks
if
if condition then statements end while while condition do statements end

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

24

statement blocks
for
for var = 1, 10 [,step] do statements end for n,v in pairs(table_var) do statements end
The Worlds Most Popular Open Source Database

for

Copyright 2004-2007 MySQL AB

25

sample function
read_query
1 function read_query(packet) 2 if packet:byte() ~= 3 proxy.PROXY_COM_QUERY 4 then 5 return 6 end 7 local query = packet:sub(2) 8 print('received ' .. query) 9 end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

26

some details
== equals ~= not equal string.byte(packet) packet:byte() string.sub(packet,2) packet:sub(2) 'abc' .. '123'
Copyright 2004-2007 MySQL AB

== 'abc123'
The Worlds Most Popular Open Source Database

27

Using tables
t = {} t[1] = 'a' --First element 1, ! 0 t[2] = 'b' table.insert(t, 'c') -- or t[ #t +1 ] = 'c' t = {'a', 'b', 'c' } t = {1 = 'a', 2 = 'b', 3 = 'c'} print (t[2]) b
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

28

Using tables
sizes = {} sizes['john'] = 'XL' sizes['paul'] = 'M' sizes['fred'] = 'L' sizes = { john = 'XL', paul = 'M', fred = 'L', }
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

29

Using tables
sizes = { john = 'XL', paul = 'M', fred = 'L', } print(sizes['john']) XL print(sizes.paul) M
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

30

Using tables
for i, v in ipairs(t) do print (i ..' -> ' .. v) end for name,size in pairs(sizes) do print(name .. ' ' .. wears .. ' ' .. size) end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

31

Watch out
/* C / C++ */ int a = 0; printf("%s\n", a ? "true" : "false"); false -- Lua a = 0 print ( a and "true" or "false") true
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

32

Watch out
-- Lua a = false print ( a and "true" or "false") false a = nil print ( a and "true" or "false") false

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

33

Finding text
query = 'SELECT id FROM t1' local cmd, column = query:match("(SELECT)%s+(%w+)") if cmd then -- do something with query end

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

34

Finding text
Regular expressions similar to Perl/PHP, but simpler
% instead of \ (captures) [character classes] ^ $ + - ? * no alternation (a|b) no modifiers /i
The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

35

Finding text (Proxy way)


local tk = require('proxy.tokenizer') local tokens = tk.tokenize(query) if tokens[1].token_name == 'TK_SQL_SELECT' then -- do something with query end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

36

Finding text (Proxy way)


-- each token is a table token = { token_name = 'TK_SQL_SELECT', text = 'select', token_id = 204 }

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

37

I/O
-- files are objects local fname = '/tmp/test.txt' assert(fh = io.open(fname,'r')) for line in fh:lines() do print(line) end fh:close()
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

38

I/O
-- files are objects local fname = '/tmp/test.txt' assert(fh = io.open(fname,'w')) for x = 1, 100 do fh:write('new row ' .. x) fh:flush() end fh:close()
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

39

Examples
http://forge.mysql.com/wiki/Lua_Scripts_For_MySQL_Proxy_Examples

all hooks session bandwidth user bandwidth blocking commands

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

40

all-hooks (1)
source: all-hooks.lua
function ... function ... function ... function ... function ... function ... function ... read_auth( auth ) connect_server() read_handshake( auth ) read_auth_result( auth ) disconnect_client() read_query (packet) read_query_result (inj)

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

41

all-hooks (2)
source: all-hooks.lua
local access_ndx = 0 function print_access(msg) access_ndx = access_ndx + 1 print( string.format('%3d %-30s',access_ndx,msg)) end function read_auth( auth ) print_access ('inside read_auth ') end function connect_server() print_access ('inside connect_server') end function read_handshake( auth ) print_access ('inside read_handshake' ) end -- ... to be continued
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

42

all-hooks (3)
source: all-hooks.lua
function read_auth_result( auth ) print_access ('inside read_auth_result ') end function read_query (packet) print_access('inside read_query \t' .. packet:sub(2)) proxy.queries:append(1, packet) return proxy.PROXY_SEND_QUERY end function read_query_result (inj) print_access('inside read_query_result \t' .. inj.query) end function disconnect_client() print_access('inside disconnect_client') end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

43

read_query & read_query_result


client
query

result

function read_query

function read_query_result

MySQL Proxy

query result

SERVER

if a query is passed directly to the server, its result is NOT evaluated by read_query_result

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

44

read_query & read_query_result


client

query

result

function read_query_result function read_query query queue


query query query

MySQL Proxy

result

only if a query is added to the query queue, its result is evaluated by read_query_result

SERVER

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

45

all-hooks (4)
source: all-hooks.lua
sample output /usr/local/sbin/mysql-proxy --proxy-lua-script=all-hooks.lua 1 inside connect_server 2 inside read_handshake 3 inside read_auth 4 inside read_auth_result 5 inside read_query select @@version_comment limit 1 6 inside read_query_result select @@version_comment limit 1 7 inside read_query select now() 8 inside read_query_result select now() 9 inside read_query 10 inside disconnect_client

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

46

session-bandwidth (1)
source: session-bandwidth.lua
local bandwidth = 0 function read_query (packet ) bandwidth = bandwidth + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERY end -- to be continued ...

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

47

session-bandwidth (2)
source: session-bandwidth.lua
-- (continue) function read_query_result(inj) local fields = inj.resultset.fields if (fields) then for i = 1, #fields do bandwidth = bandwidth + (fields[i] and fields[i].name:len() or 0) end for row in inj.resultset.rows do for i = 1, #inj.resultset.fields do bandwidth = bandwidth + (row[i] and row[i]:len() or 0) end end end print (proxy.connection.server['thread_id'],bandwidth) end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

48

session-bandwidth
source: session-bandwidth.lua
sample output: /usr/local/sbin/mysql-proxy --proxy-lua-script=bandwidth.lua 64 78 64 115 65 65 66 66 78 115 78 1118

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

49

user-bandwidth (1)
source: user-bandwidth.lua
proxy.global.bandwidth = proxy.global.bandwidth or {} local session_user function read_auth( auth ) session_user = auth.username proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] or 0 end -- to be continued ...

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

50

user-bandwidth (2)
source: user-bandwidth.lua
function read_query (packet ) proxy.global.bandwidth[session_user ] = proxy.global.bandwidth[session_user] + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERY end -- to be continued ...

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

51

user-bandwidth (3)
function read_query_result(inj) local fields = inj.resultset.fields local rows = inj.resultset.rows if fields then for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (fields[i] and fields[i].name:len() or 0) end if rows then for row in rows do for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (row[i] and row[i]:len() or 0) end end end end print (session_user .. ' -> ' .. proxy.global.bandwidth[session_user]) end
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

52

user-bandwidth (4)
source: user-bandwidth.lua
Sample session gmax -> 376 gmax -> 399 root -> 599 root -> 10641 gmax -> 478 gmax -> 20520

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

53

query-block (1)
source: query-block.lua
local tokenizer = require("proxy.tokenizer") ---- make_regexp_from_command() --- creates a regular expression for fast scanning of the command --- @param cmd the command to be converted to regexp -function make_regexp_from_command(cmd) local regexp= '^%s*'; for ch in cmd:gmatch('(.)') do regexp = regexp .. '[' .. ch:upper() .. ch:lower() .. ']' end return regexp end -- to be continued ...
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

54

query-block (2)
source: query-block.lua
local SHOW_REGEXP = make_regexp_from_command('show')

queries_to_filter = { { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLE', 'STATUS'} , }, { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLES'} , }, } -- to be continued ...

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

55

query-block (3)
source: query-block.lua
function error_result (msg) proxy.response = { type = proxy.MYSQLD_PACKET_ERR, errmsg = msg, errcode = 7777, sqlstate = 'X7777', } return proxy.PROXY_SEND_RESULT end -- to be continued ...

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

56

query-block (4)
function read_query( packet ) if (packet:byte() ~= proxy.COM_QUERY) then return end local query = packet:sub(2) for i,query_to_filter in pairs(queries_to_filter) do if query:match(query_to_filter.prefix) then local full_tokens = tokenizer.tokenize(query) local tokens = tokenizer.bare_tokens(full_tokens, true) local found = 0 local requested = #query_to_filter.keywords for j,keyword in pairs(query_to_filter.keywords) do for k, token in pairs(tokens) do if token:upper() == keyword then found = found + 1 break end end end -- to be continued ...
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database

57

query-block (5)
function read_query( packet ) -- ... if found == requested then -- to be filtered off return error_result('command <' .. table.concat(query_to_filter.keywords,' ') .. '> is not allowed' ) end

end

end

end

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

58

query-block (6)
source: query-block.lua
sample output: show tables; ERROR 7777 (X7777): command <SHOW TABLES> is not allowed

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

59

Read more
http://www.lua.org/docs.html

online Lua documentation

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

60

Read more
http://www.inf.puc-rio.br/~roberto/pil2/

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

61

Read more
http://www.wrox.com/WileyCDA/WroxTitle/productCd-0470069171.html

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

62

QUESTIONS

?
Giuseppe Maxia QA Developer December 2007
Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database