OSSCube MySQL Cluster Tutorial by Sonali at Osspac 09

www.osscube.
com
MySQL Cluster Tutorial
Presented by – Sonali Minocha

www.osscube.com
Who Am I?
MySQL Cluster Tutorial, OSSPAC 09

Singapore, © OSSCube
www.osscube.com
Architecture and Organization

www.osscube.com
What is a Cluster?

www.osscube.com
Single MySQL Server Architecture
SQL Request
MySQL Client MySQL Server
Response
www.osscube.com
MySQL Cluster Architecture

NDB API Call
SQL Request
Partitionin
MySQL Client
g
Response Replicatio
n
Message
Response
Data Nodes
SQL Node (MySQL server)
www.osscube.com
Components of MySQL
Cluster

www.osscube.com
Various Nodes
API Data
Node Node
Application
Management
Node

www.osscube.com
NdbCluster NdbCluster NdbCluster

Engine Engine Engine
(Data Nodes)

www.osscube.com
Organization

www.osscube.com
Partition

www.osscube.com
Partitioning
Horizontal
Partitioning
Vertical
Partitioning

www.osscube.com
Fragments

www.osscube.com
Node Groups

www.osscube.com
Split Brain
I have full
Servers can I have full
responsibility
not see responsibility
of the cluster
each other of the cluster

www.osscube.com
Arbitration
NG
1
Network
Split
First Node to ask will continue while the other will
be shut down

www.osscube.com
More Data Nodes
NG Network
1 Split/
Shutdown
N
G

www.osscube.com
Arbitrator Issues

www.osscube.com
Arbitrator Issues

www.osscube.com
Normal Operation
I a ive
m
Al
Al m
Ia
ive
l ive
A m
Al m
Ia
Ia
MySQL Cluster Tutorial, OSSPAC 09 ive
www.osscube.com
Missed Heartbeats
I a ive
m
Al
Alive
I am
Al m
Ia
MySQL Cluster Tutorial, OSSPAC 09 ive
www.osscube.com
Synchronous

www.osscube.com
Phase One: Commit-

Request
Commit Commit
Message Message
Response Response
Message Message

www.osscube.com
Phase Two: Successful

Commit
Transaction Transaction
Successful Successful
Commit Commit

www.osscube.com
Phase Two: Failure

(Abort Commit)
Failed Successful
Rollback Rollback

www.osscube.com
INSTALL MySQL CLUSTER

www.osscube.com
Initiate MySQL Cluster

• # file "config.ini" 2 data nodes and 2 SQL nodes# This file
is placed in the startup directory of ndb_mgmd (the#
management server)
• # The first MySQL Server can be started from any host. The
second# can be started only on the host MySQLd_5.MySQL.com
• [NDBD DEFAULT]
NoOfReplicas= 2
DataDir= /var/lib/MySQLcluster
• [NDB_MGMD]
Hostname= ndb_mgmd.MySQL.com
DataDir= /var/lib/MySQLcluster
• [NDBD]
HostName= ndbd_2.MySQL.com
• [NDBD]
HostName=ndbd_3.MySQL.com
• [MySQLD]
• [MySQLD]
HostName= MySQLd_5.MySQL.com
www.osscube.com
Management Node

www.osscube.com
Data Nodes
• my.cnf# example additions to
my.cnf for MySQL Cluster
• [ndbd]
connect
string=ndb_mgmd.MySQL.com

www.osscube.com
Data Node Configurations

Options

www.osscube.com
SQL Nodes
• my.cnf
• # example additions to my.cnf for
MySQL Cluster
# (will work on all versions)
• # enable ndbcluster storage engine,
and provide connectstring for
management
• # server host to the default port
1186
• [MySQLd]
Ndbcluster
ndbconnectstring=ndb_mgmd.MySQL.com
:1186
www.osscube.com
Config.ini Parameters
General Parameters
– Id
– Hostname
File and Directory Location Parameter

– BackupDataDir
– DataDir
– FileSystemPath

www.osscube.com
www.osscube.com
•Transaction Handling
Parameters
•TransactionInactiveTimeout
•TransactionDeadlockDetection
Timeout
•Memory Usage Parameters

•LockPagesInMainMemory
•Utility Programs
www.osscube.com
Monitoring the Startup Phases
MySQL Cluster Tutorial, OSSPAC 09 Singapore, © OSSCube.

http://osscube.com/technology/mysql
www.osscube.com
MySQL Cluster Storage Engine --NDB

www.osscube.com
NDB Storage Engine

www.osscube.com
NDB’s Storage Method
• In-Memory Storage
• Disk-Based Storage
– New in MySQL 5.1
– Undo Logging
– Fixed-Width Columns

www.osscube.com
Tablespace Tablespace
Datafile DatafileDatafile
Datafile
Log File group
Undo Log
Redo log
www.osscube.com
Indexes

www.osscube.com
Creating Indexes
• Each table in the cluster always has a hash index for a
Primary Key
• Ordered indexes
Country Table: Uniquecan be created
Constraint with
on Name and ALTER
Ordered TABLE
Index ADD
on Continent
Code Name Continent
INDEX .... BRA Brazil South America
CHE Switzerland Europe
CHN China Asia
ESP Spain Europe
FIN Finland Europe
GOR United KingdomEurope
IND India Asia
MOR Morocco Africa
SKO South Korea Asia
Data Node Data Node

www.osscube.com
First Node/First Fragment

• Hidden field/table created that will handle the management
of the Unique Index on Name
• Ordered Index (T-Tree) created for Continent field
Code_Hash Code Name Continent
1351 CHE Switzerland Europe
1785 CHN China Asia
Hidden 1943 FIN Finland Europe
Field 2031 GOR United Kingdom Europe
2345 SKO South Korea Asia
Ordered
Index
Name_Hash Name Code

Hidden 17832 Morocco MOR
Table 24556 Brazil BRA
27432 Spain ESP
28900 United Kingdom GOR
31234 China CHN
www.osscube.com
Second Node/Second Fragment

• Same as First Node/Fragment setup
• Contains the remaining data
Code Hash Code Name Continent
1231 BRA Brazil South America
1853 ESP Spain Europe
Hidden
Field 2145 IND India Asia
2197 MOR Morocco Africa
Ordered
Index
Name_Hash Name Code

Hidden 17832 Finland FIN
Table 24556 South Korea SKO
27432 Switzerland CHE
28974 India IND

www.osscube.com
Primary Key
Unique Key Lookup
Unique
Lookup
Key Lookup may
require another network hop
Function() Look up row

PK values Hash Data node
Look up row
Data node
MySQL Server

www.osscube.com
Parallel Ordered Index

Scan
T-
Tree
Data node
Inde
x
rows
MySQL
Server Data node T-
Tree
Inde
x

www.osscube.com
Parallel full table scan
Data node
rows
MySQL
Server Data node

www.osscube.com
Parallel full table scan ( WHERE

condition processed in SQL
node)
SQL Query
Data node
Few Rows
rows
Many Rows
MySQL
Where
Server
Conditi
on Data node

www.osscube.com

( WHERE condition
processed in data node)
SQL Query
Data node
Where
Few Rows Conditi
Fewrows
Rowson
MySQL
Server
Where Data node

Conditi
on

www.osscube.com
Storage of Indexes

www.osscube.com
Characteristics of NDB
storage Engine

www.osscube.com
Durability

www.osscube.com
Global Check Point
• A GCP occurs every few seconds,

when transactions for all nodes are
synchronized and the REDO log is
flushed to disk

www.osscube.com
Local Check Point

www.osscube.com
Memory - RAM
• Memory Allocation
Total Memory = N(data nodes)* Local
Memory / NoOfReplicas
-- or --
Local Memory = Total Memory *
NoOfReplicas / N(data nodes)
• Sizing up the Binary(15)

1 Byte
Memory Padded
– Fixed size in memory

– Variable sized in memory
Binary(14) Binary(14)
2 Bytes 2 Bytes
– Fixed size on disk Padded Padded

www.osscube.com
Memory Pages
Table A Table B
IndexMemory
Memory Page
Table A
(up to 8Kb)
DataMemory
Memory Page
Table A
(up to 32Kb)
IndexMemory DataMemory
Memory Page Memory Page
Table B Table B
(up to 8Kb) (up to 32Kb)
www.osscube.com

www.osscube.com
HANDLING DATA IN MySQL

CLUSTER
www.osscube.com
Disk Data Objects
Table Space
Data Data
File File
Log File
Table Space
Group
Data Data (Only One per Server)
File File
Undo Undo
Data File File
File Disk Data
Table
Disk ©
Data Disk Data
Singapore, OSSCube Table
Table
www.osscube.com
Creating the Log File Group

• Create Log File Group
CREATE LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_1.dat'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE 2M
ENGINE NDB;
• Adding Undo Files to Log File Group
ALTER LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_2.dat'
INITIAL_SIZE 12M
ENGINE NDB;

www.osscube.com
Verifying UNDO Files are

Created
• INFORMATION_SCHEMA Tables
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,
EXTRA
FROM INFORMATION_SCHEMA.FILES
WHERE FILE_NAME = 'undo_1.dat';
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,

EXTRA
WHERE FILE_NAME = 'undo_2.dat';

www.osscube.com
Creating the Tablespace

• Create a Tablespace
CREATE TABLESPACE ts_1
ADD DATAFILE 'data_1.dat'
USE LOGFILE GROUP lg_1
INITIAL_SIZE 32M
ENGINE NDB;
• Add an Additional Data File
ALTER TABLESPACE ts_1
ADD DATAFILE 'data_2.dat'
INITIAL_SIZE 48M
ENGINE NDB;
• Verifying Data Files are Created
SELECT FILE_NAME, LOGFILE_GROUP_NAME, EXTRA
WHERE TABLESPACE_NAME = 'ts_1' AND FILE_TYPE =
'DATAFILE';

www.osscube.com
Create a Disk Data Table

• Non-Indexed Columns Stored on Disk in
Tablespace
CREATE TABLE dt_1 (
memberId INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
lName VARCHAR(50) NOT NULL, fName VARCHAR(50) NOT NULL,
dob DATE NOT NULL, joined DATE NOT NULL,
INDEX(lName, fName)
) TABLESPACE ts_1 STORAGE DISK ENGINE NDB;
• Alter Existing Non-Cluster Tables

ALTER TABLE city TABLESPACE ts1 STORAGE DISK ENGINE=NDB;
• View Table Status

SHOW TABLE STATUS LIKE 'city'\G

www.osscube.com
Monitoring Free Space

www.osscube.com
Dropping Cluster Tables

www.osscube.com
MySQL CLUSTER
MANAGEMENT
www.osscube.com
MySQL Cluster’s Native Backup
Tool
Backup-1.2.ctl
• Backup Data Data Data Backup-1.2.log
Backup-1.0.2.Data
– .ctl Node Node
Backup-1.1.2.Data
– .log
– .data
• Abort Backup
Backup-1.3.ctl
ndb_mgm> ABORT
Data BACKUP 6 Data Backup-1.3.log
Backup-1.0.3.Data
Node Node
Abort of backup 6 ordered Backup-1.1.3.Data
Node 2: Backup 10 started from 1 has

been aborted. Error: 1321

www.osscube.com
MySQLdump

www.osscube.com
Restoring Backups
• Recovering MySQL Cluster's Native Backup Files
(1/3)
– Verify Management Node is Running
shell> ndb_mgm -e "SHOW"
– Start and Empty the Data Nodes
• Start each data node with the --initial
option.

www.osscube.com
Restoring Backups

www.osscube.com
Restoring Backups

www.osscube.com
Restoring MySQLdump Files

www.osscube.com
Optimization
www.osscube.com
EXPLAIN

www.osscube.com
Full Table Scan

• Without Condition Pushdown
• With Condition Pushdown

www.osscube.com

( WHERE condition
processed in SQL node)
SQL Query
Data node
Where
Few Rows Condition
rows
Many Rows
MySQL
Server Data node

www.osscube.com
Parallel full table scan ( WHERE

condition processed in data
node)
Where
SQL Query Condition
Data node
Few Rows
Fewrows
Rows
MySQL Where
Server Data node
Condition

www.osscube.com
Index Statistics

www.osscube.com
MySQL Cluster Query Cache

www.osscube.com
Data Size/Usage Issues

www.osscube.com
Query Design

www.osscube.com
MySQL Cluster
Replication
www.osscube.com
MySQL Replication
MySQL Server
Application W Slave
MySQL Server Relaylog

Master IO thread
SQL thread
MySQL Server
Binlog
Slave
Relaylog Binlog
www.osscube.com
MySQLd MySQLd
Update
Data node Update

Data node
Cluster
Data node Data node
Update
Update
Ndb API Ndb API
www.osscube.com
NDB Injector Thread

•A thread inside MySQL server
•Responsible for injecting
rows in binlog and produces
“Single connical binlog for
cluster”
• not just one MySQL Server
•It contains everything written on all
ndbApi program including MySQLd
connected to cluster
www.osscube.com
MySQL Cluster Replication
MySQLd(slave) Binlog
MySQLd(Master) Replication
NdbClsuter Handler
NdbClsuter Handler Relaylog

IO thread
Binlog Apply thread
Data node Data node

Data node Data node
Cluster
Cluster Data node Data node
Data node Data node
www.osscube.com
Thank you for your time and attention
www.osscube.com
For more information, please feel free to drop in a line to

sonali@osscube.com or visit http://www.osscube.com

www.osscube.com
QnA

OSSCube MySQL Cluster Tutorial by Sonali at Osspac 09

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

OSSCube MySQL Cluster Tutorial by Sonali at Osspac 09

Загружено:

Авторское право:

Доступные форматы

www.osscube.

MySQL Cluster Tutorial

Presented by – Sonali Minocha

MySQL Cluster Tutorial, OSSPAC 09

Architecture and Organization

MySQL Cluster Tutorial, OSSPAC 09

Single MySQL Server Architecture

MySQL Client MySQL Server

MySQL Cluster Architecture

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

NdbCluster NdbCluster NdbCluster

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

More Data Nodes

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

Phase One: Commit-

MySQL Cluster Tutorial, OSSPAC 09

Phase Two: Successful

MySQL Cluster Tutorial, OSSPAC 09

Phase Two: Failure

MySQL Cluster Tutorial, OSSPAC 09

INSTALL MySQL CLUSTER

Initiate MySQL Cluster

MySQL Cluster Tutorial, OSSPAC 09

MySQL Cluster Tutorial, OSSPAC 09

Data Node Configurations

MySQL Cluster Tutorial, OSSPAC 09

File and Directory Location Parameter

MySQL Cluster Tutorial, OSSPAC 09

•Memory Usage Parameters

Monitoring the Startup Phases

MySQL Cluster Tutorial, OSSPAC 09 Singapore, © OSSCube.

MySQL Cluster Storage Engine --NDB

MySQL Cluster Tutorial, OSSPAC 09

NDB Storage Engine

MySQL Cluster Tutorial, OSSPAC 09

NDB’s Storage Method

MySQL Cluster Tutorial, OSSPAC 09

Log File group

MySQL Cluster Tutorial, OSSPAC 09

Data Node Data Node

MySQL Cluster Tutorial, OSSPAC 09

First Node/First Fragment

Name_Hash Name Code

Second Node/Second Fragment

Name_Hash Name Code

MySQL Cluster Tutorial, OSSPAC 09

Function() Look up row

MySQL Cluster Tutorial, OSSPAC 09

Parallel Ordered Index

MySQL Cluster Tutorial, OSSPAC 09

Parallel full table scan

MySQL Cluster Tutorial, OSSPAC 09

Parallel full table scan ( WHERE