MySQL Cluster

Geert Vanderkelen
MySQL Senior Support Engineer
Sun Microsystems

Wednesday, April 29, 2009

 Agenda
• Quick Intro
• New product, new version
• Defining MySQL Cluster
• Components
• Some details
• Features

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Quick Intro: Web Sessions

httpd
mysqld
CREATE TABLE `php_session` (
`id` CHAR(32) PRIMARY KEY,
`data` TEXT,
`expires` DATETIME,
) ENGINE=MyISAM
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Quick Intro: Scaling Out

MySQL

httpd httpd

loadbalancer
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Quick Intro: Adding Cluster

ndbd ndbd

httpd httpd
mysqld mysqld

loadbalancer
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Quick Intro: Adding Cluster

ndbd ndbd

httpd ALTER TABLE `php_session` httpd

mysqld ENGINE=ndbcluster mysqld

loadbalancer
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Quick Intro: Using NDB API

ndbd ndbd

httpd httpd
ndbapi ndbapi

loadbalancer
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 What Is MySQL Cluster?
• Distributed, in-memory storage
> On disk is possible
• High Availability
> Shared-nothing architecture
> Hot backup
> Online upgrade
• Transactional
• ACID
> Two-phase commit
> Local and Global checkpointing
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 What Is MySQL Cluster? (2)
• Performance
> in-memory == fast
> high network latency == slow
• Scalable
> max. 48 data nodes
> 255 total nodes
• NDB Storage Engine
> ALTER TABLE t ENGINE=ndbcluster
• Geographical replication
> Using Row-Based MySQL Replication
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 MySQL Cluster Components
PHP Glassfish Connector/* NDB API

SQL Nodes
(mysqld)

NDB API

Management
(ndb_mgmd)
Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Data Notes
• Contain data and index
• Used for transaction coordination
• Each data node is connect to the others
• Shared-nothing
• Up to 48 data nodes

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 SQL Nodes
• .. or API nodes
• Usually MySQL servers
• Connected to all data nodes
• Using standard SQL
• Native NDB application (e.g. ndb_restore)
• NDB API
> Bypass mysqld

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Management Nodes
• Configuration
> Needed on startup for other nodes
• OK when not running
• Arbitrator during network partitioning
• Logging & Monitoring
• Accessible using ndb_mgm CLI
• MGM API
• Can run on SQL Nodes

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Requirements
• Need at least 3 machines
• Data nodes
> Need lots of memory & disk space!
> v7.0 brings more multi-threading
• SQL/API nodes
> Add more to scale
• Dedicated network
> Separate data node traffic

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 In The Data Center
• Data nodes together
> Different racks
> Different building if little latency
• Behind a firewall !!!!
> No security on data nodes
• Trusted users on mysqld
> mysql> \! ndb_mgm

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 MySQL Cluster: Some Details
• Data Distribution
> Fragmentation
> Synchronous Replication
• Failure handling
• Backup
• Cluster Geographical Replication

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Data Distribution: Fragmentation
• Fragmentation or partitioning
> Tables fragmented horizontally
> e.g. 4 data nodes => fragmented in 4 parts
> Primary data node for one fragment
• Replication
> Secondary copy employees
> Synchronous 1
2
Jan
Geert
500
400
> Nodes with same data grouped 3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400
7 Jackie 100
Copyright © 2009 Sun Microsystems, Inc. 8 Samuel 500
Wednesday, April 29, 2009
 Data Distribution: 2 Data Nodes
employees
1 Jan 500
2 Geert 400
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400
7 Jackie 100
8 Samuel 500

ndbd 1 ndbd 2

Node Group 1
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 2 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400 F2
7 Jackie 100
8 Samuel 500

ndbd 1 ndbd 2

Node Group 1
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 2 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400 F2
7 Jackie 100
8 Samuel 500

F1 F2
ndbd 1 ndbd 2

Node Group 1
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 2 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400 F2
7 Jackie 100
8 Samuel 500

F1 F2
ndbd 1 ndbd 2

F2 F1
Node Group 1
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 2 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400 F2
7 Jackie 100
8 Samuel 500

F1 F2
ndbd 1 ndbd 2

F2 F1
Fx - primary replica
Fx - secondary replica Node Group 1
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400
3 Michel 500
4 Jeanne 900
5 Greet 100
6 Michelle 400
7 Jackie 100
8 Samuel 500

ndbd 1 ndbd 2 ndbd 3 ndbd 4

Node Group 1 Node Group 2

Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900 F2
5 Greet 100
F3
6 Michelle 400
7 Jackie 100 F4
8 Samuel 500

ndbd 1 ndbd 2 ndbd 3 ndbd 4

Node Group 1 Node Group 2

Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900 F2
5 Greet 100
F3
6 Michelle 400
7 Jackie 100 F4
8 Samuel 500

F1 F3
ndbd 1 ndbd 2 ndbd 3 ndbd 4

Node Group 1 Node Group 2

Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900 F2
5 Greet 100
F3
6 Michelle 400
7 Jackie 100 F4
8 Samuel 500

F1 F3 F2 F4
ndbd 1 ndbd 2 ndbd 3 ndbd 4

Node Group 1 Node Group 2

Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900 F2
5 Greet 100
F3
6 Michelle 400
7 Jackie 100 F4
8 Samuel 500

F1 F3 F2 F4
ndbd 1 ndbd 2 ndbd 3 ndbd 4

F3 F1 F4 F2
Node Group 1 Node Group 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Data Distribution: 4 Data Nodes
employees
1 Jan 500
2 Geert 400 F1
3 Michel 500
4 Jeanne 900 F2
5 Greet 100
F3
6 Michelle 400
7 Jackie 100 F4
8 Samuel 500

Host A Host B
F1 F3 F2 F4
NG 1 ndbd 1 ndbd 2 ndbd 1 ndbd 2 ndbd 3 ndbd 4

F3 F1 F4 F2
NG 2 ndbd 3 ndbd 4
Node Group 1 Node Group 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Failure Handling
[ndbd default]
PHP GlassFish Connector/*
NoOfReplicas = 2
DataMemory = 400M
IndexMemory = 32M
DataDir = /var/lib/mysqlcluster/

[ndb_mgmd]
DataDir = /var/lib/mysqlcluster/ SQL Nodes
HostName = 192.168.0.42 (mysqld)
ndb_mgmd
[ndbd]
HostName = 192.168.0.40

[ndbd]
HostName = 192.168.0.41
Data Nodes
[mysqld] (ndbd)
HostName = 192.168.0.42
[mysqld]
[api]
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Failure Handling: SQL Node
• Apps handle failover PHP Glassfish Connector/*

• Transactions aborted
> Need retry
SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Failure Handling: SQL Node
• Apps handle failover PHP Glassfish Connector/*

• Transactions aborted
> Need retry
SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Failure Handling: Data Node
• Transactions aborted PHP Glassfish Connector/*

• Other node takes over

• Node coms back:
> Sync with partner SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Failure Handling: Data Node
• Transactions aborted PHP Glassfish Connector/*

• Other node takes over

• Node coms back:
> Sync with partner SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Failure Handling: Management Node
• No effect on other nodes PHP Glassfish Connector/*

• Logging stops
• Needed for node restart
SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Failure Handling: Management Node
• No effect on other nodes PHP Glassfish Connector/*

• Logging stops
• Needed for node restart
SQL Nodes
(mysqld)
ndb_mgmd

Data Nodes
(ndbd)

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Hot Backups
• Start it in the ndb_mgm
> ndb_mgm> START BACKUP
> Done locally on data node
• No interruption
• Only right way
> mysqldump == inconsistent data
> Schema: use mysqldump
• Restore using ndb_restore tool

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication
• Updates on all SQL nodes
> At least 2 mysqld do binary logging
> Row-Based
• Multiple channels
• NDB API updates also logged
• Master-Master possible
> Conflict detection

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication (2)

binlog

NDB API
binlog

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication (2)

binlog relaylog

NDB API
binlog relaylog

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication (2)

binlog relaylog
Channel 1

NDB API
binlog relaylog

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication (2)

binlog relaylog
Channel 1

NDB API
binlog relaylog

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Geographical Replication (2)

binlog relaylog
Channel 1

Channel 2
NDB API
binlog relaylog

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Use Case: Directory Back-End
• Drivers available
> Like Sun OpenDS and OpenLDAP
• Adds HA
• Using NDB API
> No SQL
> Fast

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 Usage: Sharding And Scaling

Get Data
NDB API
Lookup Cluster 3

Cluster 1 Cluster 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Usage: Sharding And Scaling

Get Data
NDB API
Lookup Cluster 3

Cluster 1 Cluster 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Usage: Sharding And Scaling

Get Data
NDB API
Lookup Cluster 3

Cluster 1 Cluster 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 Usage: Sharding And Scaling

Get Data
NDB API
Lookup Cluster 3

Cluster 1 Cluster 2
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 MySQL Cluster v7.0
• Different development cycle
> Everything still GPL
> MySQL 5.1 binaries lacks Cluster
• Different versioning
> MySQL Cluster v7.0 is GA!
> latest: mysql-5.1.32-ndb-7.0.5
• Manual & change log
> http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster.html
> http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-news.html

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 New In MySQL Cluster v6.2
• Online altering tables
> ALTER ONLINE TABLE ..
• Cluster connection pooling for mysqld
• Less memory overhead
> Using MaxAllocate
• Micro GCP
> Better replication performance
• Database creation discovery
• Multi data node processes per host

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 New In MySQL Cluster v6.3
• Parallel data node recovery
• Data node thread locking
• Geographical Replication
> Conflict detection & resolution
> Heartbeats
• Performance improvements
> Updating primary and unique keys
> Better batching of transactions
• OPTIMIZE TABLE
> for dynamic, in-memory tables
• Compressed backups & LCP
Copyright © 2009 Sun Microsystems, Inc.
Wednesday, April 29, 2009
 New In MySQL Cluster v7.0
• For Windows!
• Adding data node groups online!
• Multi-threading support
• Dynamic send buffer allocation
• Robust DDL operation
> Rolled back incase of failure
• NDB$INFO
> Real-time info using SQL
• IPv6 support with Geographical Replication

Copyright © 2009 Sun Microsystems, Inc.

Wednesday, April 29, 2009
 MySQL Cluster

Geert Vanderkelen
[email protected]
[email protected]

Wednesday, April 29, 2009