Scribd
Upload
268 views18 pages

Dynamic Columns

This document describes dynamic columns, a new feature in MariaDB that allows for flexible, schema-less storage of additional attributes in a database table. Dynamic columns store variable metadata in a packed blob column, and provide functions for adding, updating, deleting and querying custom column values without defining the columns explicitly. This allows for easier storage of heterogeneous object types compared to traditional relational or NoSQL approaches. The feature is already available for testing and should be integrated into MariaDB 5.3 within several weeks.

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
268 views18 pages

Dynamic Columns

This document describes dynamic columns, a new feature in MariaDB that allows for flexible, schema-less storage of additional attributes in a database table. Dynamic columns store variable metadata in a packed blob column, and provide functions for adding, updating, deleting and querying custom column values without defining the columns explicitly. This allows for easier storage of heterogeneous object types compared to traditional relational or NoSQL approaches. The feature is already available for testing and should be integrated into MariaDB 5.3 within several weeks.

Uploaded by

Oleksiy Kovyrin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
You are on page 1/ 18

MySQL Conference & Expo 2011

MariaDB
Dynamic Columns

Michael “Monty” Widenius [email protected]


Oleksandr “Sanja” Byelkin [email protected]

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


RDBMS doesn't solve all common problems

The (web) store problem:


All items need: ID, Type, Price, Country, Manufacturer)

A T-Shirt has the following additional properties:


Size, color...
A computer has the following additional properties:
CPU, MHz, memory, Watt...

There is no easy way to store many different types into a


relational database!

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


RDBMS doesn't solve all common problems

One common solutions to this is:

● Store all the 'extra' columns in a BLOB in some format


(HTML?)
● You need a lot of extra work to manipulate the blob

● Hard to access column data (usually done in client)

● Overhead in storage (especially when you use HTML)

● All values are 'text'

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


RDBMS doesn't solve all common problems

Another common solution:


● Create one table for all the 'extra' columns:

CREATE TABLE extra


(id int auto_increment, extra_column_id char(10),
value varchar(255));
INSERT INTO items set type=“t-shirt”, price=10;
INSERT INTO extra (NULL, LAST_INSERT_ID(), “color”,
“Blue”),(NULL, LAST_INSERT_ID(), “Size”, “M”);
The problems with this approach is:
● Every access to an extra column requires a key/row

lookup
● Slow performance (if database is not optimized for this)

● Big overhead in storage (especially in index)

● Risk for errors as data is not typed

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns

Dynamic columns is a bridge between relational


databases and non relational databases
● With dynamic columns all extra columns are stored in a
packed blob, maintained by the database.
● You can add more columns, remove or query them for a

row.
● You can access columns in the server or retrieve the full

blob to the client and manipulate it there.


● You can use virtual columns to create indexes on some

values.
● True indexes for dynamic columns is planned for later.

● Implemented through functions for use by ODBC, & etc.

● First implementation uses integer to access columns.

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: supported
types
● unsigned int
● int

● char [character set <cs>]

● double

● decimal [(<len>, <frac>)]

● time

● date

● datetime

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: syntax

Creating a table with a dynamic column for the store:

CREATE TABLE item (


ID int auto_increment primary key,
Type_id int,
Price decimal(7,2),
Country_id int,
Manufacturer_id int,
extra blob);

Where column 'extra' is dedicated to store dynamic


columns. It could be any column able carry text.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: syntax

Creating/initializing a dynamic_column:

COLUMN_CREATE(column_nr, value [as type],


[column_nr, value [as type]], ...)

INSERT into item values


(NULL, 1 /* T-shirt */, 10, 1 /* Germany */, 1 /* Nike */,
COLUMN_CREATE(1 /* color */, "Blue", 2 /* Size */, "M"));

INSERT into item values


(NULL, 2 /* computer */, 1000, 1 /* Germany */, 2 /* intel
*/,
COLUMN_CREATE(3 /* cpu */, "T9400", 5 /* MHz */, 800 as
unsigned int));

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: syntax

Updating a dynamic column:

COLUMN_ADD(string, column_nr, value [as


type],
column_nr, value [as type]]...)

UPDATE item SET extra=


COLUMN_ADD(extra, 6 /* Memory */, 2048)
WHERE id=2;

If the column already exists, it will be overwritten.

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: syntax

Deleting a dynamic column (if it exists):

COLUMN_DELETE(string, column_nr1,
column_nr2, ...);

UPDATE item SET extra=


COLUMN_DELETE(extra, 6)
WHERE id=2;

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: syntax

Querying a dynamic column:

COLUMN_EXISTS(string, column_nr);
SELECT
ID, Type_id, Price, Country_id, Manufacturer_id from item
where COLUMN_EXISTS(extra, 3);

Querying which columns exist:

COLUMN_LIST(string);
SELECT COLUMN_LIST(extra) FROM item WHERE id=1;
→ “1,2”

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: syntax

Retrieving a dynamic column:

COLUMN_GET(column_nr, string as type);

SELECT id, COLUMN_GET( 1 /* color*/, extra as char)


from item;
→ 1 Blue
→ 2 NULL

You can of course also do things like:


SELECT id, COLUMN_GET(1, extra as char) FROM item
where Type_id=1
order by COLUMN_GET(1, extra as char);

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: C library

C Library allows the same manipulations with dynamic


columns on the client side.

A description can be found in the worklog


http://askmonty.org/worklog/Server-Sprint/?tid=34 and in the
source.

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: encoding

How is the dynamic column encoded?

Header:
<flag><number_of_columns>
Sorted index:
<column_nr1><offset/type><column_nr2><offset/type>
Each column is stored as:
<data1><data2>

Where 'offset' is offset from beginning of data part, 'type' is


3 bits in offset.
Length of the data could be calculated by offsets of 2
neighbor fields.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
Dynamic columns: data encoding

● Unsigned integer is just a variable integer field.


● Signed integer coded to make the variable size efficient:

0 → 0
-1 → 1
1 → 2
-2 → 3
2 → 4
...
● Double, date, time, and date time are fixed-size fields
● String stores collation number and the string

● Decimal stores sizes of parts before and after decimal point

and decimal in MySQL format.


● NULL – means removing the field.

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: avaliability

When will dynamic columns be available?

First version is already pushed in separate tree for testing.


lp:~maria-captains/maria/5.3-mwl34

Should be available in main MariaDB 5.3 within several


weeks.

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Dynamic columns: plans

● Adding name directory


● Adding functional indices

● Supporting popular NoSQL data exchange formats

(for example, JSON or XML)


● Adding engine support for NoSQL databases as

HBase

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Thanks

Q&A

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.

You might also like