THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY.

COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

D62424
Edition 1.0
D61820GC10

December 2010
Student Guide

SQL-4402 Release 3.0

MySQL Performance Tuning

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Copyright © 2009, 2010, Oracle and/or its affiliates. All rights reserved.

Disclaimer

This document contains proprietary information, is provided under a license agreement containing restrictions on use and
disclosure, and is protected by copyright and other intellectual property laws. You may copy and print this document solely for
your own use in an Oracle training course. The document may not be modified or altered in any way. Except as expressly
permitted in your license agreement or allowed by law, you may not use, share, download, upload, copy, print, display,
perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express
authorization of Oracle.

The information contained in this document is subject to change without notice. If you find any problems in the document,
please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This
document is not warranted to be error-free.

Sun Microsystems Inc., Disclaimer

Oracle University and BOS-it GmbH & Co.KG use only

This training manual may include references to materials, offerings, or products that were previously offered by Sun
Microsystems Inc. Certain materials, offerings, services, or products may no longer be offered or provided. Oracle and its
affiliates cannot be held responsible for any such references should they appear in the text provided.

Restricted Rights Notice

If this documentation is delivered to the U.S. Government or anyone using the documentation on behalf of the U.S.
Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS

The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are
restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective
owners.
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

MySQL Performance Tuning Student Guide

This is the Student guide for the MySQL Performance Tuning training course. This guide was developed using the
MySQL 5.0 and/or MySQL 5.1 Community Edition - Generally Available (GA) Releases.

Oracle University and BOS-it GmbH & Co.KG use only

iii
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

Conventions used in this training guide

The following typographical conventions are used throughout this training guide:
● Computer input and output is printed in this format: Computer input or output. This is also used
for the names of executable programs and file locations.
● Keywords from the SQL language appear in this format: SQL KEYWORD. SQL keywords are not case
sensitive and may be written in any letter case, but the training guide uses uppercase.
● Placeholders for user input inside appearing inside computer input appear in this format: <user input>

Oracle University and BOS-it GmbH & Co.KG use only

● For emphasis, the folowing style is used: Emphasis
● For extra emphasis, the following style is used: Extra Emphasis
When commands are shown that are meant to be executed from within a particular program, the prompt shown
preceding the command indicates which command to use. For example, shell> indicates a command that you
execute from your shell, and mysql> indicates a statement that you execute from the mysql client program:

shell> mysql -u root -h 127.0.0.1

mysql> SELECT * FROM world.City;

The “shell” is your command interpreter. On Linux, this is typically a program such as sh, csh, or bash. On
Windows, the equivalent program is command.com or cmd.exe, typically run in a console window. When you enter
a command or statement shown in an example, do not type the prompt shown in the example.
Database, table, and column names must often be substituted into statements. To indicate that such substitution is
necessary, this manual uses db_name, tbl_name, and col_name. For example, you might see a statement like
this:

mysql> SELECT col_name FROM db_name.tbl_name;

This means that if you were to enter a similar statement, you would supply your own database, table, and column
names for the placeholdes db_name, tbl_name, and col_name., perhaps like this:
mysql> SELECT author_name FROM biblio_db.author_list;
In syntax descriptions, square brackets ([ and ]) indicate optional words or clauses. For example, in the following
statement, IF EXISTS is optional:

DROP TABLE [IF EXISTS] tbl_name;

When a syntax element consists of a number of alternatives, the alternatives are separated by vertical bars (pipe, |).
When one member from a set of choices may be chosen, the alternatives are listed within square brackets ([ and ]):

TRIM([[BOTH | LEADING | TRAILING] [remstr] FROM] str)

When one member from a set of choices must be chosen, the alternatives are listed within braces ({ and }):

{DESCRIBE | DESC} tbl_name [col_name | wild]

iv
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

An ellipsis (...) indicates the omission of a section of a statement, typically to provide a shorter version of more
complex syntax. For example, INSERT ... SELECT is shorthand for the form of INSERT statement that is
followed by a SELECT statement.
An ellipsis can also indicate that the preceding syntax element of a statement may be repeated. In the following
example, multiple reset_option values may be given, with each of those after the first preceded by commas:

RESET reset_option[, reset_option] ...

Oracle University and BOS-it GmbH & Co.KG use only

Commands for setting shell variables are shown using Bourne shell syntax. For example, the sequence to set the CC
environment variable and run the configure command looks like this in Bourne shell syntax:

shell> CC=gcc ./configure

If you are using csh or tcsh, you must issue commands somewhat differently:

shell> setenv CC gcc

and

shell> ./configure

v
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

Supporting images used in this Training Guide

The following is a summary of the standard images used in this manual to support the instruction:
IMAGE NAME DESCRIPTION
Preparation This image is used to describe the steps required to be completed
prior to performing a hands-on exercise.

Oracle University and BOS-it GmbH & Co.KG use only

Written exam This image is used to identify that the student is going to be tested
upon the material previously presented in the instructional material

Inline Lab Throughout the course the instructor will conduct labs in line with
the instruction, which are designed to help you to understand the
“nuts and bolts” (inner-workings) of the topic.

Further Practice This image is used to convey to the student that there is a final
Lab exercise to complete prior to the completion of the chapter.

Student notes This image identifies an area on a page designated for students to
write notes associated with the class.

123 Slide number box Indicates the number of an existing slide that corresponds to the text.

vi
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

Acknowledgments
Sun Microsystems would like to thank the many individuals that played a part in bringing this training material to
the numerous students who will benefit from the knowledge and effort that each of these contributors put into the
training. Even though there were a large number of contributions from many Sun Microsystems' employees, the
following list of contributors played a vital role in developing this material and ensuring that its contents were
accurate, timely and most of all presented in a way that would benefit those that are utilizing it for the benefit of
improving their skills with MySQL.
Max Mether, Course Development Manager

Oracle University and BOS-it GmbH & Co.KG use only

Sean Pringle, Course Developer and Editor
Jeff Gorton, Assistant Course Developer
Harrison Fisk, Subject Matter Expert
Peter Zaitsev, Subject Matter Expert

vii
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

2 Course Objectives
The MySQL Performance Tuning course is designed for Database Administrators and others who wish to monitor
and tune MySQL. This course will prepare each student with the skills needed to utilize tools for monitoring,
evaluating and tuning. Students will evaluate the architecture, learn to use the tools, configure the database for
performance, tune application and SQL code, tune the server, examine the storage engines, assess the application
architecture, and learn general tuning concepts.
● Develop a tuning strategy.

Oracle University and BOS-it GmbH & Co.KG use only

● Display knowledge of MySQL architecture.
● Display knowledge and ability to use diagnostic tools.
● Display knowledge and ability to use tuning tools.
● Write queries against the INFORMATION_SCHEMA database and be able to decipher the metadata
obtained.
● Understand the relational database model and both the positive and negative implications on performance.
3 ● Display, decipher and edit server configuration variables to improve performance of your applications.
● Display and analyze status variables to ensure that their applications are utilizing the settings in the most
effective manner.
● Write queries that take advantage of performance enhancements dealing with queries and indexing.
● Decide which of the storage engines should be used in their specific application needs.
● Evaluate the application architecture for efficient design, structure, caching, number of connections and
other factors affecting performance.
● Evaluate techniques for loading data into the database and effects on performance.

viii
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

Table of Contents
1 Introduction............................................................................................................................... 1-1
1.1 Learning Objectives........................................................................................................... 1-1
1.2 MySQL Overview..............................................................................................................1-2
1.2.1 Sun Acquisition.......................................................................................................... 1-2
1.2.2 MySQL Partners.........................................................................................................1-3

Oracle University and BOS-it GmbH & Co.KG use only

1.3 MySQL Products............................................................................................................... 1-4
1.3.1 MySQL Database Server Products.............................................................................1-4
1.3.2 MySQL GUI Tools..................................................................................................... 1-4
1.3.3 MySQL Connectors....................................................................................................1-5
1.3.4 Solutions for Embedding MySQL............................................................................. 1-6
1.4 MySQL Services................................................................................................................ 1-7
1.4.1 MySQL Training........................................................................................................ 1-7
1.4.2 MySQL Certification..................................................................................................1-7
1.4.3 MySQL Consulting.................................................................................................... 1-7
1.4.4 MySQL Support......................................................................................................... 1-7
1.5 The MySQL Enterprise Subscription.................................................................................1-9
1.5.1 MySQL Enterprise Server.......................................................................................... 1-9
1.5.2 24x7 Production Support............................................................................................1-9
1.5.3 MySQL Enterprise Monitor..................................................................................... 1-10
1.5.4 Obtaining a MySQL Enterprise Subscription.......................................................... 1-11
1.6 Supported Operating Systems..........................................................................................1-12
1.7 MySQL Certification Program........................................................................................ 1-13
1.8 Training Curriculum Paths...............................................................................................1-14
1.9 MySQL Website...............................................................................................................1-18
1.9.1 MySQL Community Web Page................................................................................1-19
1.9.2 MySQL Online Documentation............................................................................... 1-21
1.10 Installing MySQL.......................................................................................................... 1-24
1.11 Installing the 'world' database........................................................................................ 1-27
1.12 Chapter Summary.......................................................................................................... 1-29
2 MySQL Architecture................................................................................................................. 2-1
2.1 Learning Objectives........................................................................................................... 2-2
2.2 MySQL Architecture Overview (High Level)................................................................... 2-3
2.2.1 Core Shared Subsystems............................................................................................ 2-3
2.2.2 Query Cache...............................................................................................................2-5
2.2.3 Query Parsing, Optimization and Execution..............................................................2-5
2.2.4 Storage Engine Abstraction........................................................................................2-5
2.3 Client/Server Architecture................................................................................................. 2-6
2.4 Storage Engine Concept.....................................................................................................2-7
2.4.1 Transactional vs. Non-Transactional..........................................................................2-7

x
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

2.4.2 Table Storage Engines................................................................................................ 2-8

2.4.3 SHOW ENGINES.................................................................................................... 2-12
2.5 Locks................................................................................................................................2-20
2.6 Chapter Summary............................................................................................................ 2-23
3 MySQL Performance Tools....................................................................................................... 3-1
3.1 Learning Objectives........................................................................................................... 3-2
3.2 Benchmarking Tools.......................................................................................................... 3-3
3.2.1 mysqlslap....................................................................................................................3-3

Oracle University and BOS-it GmbH & Co.KG use only

3.2.2 Super Smack...............................................................................................................3-8
3.2.3 MyBench.................................................................................................................... 3-9
3.2.4 SysBench..................................................................................................................3-10
3.3 Administration Tools........................................................................................................3-11
3.3.1 Server Logs.............................................................................................................. 3-11
3.3.2 mysqladmin.............................................................................................................. 3-13
3.3.3 MySQL Administrator..............................................................................................3-14
3.4 Specific SHOW commands............................................................................................. 3-15
3.5 Chapter Summary............................................................................................................ 3-16
4 Schema/Database Design.......................................................................................................... 4-1
4.1 Learning Objectives........................................................................................................... 4-2
4.2 Data Modeling .................................................................................................................. 4-3
4.3 Normalization.................................................................................................................... 4-5
4.3.1 Anomalies and Ideal Fields........................................................................................ 4-5
4.3.2 Normalization Forms................................................................................................. 4-6
4.4 Denormalization...............................................................................................................4-11
4.5 Data Types....................................................................................................................... 4-13
4.5.1 MySQL Data Types..................................................................................................4-13
4.5.2 Numeric Data Types.................................................................................................4-14
4.5.3 Temporal Data Types................................................................................................4-15
4.5.4 String Data Types..................................................................................................... 4-15
4.5.5 Binary Data Types.................................................................................................... 4-16
4.5.6 PROCEDURE ANALYSE....................................................................................... 4-17
4.6 Partitioning.......................................................................................................................4-21
4.6.1 MySQL Built-in Partitioning................................................................................... 4-21
4.7 Chapter Summary............................................................................................................ 4-25
5 Indexing..................................................................................................................................... 5-1
5.1 Learning Objectives........................................................................................................... 5-2
5.2 Indexes............................................................................................................................... 5-3
5.3 MySQL Index Types.......................................................................................................... 5-4
5.3.1 Index Overview........................................................................................................5-12
5.4 Index Optimizations.........................................................................................................5-13
5.5 Chapter Summary............................................................................................................ 5-15
6 Statement Tuning.......................................................................................................................6-1

xi
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

6.1 Learning Objectives........................................................................................................... 6-2

6.2 General SQL Tuning.......................................................................................................... 6-3
6.3 EXPLAIN.......................................................................................................................... 6-4
6.3.1 EXPLAIN Output Column - select_type .................................................................. 6-5
6.3.2 EXPLAIN Output Column - type.............................................................................. 6-6
6.3.3 EXPLAIN Output Column - key..............................................................................6-17
6.3.4 EXPLAIN Output Column - Extra...........................................................................6-18
6.3.5 Tuning Example....................................................................................................... 6-19

Oracle University and BOS-it GmbH & Co.KG use only

6.4 MySQL Optimizer........................................................................................................... 6-23
6.5 Finding Problematic Queries........................................................................................... 6-24
6.6 Chapter Summary............................................................................................................ 6-29
7 Caching and MySQL................................................................................................................. 7-1
7.1 Learning Objectives........................................................................................................... 7-2
7.2 Choosing an Architecture...................................................................................................7-3
7.2.1 Architecture Design Best Practices............................................................................ 7-3
7.2.2 Architecture - The Big Picture?..................................................................................7-4
7.3 Caching.............................................................................................................................. 7-6
7.4 Caching Benefits................................................................................................................7-8
7.5 Non-MySQL caches to consider........................................................................................ 7-9
7.6 Query Cache.....................................................................................................................7-11
7.6.1 Query Cache Variables............................................................................................. 7-12
7.7 MEMORY/MyISAM Tables as Cache............................................................................ 7-17
7.8 Chapter Summary............................................................................................................ 7-21
8 MySQL Server Configuration................................................................................................... 8-1
8.1 Learning Objectives........................................................................................................... 8-2
8.2 Server Connection Parameters...........................................................................................8-3
8.2.1 The table cache server variables (table_open_cache and table_definition_cache)....8-3
8.2.2 max_connections........................................................................................................8-8
8.2.3 open_files_limit..........................................................................................................8-8
8.2.4 thread_cache_size...................................................................................................... 8-9
8.3 Table Cache and Multi-threading Issues..........................................................................8-10
8.4 Individual Thread Parameters.......................................................................................... 8-12
8.5 General Monitoring..........................................................................................................8-15
8.5.1 SHOW VARIABLES............................................................................................... 8-17
8.5.2 SET/SELECT @@...................................................................................................8-18
8.5.3 Other SHOW commands..........................................................................................8-21
8.6 Table Handlers................................................................................................................. 8-22
8.7 Chapter Summary............................................................................................................ 8-24
9 MyISAM Storage Engine.......................................................................................................... 9-1
9.1 Learning Objectives........................................................................................................... 9-2
9.2 MyISAM Characteristics................................................................................................... 9-3
9.3 Where To Use MyISAM.................................................................................................... 9-5

xii
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

9.3.1 Where Not to Use MyISAM...................................................................................... 9-5

9.4 MyISAM Row Storage Format..........................................................................................9-7
9.5 Indexing............................................................................................................................. 9-9
9.6 Locking............................................................................................................................ 9-11
9.7 Key Cache........................................................................................................................9-14
9.7.1 Multiple Key Caches................................................................................................9-15
9.7.2 Preloading Indexes................................................................................................... 9-17
9.7.3 Midpoint Insertion Strategy..................................................................................... 9-17

Oracle University and BOS-it GmbH & Co.KG use only

9.7.4 Other Key Cache Status Variables........................................................................... 9-18
9.8 MyISAM Tuning..............................................................................................................9-22
9.9 Other Optimization Tools for MyISAM.......................................................................... 9-23
9.10 MERGE Storage Engine................................................................................................ 9-27
9.11 Chapter Summary.......................................................................................................... 9-31
10 InnoDB Storage Engine.........................................................................................................10-1
10.1 Learning Objectives....................................................................................................... 10-2
10.2 InnoDB...........................................................................................................................10-3
10.3 InnoDB Transactions..................................................................................................... 10-7
10.4 Internal InnoDB Buffers................................................................................................ 10-8
10.5 InnoDB Indexes........................................................................................................... 10-13
10.6 Crash Recovery............................................................................................................10-14
10.7 InnoDB Best Practices................................................................................................. 10-17
10.8 InnoDB Tuning............................................................................................................ 10-18
10.9 Comparing InnoDB and MyISAM.............................................................................. 10-27
10.10 Chapter Summary...................................................................................................... 10-32
11 Alternate Storage Engines......................................................................................................11-1
11.1 Learning Objectives....................................................................................................... 11-2
11.2 MEMORY Storage Engine.............................................................................................11-3
11.2.1 MEMORY Best Practices....................................................................................... 11-4
11.3 FEDERATED Storage Engine....................................................................................... 11-8
11.4 ARCHIVE Storage Engine...........................................................................................11-10
11.5 CSV Storage Engines................................................................................................... 11-11
11.6 BLACKHOLE Storage Engine.................................................................................... 11-15
11.7 How Using Multiple Storage Engines Can Affect Performance..................................11-17
11.8 Creating a Custom Storage Engine.............................................................................. 11-18
11.9 MySQL Cluster............................................................................................................ 11-19
11.10 Chapter Summary.......................................................................................................11-22
12 Conclusion............................................................................................................................. 12-1
12.1 Learning Objectives....................................................................................................... 12-2
12.2 Training and Certification Website............................................................................... 12-3
12.3 Course Evaluation..........................................................................................................12-5
12.4 Thank you!..................................................................................................................... 12-6
12.5 Q&A Session..................................................................................................................12-7

xiii
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Preface

Appendix A MySQL Lab Preparation......................................................................................... A-1

Appendix B Introduction to Linux...............................................................................................B-1
Appendix C Basic vi Commands.................................................................................................C-1
Appendix D Quiz Answers.......................................................................................................... D-1
Appendix E Further Practice Solutions........................................................................................E-1
Appendix F SuperSmack..............................................................................................................F-1
Appendix G MySQL Enterprise Monitor.................................................................................... G-1
Appendix H Better Performance, Scalability, and Recovery.......................................................H-1

Oracle University and BOS-it GmbH & Co.KG use only

xiv
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

7
1 INTRODUCTION

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

8 1.1 Learning Objectives

This chapter introduces you to MySQL and the support organization for MySQL. Upon completion of this chapter
you will be able to:
● Explain the origin and status of the MySQL product
● List the available MySQL products and professional services
● Describe the MySQL Enterprise subscription
● List the currently supported operating systems
● Describe the MySQL Community web page

Oracle University and BOS-it GmbH & Co.KG use only

● Describe the MySQL Certification program
● List all the available MySQL courses

1-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

1.2 MySQL Overview

9 MySQL is a Relational Database Management System (RDBMS) that was originally developed by 'MySQL AB'. It was
(and continues to be) developed and marketed as a family of high performance, affordable database servers and tools.
Contributing to building the mission-critical, high-
volume systems and products worldwide is what makes
MySQL the world’s most popular open source database,
as well as its reliability, excellent performance and ease
of use.
MySQL is not only the world’s most popular open
source database. It is also the fastest growing database

Oracle University and BOS-it GmbH & Co.KG use only

in the industry, with over 70,000 downloads per day.
Ranging from large corporations to specialized
embedded applications.
The original company, 'MySQL AB' was founded in
Sweden by two Swedes and a Finn: David Axmark,
Allan Larsson and Michael "Monty" Widenius who have
worked together since the 80's. MySQL AB (Swedish
for "Inc.") was the sole owner of the MySQL server
source code, the MySQL trademark and the mysql.com
MySQL is installed on every
domain worldwide. Since then, MySQL AB branched continent in the world
out across the globe into many different 'MySQL'
divisions.
(Yes, even Antarctica!)

1.2.1 Sun Acquisition

10
In early 2008, MySQL was acquired by Sun Microsystems, Inc., giving the MySQL product line the considerable
resources of a major mainstream company. Together, Sun and MySQL will now provide global enterprise-class support
24x7x365. Our enterprise-class customers worldwide can now take advantage of our market-leading open source
database on their choice of hardware, operating system and language
with up to a 90% lower total cost of ownership than many traditional
database solutions.
MySQL has always been, and will continue to be, a strong supporter
of the open source philosophy and the open source community, and
strives to work with partners who share the same values and mindset.
This will be no different within Sun. Sun is a world-leader in open
source, with products and technologies such as Java, OpenSolaris, Open Office, NetBeans, OpenSparc and many more.
The MySQL mission is still the same...
Make superior database software available and affordable to all!

1-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

1.2.2 MySQL Partners

MySQL has had the privilege of forming alliances with excellent partners and attracting some of the most impressive
11 customers in the industry! When you join our ranks, you are joining a winning team with a wide variety of MySQL
implementations. It never ceases to amaze us, the innovative and powerful ways in which our tools are being used.
To name only a few;

Oracle University and BOS-it GmbH & Co.KG use only

W e b / W eb 2.0 O E M / IS V 's

O n D e m an d , S a a S , H o s tinT ge le c o m m u n ic a tio n s E n te rp ris e 2 .0

O p e n-s o u rc e is p o w e rin g th e W o rld !

"We have used MySQL far more than anyone expected. We went from experimental to mission-critical in a
couple of months."
Jeremy Zawodny--MySQL Database Expert Yahoo! Finance

1-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

1.3 MySQL Products

1.3.1 MySQL Database Server Products

Sun provides database products to meet the needs of ISV/OEM, Enterprise, and Community Server users. MySQL
12 database products are recognized for superior ease of use, performance, and reliability.
● MySQL Enterprise Server
The most reliable, secure and up-to-date version of the world's most popular open source database for cost-
effectively delivering E-commerce, Online Transaction Processing (OLTP), and multi-terabyte Data
Warehousing applications. (Available only with the MySQL Enterprise subscription).

Oracle University and BOS-it GmbH & Co.KG use only

● MySQL Community Server
The MySQL database server for open source developers and technology enthusiasts
who want to get started with MySQL. Supported by the large MySQL open source
community. Under the General Public License (GPL), benefits to the open source
community include a commercial-grade framework that is free of charge.
● MySQL Embedded Database
The most popular choice for OEMs/ISVs who want to cost-effectively embed or bundle a reliable and high-
performance relational database.
● MySQL Cluster
A fault tolerant database clustering architecture for deploying highly available mission-critical database
applications. Provides carrier-grade availability and performance.

13 1.3.2 MySQL GUI Tools

MySQL is announcing the end-of-life of the MySQL GUI Tools Bundle. This bundle
included the following tools which formed a comprehensive graphical user interface to your
MySQL database:
● MySQL Migration Toolkit
Using a wizard-driven interface, the MySQL Migration Toolkit implements a proven
methodology and walks you through the necessary steps to successfully complete a database migration project.
● MySQL Administrator
A powerful graphical administration console that enables you to easily administer your MySQL environment and
gain significantly better visibility into how your databases are operating.
● MySQL Query Browser
An extremely user-friendly graphical tool for creating, executing, and optimizing SQL queries for your MySQL
Database Server.

1-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

NOTE: MySQL Support will continue to answer questions and provide assistance related to MySQL GUI
Tools Bundle, as well as assist our customers in upgrading from those tools to MySQL Workbench 5.2 until
June 30, 2010.

The features from the above graphical tools are being added to the existing MySQL Workbench starting with the 5.2
revision. Users should plan to upgrade to MySQL Workbench 5.2 GA.
14 MySQL Workbench
The MySQL Workbench GUI Tool (as of the 5.2 revision) will give DBAs and developers an integrated tools
environment for:

Oracle University and BOS-it GmbH & Co.KG use only

● Database Design & Modeling
● SQL Development (replacing MySQL Query Browser)
● Database Administration (replacing MySQL Administrator)
MySQL Workbench 5.2 also provides:
● Remote Administration (using SSH-Tunneling)
● Python scripting and plugins
● Native GUI for Windows, Mac, Linux
A future release of MySQL Workbench (post 5.2) will add a migration plugin, providing features comparable to the
MySQL Migration Toolkit component of the MySQL GUI Tools Bundle.
NOTE: The MySQL Support team will continue to provide technical assistance for MySQL Migration Toolkit
until such migration support functionality is released in MySQL Workbench, but no new development or bug
fixes will be provided against the existing MySQL Migration Toolkit product.

MySQL Workbench is offered in the following editions:

○ Community Edition — available under the open source GPL license.
○ Standard Edition — requires purchase of annual subscription. Includes additional modules and plugins
to improve DBA productivity.

1.3.3 MySQL Connectors

MySQL database drivers (also known as connectors), provide database client connectivity for a wide range of
15 programming languages. MySQL includes the following connectors:
● Developed by MySQL
○ MySQL C API
Our native client library (libmysql), which can be wrapped by other languaages.
○ MySQL Connector/ODBC
Connect to a MySQL database server using the Open Database Connectivity (ODBC) API on all
Microsoft Windows and most Unix-like platforms. The ODBC driver builds on the client/server protocol
implementation provided by libmysql.

1-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

○ MySQL Connector/J
A JDBC (Java Database Connectivity) 4.0 driver for Java 1.4 and higher. Provides a java native
implementation of the MySQL client/server protocol.
○ MySQL Connector/NET
A fully managed ADO.NET provider for the .NET framework (version 1.1 and 2.0). Provides a .NET
implementation of the MySQL client/server protocol.
○ MySQL Connector/C
A C client library for client-server communication. It is a standalone replacement for the MySQL Client
Library shipped with the MySQL Server.

Oracle University and BOS-it GmbH & Co.KG use only

○ MySQL Connector/C++
MySQL database connector for C++. It mimics implements a significant subset of the JDBC 4.0 API.

● Developed by Community
○ PHP - mysqli, ext/mysqli, PDO_MYSQLND, PHP_MYSQLND
Provides MySQL connectivity for PHP programs. Currently there are two MySQL specific PHP extensions
available that use libmysql: the mysql and mysqli extensions. There is also MySQL support for the
generic PHP Database objects (PDO) extension. In addition there is the PHP native driver called mysqlnd
which can replace libmysql in the mysqli extension.
○ Perl - DBD::mysql
○ Python - MySQLdb
○ Ruby – DBD::MySQL, ruby-mysql
○ C++ Wrapper - for MySQL C API (MySQL++)
NOTE: The above connectors (and their documentation) can be downloaded from our MySQL Connectors web
page: http://mysql.com/products/connector/ .

16 1.3.4 Solutions for Embedding MySQL

Apart from these Connectors that provide client side connectivity, MySQL also provides libraries to embed a MySQL
database server within a program. Currently the following solutions can be used to embed MySQL:
● libmysqld
The embedded edition of the mysqld server program wrapped in a shared library. Allows the MySQL Server to
be embedded in C programs.
● MySQL MXJ
A JAR wrapper around mysqld binaries. This allows java programs and J2EE environments to instantiate (and
install) a MySQL server.

1-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

17 1.4 MySQL Services

1.4.1 MySQL Training

Sun Microsystems offers a comprehensive set of MySQL training courses that give you a competitive edge in
building world-class database solutions.
Courses can be chosen individually, as part of a bundle, and/or following our suggested curriculum path for
Developers and Database Administrators (DBAs).

1.4.2 MySQL Certification

Oracle University and BOS-it GmbH & Co.KG use only

MySQL Certification Program is a high quality certification program that provides Developers and DBAs with the
credentials to prove they have the knowledge, experience and skills to use and manage MySQL Server.
With MySQL personal certifications, you get to show that you are among the best-of-breed MySQL users and lay the
foundation for becoming a trusted and valuable resource for your company and customers.

1.4.3 MySQL Consulting

We offer a full range of consulting services. We have an affordable solution for you whether you are starting a new
project, needing to optimize an existing MySQL application, or migrating from a proprietary database to MySQL.
Using industry best practices and proven methodologies, your MySQL certified consultant will help you deliver on-
time and on-budget.

1.4.4 MySQL Support

MySQL offers a full range of support options. MySQL Technical Support is designed to save you time and to ensure
you achieve the highest levels of performance, reliability, and uptime.
● Community support
○ Mailing Lists
○ Forums (http://forums.mysql.com/)
○ Community Articles (http://dev.mysql.com/tech-resources/articles)
○ Bugs Database (http://bugs.mysql.com/)
○ No direct access to support engineers
○ PlanetMySQL Blogs (http://www.planetmysql.org/)
○ MySQL Reference Manuals (http://dev.mysql.com/doc)
○ MySQLForge (http://forge.mysql.com/)
○ Twitter (http://twitter.com/mysql_community)

1-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

● Purchased support
○ Enterprise subscription
○ Support for MySQL Cluster
○ Support for MySQL Embedded (OEM/ISV)
○ Online Knowledge Base

Oracle University and BOS-it GmbH & Co.KG use only

1-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

1.5 The MySQL Enterprise Subscription

18
A MySQL Enterprise subscription gives you access to the Enterprise Server as well
as other premium products and services. It is a comprehensive set of enterprise-grade
software, support and services to ensure the highest levels of reliability, security and
uptime. As a proactive service that helps you eliminate problems before they occur, it
gives you everything you need in a single, unified offering to successfully develop
and deploy business critical applications using MySQL.

19 1.5.1 MySQL Enterprise Server

The most reliable, secure and up-to-date version of the world's most popular open source database for cost-effectively

Oracle University and BOS-it GmbH & Co.KG use only

delivering E-commerce, Online Transaction Processing (OLTP), and multi-terabyte Data Warehousing applications.
With an Enterprise subscription you also get the following in addition to the Enterprise Server:
● Updates and Service Packs
○ Automatically receive both monthly rapid updates and quarterly service packs with the latest bug-fixes
and security updates.
● Emergency Hot Fix Builds
○ Can request a special hot fix build that resolves a critical issue that is not addressed by a service pack.
● Drivers (Connectors)
○ Many different drivers are supported. See MySQL Enterprise website for details.
● Platforms
○ Many different platforms are supported. See MySQL Enterprise website for details.

20 1.5.2 24x7 Production Support

● Service Level
○ MySQL Enterprise gives you the flexibility to choose a service level that matches your requirements.
■ Basic, Silver, Gold, or Platinum (with incrementally increasing quantity and quality of support)
● Problem Resolution Support
○ Ensures you receive high priority service from the MySQL Support Team for quick resolution of
technical problems as they occur. The services available to you are dependent on your chosen service
level.
● Consultative Support
○ Expert MySQL Consultants provide you with proactive advice on how to properly design and tune your
MySQL servers, schema, queries and replication to ensure the highest possible reliability.
● Technical Account Manager (TAM)

1-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

○ You have the option of adding a Technical Account Manager (Platinum level only) to be your liaison within
MySQL. Your TAM be your single point of contact, providing a custom review of your systems, regular
phone calls and on-site visits, to guarantee that you get the most out of MySQL Support Services.
● Online Knowledge Base
○ For quick self-help knowledge, you will have access to a comprehensive and easily searchable knowledge
base library with hundreds of technical articles regarding difficult problems on popular database topics such
as performance, replication, configuration, and security.
NOTE: For more detailed information regarding Enterprise support, please see our Enterprise web page:
http://www.mysql.com/products/enterprise/support.html .

Oracle University and BOS-it GmbH & Co.KG use only

1.5.3 MySQL Enterprise Monitor
The MySQL Enterprise Monitor is a web-based monitoring and advising system. The Enterprise Monitor helps MySQL
21 DBAs manage more MySQL servers in a scale-out environment, tune their current MySQL servers and find and fix
problems with their MySQL database applications before they can become serious problems or costly outages.
The principle features of Enterprise Monitor:
● Enterprise Dashboard
○ Manage all MySQL servers from a consolidated console
● Server or Server-group management
○ Auto detection, grouping and monitoring of replication and scale-out topologies
● Monitoring page
○ "At a glance" global health check of key systems
● MySQL provided Advisors and Advisor Rules
○ Enforce MySQL Best Practices
● Advisor Rule Scheduler
○ Schedule unattended operations
● Customizable Thresholds and Alerts
○ Identify Advisor Rule violations
● Custom Advisor Rules
○ User defined advisor rules
● Events and Alert history
○ Lists all Advisor Rules executions
● Specialized Scale-Out/Replication assistance
● Query Analyzer
○ Comprehensive SQL statement performance analysis by collecting, monitoring, and analyzing queries as
they run on their MySQL servers.

1 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

22 1.5.4 Obtaining a MySQL Enterprise Subscription

For volume discounts or to order by phone, the MySQL Enterprise subscription can be obtained by contacting Sun
Microsystems/MySQL sales personnel:

Oracle University and BOS-it GmbH & Co.KG use only

A permanent MySQL Enterprise subscription can be purchased through our website:
https://shop.mysql.com/enterprise/.
A 30-day trial (with limited features) is also available: http://www.mysql.com/trials/ .

1 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

23 1.6 Supported Operating Systems

MySQL runs on more than 20 platforms, giving users the kind of flexibility that puts them in control. Windows, Linux
and Solaris are the most popular operating systems for running MySQL. Versions of MySQL are currently available for
the following operating systems:
● FreeBSD
● HP-UX
● IBM AIX and i5
● Linux (multiple)

Oracle University and BOS-it GmbH & Co.KG use only

● Mac OS/X
● Microsoft Windows (multiple)
● Novell netware
● Open BSD
● SGI Irix
● Solaris
● Source code
● Special builds

This list is continually being updated. For the most current information, please check our website:
http://www.mysql.com. Source code and special builds are also available.

1 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

24 1.7 MySQL Certification Program

The MySQL Certification Program is a high quality certification program that provides developers and DBAs with
the credentials to prove they have the knowledge, experience and skills to use and manage MySQL Server. MySQL
provides several certification types and levels:
1. Certified MySQL Associate (CMA) - an entry level certification. It is intended for those that are relatively
new to using the MySQL database server and covers basic database management system concepts as well as
basic SQL. We recommend the CMA certification for MySQL users that know the basics, but have not yet
obtained the experience gained by professional MySQL DBAs or Developers.
2. Certified MySQL 5.0 Developer (CMDEV) - proves mastery of the fundamental skills of using MySQL
including creating and using databases and tables, inserting, modifying, deleting, and retrieving data.

Oracle University and BOS-it GmbH & Co.KG use only

3. Certified MySQL 5.0 Database Administrator (CMDBA) - proves mastery of the ability to manage
MySQL Server including such advanced areas of database management, installation, security, disaster
prevention and optimization.
4. Certified MySQL Cluster DBA (CMCDBA) - part of the DBA track which represents an advancement
level exceeding CMDBA certification. In order to attain CMCDBA certification, you must attain CMDBA
certification and pass one CMCDBA exam.

Certification Web Page

For more information on the certification program and the content of the exams, see our Certification web page;
http://www.mysql.com/certification/ .

Exam Administration
All exams are administered through several testing centers available world-wide. Visit the MySQL certification web
page, online forum, or email [email protected] for more information.

1 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

1.8 Training Curriculum Paths

Sun Microsystems offers the most comprehensive set of MySQL training courses, providing a competitive advantage in
creating high-quality database solutions. In addition to our open courses, we also offer in-house training.
The MySQL training services staff has put together great courses designed for success, and an excellent training path for
each individual to reach his/her training goals. There are two curriculum paths:

Developer Path
● Introductory Courses
○ MySQL and PHP: Developing Dynamic Web Applications – This course provide the tools needed for the

Oracle University and BOS-it GmbH & Co.KG use only

development of dynamic web applications. This course takes the student from the basics to the advanced
features in four hands-on, heavy duty training days. Each student will have the opportunity to build real-
world applications during the course. Prerequisites: Basic experience with designing HTML pages
including HTML forms and experience with any programming language. Basic PHP skills. 4 days in length.
○ MySQL for Beginners – This course covers the fundamentals of SQL and relational databases, using
MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous experience with
any command-line program (such as MS-DOS or the MS Windows command prompt) is beneficial. 4 Days
in length.
● Intermediate Courses
○ MySQL for Developers – This instructor-led course is designed for students planning on developing
applications that make use of MySQL 5.0 (and higher). This course covers essential SQL statements for
data design, querying, and programming. In addition, it will prepare students for the MySQL Developer
certification. Prerequisites: Some experience with Relational Databases and SQL. 5 days in length.
○ MySQL 5.0 Upgrading and New Features – This instructor-led course will provide in-depth knowledge
needed to become proficient using MySQL 5.0. This training course will provide quality time both on the
topic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL users who want to
become proficient using MySQL 5.0. 3 Days in length.
● Advanced Courses
○ MySQL Advanced Stored Procedures – This instructor-led course with focus on labs is designed to teach
you how to maximize the use of stored procedures along with the knowledge to discern when an application
should contain stored procedures and when they should not. Prerequisites: Having attended the MySQL for
Developers course, or similar knowledge. 2 Days in length.
○ MySQL Developer Techniques – This instructor-led course will provide database developers with
additional skills in the design, development and maintenance of data and the queries to mine such data.
This course places an emphasis on best practices for developers that improve response time of query
executions. The database developer will also learn additional query writing techniques for creating reports,
and creating trees and hierarchies, that can be used within MySQL to support the needs of the end users.
Prerequisites: Having attended the MySQL for Developers course, or similar knowledge. 3 Days in length.

1 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Database Administrator (DBA) Path

● Introduction Courses
○ MySQL for Beginners – This course covers the fundamentals of SQL and relational databases, using
MySQL as a teaching tool. Prerequisites: Basic computer literacy is required. Previous experience
with any command-line program (such as MS-DOS or the MS Windows command prompt) is beneficial.
4 Days in length.
● Intermediate Courses
○ MySQL 5.0 Upgrading and New Features – This instructor-led course will provide in-depth
knowledge needed to become proficient using MySQL 5.0. This training course will provide quality

Oracle University and BOS-it GmbH & Co.KG use only

time both on the topic, hands-on labs and with the expert instructor. Prerequisites: Existing MySQL
users who want to become proficient using MySQL 5.0. 3 Days in length.
○ MySQL for Database Administrators – This course covers essential DBA tasks such as, installation
and upgrading, user management, disaster recovery, and optimization. In addition, it will prepare
students for the MySQL Database Administrator certification. Prerequisites: Some experience with
Relational Databases and SQL. 5 days in length.
● Advanced Courses
○ MySQL Performance Tuning – The MySQL Performance Tuning course is designed for Database
Administrators and others who wish to monitor and tune mysql. This course will prepare each student
with the skills needed to utilize tools for monitoring, evaluating and tuning. Students will evaluate the
architecture, learn to use the tools, configure the database for performance, tune application and SQL
code, tune the server, examine the storage engines, assess the application architecture, and learn general
tuning concepts. Prerequisites: Attendance to the MySQL for Database Administrators or an equivalent
mastery of database concepts, SQL and the MySQL server. 4 Days in Length.
○ MySQL Cluster - Learn how to install and configure the cluster nodes to ensure high availability. Also
learn about all of the high availability features that are implemented in the MySQL Cluster storage
engine — fail-over between storage nodes, network partitioning protocol, two-phase commit and much
more. Prerequisites: Attendance to the MySQL for Database Administrators or an equivalent mastery
of database concepts, SQL and the MySQL server. 3 Days in length. Note: This is the sole course
offering in the Cluster certification (CMCDBA) path.
○ MySQL High Availability – This course is designed for experienced database administrators and
system architects that want to analyze and form a basis of understanding different high availability
options, including clustering and replication solutions within MySQL. This course will provide the tools
required to make the decision of what high availability solution is appropriate and how to implement a
system with the correct design. Prerequisites: Attendance to the MySQL for Database Administrators or
an equivalent mastery of database concepts, SQL and the MySQL server. 3 days in length.

1 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

25 Curriculum Path Chart

Intermediate Introductory
MySQL
and PHP MySQL for Certification:
Beginners CMA
4 days 4 days

Oracle University and BOS-it GmbH & Co.KG use only

MySQL for MySQL for
Developers Certification: DBAs Certification:
CMDEV I-II CMDBA I-II
5 days 5 days
Special Topics

MySQL
Advanced MySQL MySQL
Advanced/

Developer Performance MySQL High MySQL

Stored Cluster
Procedures Techniques Tuning Availability
2 days 3 days 4 days 3 days 3 days

Certification:
CMCDBA

Developer DBA

1 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Virtual Classroom
26 MySQL also offers virtual (online) courses covering various topics in relation to the MySQL suite of tools. These
classes are instructor-led and delivered synchronously, via the web. Information regarding the available courses can
be found on the MySQL training web page.
Introductory

PHP MySQL MySQL

Elements Workbench Enterprise
Monitor

Oracle University and BOS-it GmbH & Co.KG use only

4 hours 4 hours 4 hours
Intermediate

MySQL Stored MySQL MySQL

MySQL Procedure Storage
Transactions Partitioning
Fundamentals Engines
4 hours 4 hours 4 hours 4 hours
Special Topics
Advanced/

Writing MySQL Stored

MySQL UDFs Procedure
Techniques
2x4 hours 4 hours

Developer DBA

1 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

27 1.9 MySQL Website

Everything you ever wanted to know about MySQL (and more) can be found on our website: http://www.mysql.com *.
From the home page, you may navigate the web site with the tabs (pull-down menus) across the top, the menu along the
left side, and/or the many links on the page.

Oracle University and BOS-it GmbH & Co.KG use only

* MySQL information, products and services can also be accessed through the Sun Microsystems website:
http://www.sun.com/software/products/mysql/ .

1 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

28 1.9.1 MySQL Community Web Page

The MySQL Community web page is located at http://dev.mysql.com and is maintained by MySQL. The
"Developer Zone" page, is the main support tool for the MySQL open source community and can also provide
valuable insight for Enterprise users.

Oracle University and BOS-it GmbH & Co.KG use only

1 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

29 Information such as the following can be found on the Developer Zone web page:
● Current product and service promotions
● Get Started with MySQL
○ Installation information page for MySQL beginners
● Developing with:
○ Links to specific information on using MySQL with; PHP, Perl Python, Ruby, Java/JDBC and
.Net/C#/Visual Basic
● MySQL Librarian

Oracle University and BOS-it GmbH & Co.KG use only

○ A collection of community-generated and cross referenced content related to MySQL
○ A place where the community collaboratively builds and maintains content, such as links and descriptions to
MySQL-related presentations, pictures, videos and articles
● MySQL Server Community Edition
○ All current General Available (GA), Release Candidate (RC) and Alpha versions of the MySQL Server
● New Releases
○ Latest code release for all MySQL products
● Software Previews
○ New features available for preview by users
● What's New
○ Latest news about the company, products, services, etc.
● MySQL Training
○ List of upcoming courses provided by MySQL Training Services
● MySQL Quickpoll
○ Give your opinions about current MySQL-related issues
● Stay Connected
○ Blogs, Lists, Guilds, MySQL Meetups and Quality Contributions
● Resources
○ White Papers, Articles, Training, Webinars, and MySQL Newsletter
● And much more!

1 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

30 1.9.2 MySQL Online Documentation

Documentation for all MySQL products can be found from our website, on the "Documentation" page:
http://dev.mysql.com/doc/ . The page includes links for downloading the following;
● MySQL Reference Manual
● Excerpts from the Reference Manual
● MySQL GUI Tools Manuals
● Expert Guides

Oracle University and BOS-it GmbH & Co.KG use only

● MySQL Help Tables
● Example Databases
● Meta Documentation
● Community Contributed Documentation
● Printed Books
● Additional Resources

1 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Inline Lab 1-A

In this exercise you will review some web pages on the MySQL website using the web browser of your choice.

Step 1. Place cursor in web browser URL field.

Action: Click in the text area of the Address/Location Toolbar.

Effect: The URL currently listed will be selected.
Step 2. MySQL web address

Action: In the toolbar text area, type:

Oracle University and BOS-it GmbH & Co.KG use only

www.mysql.com

Effect: The MySQL Homepage is displayed.

Step 3. Products

Action: Click on the Products tab located at the top of the MySQL home page. Scroll down the list to review the
various product information provided.
Effect: A list of currently available products will appear, with links for further information and downloads.
Step 4. MySQL Enterprise

Action: Review the details of the new MySQL Enterprise program, by clicking on the Learn More >> link.
Effect: MySQL® Enterprise™ provides a comprehensive set of enterprise-grade software, support and services
directly from the developers of MySQL to ensure the highest levels of reliability, security and uptime.
Step 5. Services

Action: Click on the Services tab located near the top of the MySQL home page.
Effect: A list of currently available services will appear, with links for further information and downloads.
Step 6. MySQL Training and Certification

Action: Review the details of the MySQL Training & Certification program, by clicking on the Learn More >>
link.
Effect: Featured information on this page will be updated periodically. For specific Training sub-topics select one of
the links in the sub-menu in the upper-left corner of the page.
Step 7. Certification

Action: From the MySQL Training & Certification web page, select the Certification link in the sub-menu and
review the contents.
Effect: Featured information on this page will be updated periodically. For specific Certification sub-topics select
one of the links in the sub-menu in the upper-left corner of the page.

1 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Step 8. Return to Services

Action: Click on the Services tab located at the top of the MySQL home page.
Effect: Returns to the top level Services page.
Step 9. Support Services

Action: Review the details of MySQL Support programs, by clicking on the Learn More >> link. (Continued
on next page.)
Effect: Shows the various support programs available.
Step 10. MySQL Community

Oracle University and BOS-it GmbH & Co.KG use only

Action: Click on the Developer Zone tab located at the top of the MySQL home page.
Effect: The community page contains many links to more information and access to various open
communication forums.
Step 11. News & Events

Action: Return to the MySQL home page using the MySQL.com tab at the top of the page, then click on the
News & Events tab located near the top of the MySQL home page.
Effect: Shows the latest MySQL happenings, from news stories to upcoming seminars.

1 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

31 1.10 Installing MySQL

It is recommended to download the MySQL database server from the MySQL web site downloads page:
http://dev.mysql.com/downloads/. There are several different platforms supported, and there are differences in the
specific installation details for each operating system.

Inline Lab 1-B

In this lab you will setup the pre-loaded MySQL Server executable (designed specifically for the Oracle Linux classroom
environment).

Step 1. Setup the MySQL Server file structure

Oracle University and BOS-it GmbH & Co.KG use only

Action: Open a terminal window from the desktop.
Effect: A terminal window opens in the /home/oracle directory.

Action: Log in as root using the following command (the password for root is oracle):

su -

Effect: You are now logged in as root in /root directory.

Action: Change into the /usr/local directory using the following command:

cd /usr/local

Effect: This will be the directory where the MySQL folder will be located.

Action: Create a symbolic link from the MySQL folder in the /stage directory to a /usr/local/mysql
directory by issuing the following command:

ln -s /stage/mysql-5.1.44-linux-i686-icc-glibc23 mysql

Effect: The /usr/local/mysql directory now exists.

Action: Execute the mysql_install_db script to initialize the MySQL data directory and creates the system
tables using the following commands:

cd mysql
scripts/mysql_install_db

Effect: The MySQL system tables are installed and the help files are filled.

1 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Step 2. Setup the MySQL server to execute with the mysql user

Action: Execute the following command to create a group called mysql:

groupadd mysql

Effect: The group mysql is created and will be used in the next few steps.

Action: Execute the following command to create a user called mysql assigned to the mysql group using the
following command:

Oracle University and BOS-it GmbH & Co.KG use only

useradd -g mysql mysql

Effect: The user mysql is created and assigned to the group mysql.

Action: Change the owner of the /usr/local/mysql/data directory to mysql using the following command:

chown -R mysql:mysql data

Effect: The data directory that houses the mysql data is now owned by the user mysql instead of root.
Step 3. Start the MySQL server

Action: Enter the following command to copy the mysql server executable to the /etc/init.d directory:

cp support-file/mysql.server /etc/init.d/mysql

Effect: This process will allow the MySQL server to start up every time the server is started/restarted.

Action: Start the MySQL server by entering the following command:

/etc/init.d/mysql start

Effect: This will start the MySQL server file that was just copied to the /etc/init.d directory. By starting
it at this location, you are assured that the MySQL server will start on server start/restart.

1 - 25
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Action: Edit the /etc/bashrc file and add the path to the MySQL bin directory at the end of the file using the
following commands:
• vi /etc/bashrc
• Press the capital letter G to go the end of the file
• Press the capital letter A to insert text at the end of the line.
• Press the <Enter> to go down to the next line.
• Type PATH=${PATH}:/usr/local/mysql/bin

Oracle University and BOS-it GmbH & Co.KG use only

• Close out and save the file by pressing the <Esc> key followed by the lower case letters wq and then the
<Enter> key.
Effect: This will add the bin directory for the MySQL application to the path. This will allow you to call any mysql
applications from any directory.

Action: Refresh the bashrc file by executing the following command:

source /etc/bashrc

Effect: This will allow the bashrc file to be refreshed without having to restart the server.

NOTE: For full installation from the MySQL website on the Windows and/or Linux Operating systems, see
Appendix A at the end of this training guide.

1 - 26
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

32 1.11 Installing the 'world' database

MySQL provides three example databases to be used for testing server features and training. These databases can be
found on the MySQL website documentation page; http://dev.mysql.com/doc/index-other.html. The 'world'
database will be used throughout this course.

Inline Lab 1-C

This lab requires you to use the Firefox web browser and the MySQL command line client in order to install the pre-
world database. In order to use the pre-made tables in the world database, you will need to create the database
(empty) and then upload the file containing the table data.

Oracle University and BOS-it GmbH & Co.KG use only

Step 1. Download the world database

Action: Open the Firefox web browser (icon located on the desktop) and go to the following website to
download the world example database:

http://dev.mysql.com/doc/index-other.html

Effect: This will open up the Other MySQL Documentation page on the MySQL developers website.

Action: Select the Gzip link to the right of the world database text located under the Example Databases
heading.
Effect: This will open up a pop-up window asking you what you would like to do with the file you are about to
download. Choose the Save File option and press OK. The file is automatically downloaded to the
/home/oracle/Desktop directory.

Action: Open up a terminal window (or use the terminal window from the last lab) and enter the following
commands:

cd /home/oracle/Desktop
gunzip world.sql.gz

Effect: This will unzip the world.sql.gz file leaving just the world.sql file to be used in the next few
steps.

1 - 27
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

Step 2. Start the MySQL client from a command line prompt.

Action: Type the following at a command line prompt;

shell> mysql -u root

Effect: A message will appear confirming that the connection to your version of the server has been launched;

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 1

Oracle University and BOS-it GmbH & Co.KG use only

Server version: 5.1.44 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input
statement.

Step 3. Create 'world' database

Action: Type the following in the MySQL command line client:

mysql> CREATE DATABASE world;

Effect: Creates the (empty) world database. Returns following message;

Query OK, 1 row affected (0.02 sec)

Step 4. Select the 'world' database

Action: Type:

mysql> USE world;

Effect: Instructs client to use the newly created world database. Returns the following message;

Database changed

Step 5. Build 'world' database tables

Action: Type:*

mysql> SOURCE /home/oracle/Desktop/world.sql;

Effect: * Instructor will give you the file path if it is different than shown here. Several 'Query' messages will scroll
pass while tables and data are being uploaded for the world database.

1 - 28
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 1 - Introduction

33 1.12 Chapter Summary

This chapter introduced you to MySQL and the support organization for MySQL. You should now be able to:
● Explain the origin and status of the MySQL product
● List the available MySQL products and professional services
● Describe the MySQL Enterprise subscription
● List the currently supported operating systems
● Describe the MySQL Community web page
Describe the MySQL Certification program

Oracle University and BOS-it GmbH & Co.KG use only

●
● List all the available MySQL courses

1 - 29
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

34
2 MYSQL ARCHITECTURE

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

35 2.1 Learning Objectives

This lesson provides the information necessary to understand the MySQL architecture. In this lesson, you will learn
to:
● Describe the complete MySQL architecture at a high level
● Discuss the client/server architecture
● Describe MySQL storage engines
● Understand the MySQL memory areas
● Describe locks

Oracle University and BOS-it GmbH & Co.KG use only

2-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

36 2.2 MySQL Architecture Overview (High Level)

MySQL’s architecture is a web of task-related functions which work together to complete the responsibilities of the
database server. These function sets combined consist of more than 500,000 lines of code. The following diagram is an
overview of the subsystems that are layered and interact with each other through a well-defined function application
program interface (API). In most cases, each subsystem is responsible for receiving, processing and then passing on
information for the next subsystem to complete its designated tasks. The individual subsystems are quite independent
from each other and this leads to greater freedom (such as with the ability to issue statements that are storage engine
independent).

Client
Client Program
Program

Oracle University and BOS-it GmbH & Co.KG use only

C Core
Core Shared
Shared Subsystems
C client
client API
API Subsystems

Query Base Function Library

Query Parsing
Parsing

Query Optimization Process, Thread, and Resource

Query Cache
Cache Optimization
Management
Execution
Execution
Cache and Buffer Management
Storage
Storage Engine
Engine Abstraction
Abstraction
Storage
Storage Engine
Engine Implementations
Implementations Networking
MyISAM MEMORY Management and Communication
handler and handler and
library library Log Management
NDB cluster
InnoDB handler Access and Grant Management
handler and
and library
library

And so on ...

2.2.1 Core Shared Subsystems

Each of the subsystems within MySQL could be a course of its own. For the purpose of this course, the following brief
descriptions of the core shared subsystems are presented to give a further explanation to the high level overview and
provide a better understanding of the performance features within MySQL, but are sparse enough to ensure that they do
not overshadow the purpose of the course.

2-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

37 Process, thread and resource management

MySQL utilizes a thread-based server architecture which allows the various executing threads (or lightweight
processes) to access key shared resources. MySQL’s multi-threaded single process architecture ensures that multiple
executing threads do not conflict with each other or overwrite critical data. The most notable advantages to using a
thread-based server architecture are:
● Cost savings – It is less costly to create or destroy threads than processes. By threads using the parent
process’s address space, additional address space for a new thread is not needed.
● Switching inexpensive – Due to threads running in the same server space it is fairly inexpensive to switch
between threads.

Oracle University and BOS-it GmbH & Co.KG use only

● Minimal overhead – Due to threads having access to the parent’s process address space, overhead is
minimal within shared resources.
NOTE: Conflicting processes

Due to the fact that MySQL is a thread-based server architecture there is the potential for problems to occur
when two processes attempt to use the same common resource. Mutex (mutual exclusion) algorithms or
semaphores are able to minimize the potential for any possible conflicts (one process canceling out the other)
in concurrent programming; however, even this process is not 100% reliable. With that said, in MySQL there
are times when using certain "work arounds" have to be implemented to prevent any potential conflicts. Some
of these "work arounds" will be discussed in the process of this course.

38 Cache and buffer management

This subsystem is devoted to the caching and retrieval of the numerous types of data used by all the threads executing
within the server process. The data caches (buffers) enable MySQL to reduce the number of requests for expensive
disk-based I/O operations through the less expensive operation of returning data already stored in memory (buffers).
The cache and buffer management subsystem is not actually responsible for storing data but is responsible for
breaking up logical pages into blocks and smaller units that can be read from the permanent storage media.
39 Networking management
The responsibility of this subsystem is to make the various communication protocols (TCP/IP, Named Pipes, etc.)
transparent for the connection thread by handling the work of sending and receiving network packets containing
MySQL connection requests and commands across a variety of platforms. This includes the handling of Secured
Socket Layers (SSL).
40 Log management
This subsystem was created to allow a variety of log event subclasses to be maintained by one log class, giving
developers the ability to add logs and log events without breaking the system’s core functionality. By separating the
logging system into its own subsystem, a variety of system activities (startup, multi-statement transactions, auto-
increment value changes, etc.) can be logged via the subclass events.

2-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Access and grant management

41
This subsystem is ultimately responsible for the security between the client and the server by defining all the GRANTs
needed to execute a given command. This subsystem verifies user access rights during the login process and each query
execution, along with some functions to modify the in-memory versions of the grant tables, and functions for password
generation.

42 2.2.2 Query Cache

This “subsystem” is a set of classes that function as a component (unlike many other subsystems in MySQL) that is
responsible for not only caching the SQL commands executed on the server but also storing the results of those
commands. The Query Cache is ideal for static and semi-static web sites.

Oracle University and BOS-it GmbH & Co.KG use only

NOTE: More to come

As with many of the other subsystems discussed in this overview, the query cache is a key discussion in
performance tuning and will be discussed in greater detail during the remainder of this course.

43 2.2.3 Query Parsing, Optimization and Execution

These subsystems are the brains of the MySQL server due to their responsibility for receiving SQL statements,
deconstructing the requested statements into a variety of data structures and then executing them along the best path.

Query parsing
This is the process of deconstructing the SQL statement into a parse tree. This is an extremely complex process that does
not have any user variables that can be modified.

Optimization
This subsystem is responsible for finding the "optimal" execution plan for each query.
NOTE: More to come

Optimization is a key discussion in performance tuning and will be discussed in greater details throughout the
remainder of this course.

Execution
This process is called the statement execution unit and is responsible for executing the optimized path for the SQL
command passed through the parser and optimizer. This process is ultimately responsible for ensuring the results of the
query executed reach the originating client.

44 2.2.4 Storage Engine Abstraction

This subsystem enables MySQL to use different handlers (subroutines) of the table data within the system architecture.
The majority of this handler API is used to convert data, schema and index formats into the format needed by MySQL’s
internal (in-memory) record format, but not all storage engines completely implement the handler API.

2-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

45 2.3 Client/Server Architecture

MySQL operates on a client-server model, not unlike many other network applications. The MySQL server handles
incoming and outgoing traffic to and from a database server process. In order to access the data in the database a
MySQL server must be running. The MySQL server, named mysqld, "listens" on a particular port for incoming
connection requests -- the default port for MySQL being 3306. Unless this port is manually reconfigured there is no
need to worry about specifying a port when using a MySQL client.
The "client" in MySQL is whatever application is sending a request to the database via the server. A client is being
used to communicate with the MySQL server anytime a command is sent, whether that is a PHP script or one of the
standard MySQL clients.
On Windows local connections can use a shared pipe or shared memory, however, most local connections

Oracle University and BOS-it GmbH & Co.KG use only

●
use TCP/IP.
● Unix/Linux local connections use a Unix Socket file.
● Remote connections have to be made via TCP/IP.

2-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

46 2.4 Storage Engine Concept

MySQL is unique among databases in that when a table is created, the table type can be specified for each individual
table. MySQL supports a number of table types with each having their own unique advantages and limitations. The
following diagram represents a simplified view of the MySQL server and its interaction with the storage engines.
The following properties are storage engine dependent:
● Storage medium – Each storage engine uses its own method of storing data.
● Transactional capabilities – Certain storage engines handle transactional processing which ensures that
integrity of a database is maintained during the processing of multiple SQL statements. Storage engines that are
non-transactional will ignore transaction statements if executed against them.

Oracle University and BOS-it GmbH & Co.KG use only

● Locking – Each storage engines handles the processes of the synchronization mechanism for enforcing limits on
access to a resource in an environment where there are many threads of execution.
● Backup and recovery – Based on the storage medium used, the backup of the table data and the recovery of
that data can be distinct.
● Optimization – 'There are specific advantages associated with most storage engines (over other storage engines
based on the task it is designed to excel at) for the optimization of the storage of the data and retrieval of the data
through the MySQL server.
● Special features – There are a number of features that exist only in certain engine types, including full-text
search, referential integrity and the ability to handle spatial data.
NOTE: Storage engine independent

Most of the MySQL server operates in the same way no matter what storage engine is used: all the usual SQL
commands are independent of the storage engine. Naturally, the optimizer may need to make different choices
depending on the storage engine, but this is all handled through a standardized interface (API) which each storage
engine supports.

47 2.4.1 Transactional vs. Non-Transactional

Prior to discussing specific storage engines, it is important to discuss an important topic concerning a specific feature of
storage engines: the ability to handle transactions.

Transactional storage engines

Transactional storage engines provide data reliability during the update or insert of information by ensuring that data
added to the database is only committed when other conditions and stages in the application execution have completed
successfully. This is crucial in scenarios such as banks where information is taken out of (debited) one account and posted
(credited) to another account in a multi-step process. If either step failed, a transactional storage engine would cancel both
steps and the changes would be lost. If all the steps completed, the transactional engine would confirm it by committing
the changes to the database. The following is a summary of the benefits of transactional storage engines:
● Data safety - Even if MySQL crashes or has hardware problems, data can be recovered, either by automatic
recovery or from a backup plus the transaction log.
● Combined statements – Many statements can be combined and accepted all at the same time with the
COMMIT statement.

2-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

● Changes ignored – A ROLLBACK command can be used to ignore all the changes that were made prior to
committing to data.
● Failed updates protected - If an update fails, all the changes are reverted from the start of the transaction.
(With non-transactional tables, all changes that have taken place are permanent.)
A limitation to transactional storage engines is the performance overhead associated with managing the data integrity.
The following is a description of the ACID compliancy test for storage engines that claim to be transactional:
● Atomicity – Does the transactional storage engine ensure that all the tasks in a transaction are done, or none
of them? The transaction must be completed, or else it must be undone (rolled back).
● Consistency - Does every transaction preserve the integrity constraints -- the declared consistency rules -- of

Oracle University and BOS-it GmbH & Co.KG use only

the database? It cannot place the data in a contradictory state.
● Isolation – Does the transactional storage engine ensure that two simultaneous transactions do not interfere
with one another? Are intermediate results within a transaction kept separate from (or not visible to) other
transactions?
● Durability – Are completed transactions free from being aborted later or their results being discarded? They
must persist through restarts of the DBMS after crashes.

48 Non-transactional storage engines

For non-transactional storage engines, each transaction processed is immediately committed to the database and there
is no storage engine level ability to allow rollbacks of transactions. However, without the added overhead of
managing commits and rollbacks, non-transactional storage engines generally perform better than transactional
storage engines, have lower disk space requirements and use less memory to perform updates. However, the
disadvantage is the possibility of having half-run queries in the event of a crash or failed constraints in the middle of a
query. If transactional only statements (such as START TRANSACTION, ROLLBACK, etc.) are executed against a
table containing non-transactional storage engines, a warning is issued (because it can not perform the specific action)
but the queries are still executed against the server.

2.4.2 Table Storage Engines

This flexibility to choose how data is stored and indexed is a major reason why MySQL is so popular. The 'one size
fits all approach' involving other solutions means that either performance is sacrificed, functionality is diminished, or
"work arounds" requiring hours of fine tuning. With MySQL, the engine used is simply changed to meet the needs of
the specific table. The following are a series of charts describing the major features of each storage engine available
in MySQL 5.1, followed by a brief synopsis of each of the storage engines:

2-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

49

Oracle University and BOS-it GmbH & Co.KG use only

MyISAM
The MyISAM engine is the default engine in most MySQL installations and is a derivative of the original ISAM engine
type supported in the early versions of the MySQL server. The engine provides an impressive combination of
performance and functionality, although it lacks transactional capabilities. MyISAM provides high-speed storage and
retrieval, as well as full-text searching capabilities. MyISAM has the most flexible configurations for how to store and
cache data/indexes.

InnoDB
The InnoDB engine provides support for most of the database functionality associated with the MyISAM engine (Full-
text and GIS indexes are not supported) along with full transaction ACID compliant capabilities. The key to the InnoDB
system is a database, caching and indexing structure where both indexes and data are cached in memory as well as being
stored on disk. This enables very fast recovery, and works even on very large data sets. With row level locking, data can
be added to an InnoDB table without the engine locking the table with each modification, thus speeding up both the
recovery and storage of information in the database.

2-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

NDBCluster
The NDBCluster storage engine runs in server memory and supports transactions and synchronous replication.
“Synchronous replication" is within nodes in the cluster only and is automatic. The data is thus spread across many
possible servers. The engine spreads data redundantly across many nodes in the cluster. The storage engine allows one
of the nodes to go offline and does not interrupt data availability. The storage engine uses row-level locking. All reads
are non-locking by default. READ-COMMITTED is the only supported isolation level. As with clusters in general,
the NDB Cluster storage engine ensures individual node crashes do not stop the cluster. This storage engine provides
for automatic synchronization of data node at restart as well as recovery from checkpoints and logs at cluster crash.
The NDB Cluster storage engine can perform many maintenance tasks online, including on-line backup and on-line
software upgrades. In addition, the storage engine supports unique hash and T-tree ordered indexes.

Oracle University and BOS-it GmbH & Co.KG use only

50

MEMORY
The MEMORY storage engine (previously known as the HEAP storage engine) stores all data in memory and has no
footprint on disk; with that said, once the MySQL server has been shut down any information stored in a MEMORY
table will have been lost. However, the format of individual tables is kept, thus enabling the creation of temporary
tables that can be used to store information for quick access without having to recreate the tables each time the
database server is started. Other uses for the MEMORY storage engine include session handling, simulations and
summary tables.

2 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

ARCHIVE
The ARCHIVE storage engine acts as a long-term storage device for data that will not be modified in any way once it
enters into the database. With little overhead needed to only manage INSERT and SELECT statements, the information is
stored very efficiently due to it being compressed and non-modifiable. The ARCHIVE storage engine is perfect for
storing and retrieving log data or information that is no longer in active use. Complex searches against ARCHIVE tables
should be minimized due to the storage engine having to uncompress and read the entire table.

CSV
The CSV storage engine stores data in the form of a CSV (Comma Separated Values) file, not in a binary format. This is
not an efficient method for storing large volumes of data, or larger data types like BLOB, although such types are
supported. Since the data is stored in the CSV format, the files created are exceedingly portable and thus easy to import

Oracle University and BOS-it GmbH & Co.KG use only

into many different software packages, including Excel, OpenOffice and database systems like Access or FileMaker. The
CSV storage engine is the default engine used for MySQL log tables.

51

FEDERATED
The FEDERATED storage engine enables access to data from remote MySQL database tables as if they were local tables.
This storage engine in essence turns the MySQL server into a proxy for a remote server, using the MySQL client access
library to connect to the remote host, execute queries and then reformat the data into the localized format. This allows the
server, not a client, to access a remote database and can be used to combine data from multiple hosts or for copying
specific data from remote databases into local tables without the use of data exports and imports. The use of the
FEDERATED storage engine has an effect on performance, especially when executing complex queries . Data storage,
indexes and locking are all dependent on the remote tables defined storage engine.

2 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

BLACKHOLE
The BLACKHOLE engine does not actually store any data. Even though the storage engine is truly a "Blackhole" for
data, tables and indexes can be created and all SQL statements that would add or update information to the database
can be executed without actually writing any data. In addition, any locking statements issued against the storage
engine are ignored. With the BLACKHOLE storage engine, the database structure is retained thus allowing indexes
on the (non-existent) information. This provides an excellent testing ground for database structures without actually
creating any data against those structures. All SQL statements against a BLACKHOLE table are written to the binary
log which in turn can then replicated to slave servers.

2.4.3 SHOW ENGINES

Oracle University and BOS-it GmbH & Co.KG use only

SHOW ENGINES displays status information about the server's storage engines. This is useful for checking whether
52 a storage engine is supported, or to see what the default engine is. The following is a portion of what will be displayed
with the SHOW ENGINES command:

+------------+---------+-----------------------------+--------------+-----+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+------------+---------+-----------------------------+--------------+-----+------------+
| ndbcluster | YES | Clustered, fault-toler ... | YES | NO | NO |
| MRG_MYISAM | YES | Collection of identical ... | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage ... | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in ... | NO | NO | NO |
| FEDERATED | YES | Federated MySQL ... | YES | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| InnoDB | YES | Supports transaction ... | YES | YES | YES |
| MyISAM | DEFAULT | Default engine as of ... | NO | NO | NO |
+------------+---------+-----------------------------+--------------+-----+------------+

For the purpose of performance tuning, this list describes which storage engines can be utilized in the current server
configuration and which storage engine is the default for all new tables created within the server.
NOTE: Installing necessary storage engines

If there are storage engines not supported by the server and they will assist in performance, it is best to ensure
that the server has those storage engines installed prior to going live with an application. If applications are
already running on the server and there is a need to utilize a storage engine not available, it is best to perform
the operation in off hours, when a complete backup of the data can be taken, and a rebuild of the MySQL
server can be made with the storage engines required. It is best to build the MySQL server with only the
storage engines required to minimize the size of the build and the amount of memory the additional storage
engines will utilize.

2 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

InLine Lab 2-A

In this exercise you will examine the results of different storage engines upon inserts of data along with utilizing some of
the tools referenced up to this point. Two stored procedures will be created to support this lab. Install a fresh copy of the
world database to remove any indexes or other actions that may have altered the contents of the database.

Step 1. Create a new MyISAM table

Action: In the world database, create a table based on the City table by issuing the following command:

CREATE TABLE City_Test LIKE City;

Oracle University and BOS-it GmbH & Co.KG use only

Effect: This will create a new table based on the City table but with no data. Determine which storage engine is
being used by the City_Test table by issuing the following command:

SHOW CREATE TABLE City_Test;

Effect: The storage engine should be MyISAM. If it is not, alter the table to use the MyISAM storage engine by
typing the following command:

ALTER TABLE City_Test ENGINE=MyISAM;

Step 2. Prepare to load data

Action: Now the new table needs to be loaded with data; however, we wish to load lots of data. There are multiple
ways to accomplish this, but for class purposes, and to save time, we will create a stored procedure to perform the
task. Issue the following commands in the mysql client to create a stored procedure called Load_Data:

1. DROP PROCEDURE IF EXISTS Load_Data;

2. DELIMITER //
3. CREATE PROCEDURE Load_Data (r_value INT)
4. BEGIN
5. DECLARE v_count INT DEFAULT 0;
6. WHILE v_count < r_value DO
7. SET v_count = v_count + 1;
8. INSERT INTO City_Test SELECT NULL,
9. Name, CountryCode, District,
10. Population FROM City;
11. END WHILE;
12. END //
13. DELIMITER ;

NOTE: Due to the fact that there is no way to edit a specific portion of a stored procedure in the mysql client, it
is best to create the stored procedure code in an external application (such as Text Editor) for editing purposes.

2 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

The following explains this script line by line:

Line 1: This command will drop the procedure if it already exists.
Line 2: This line changes the standard termination for SQL commands to a double slash (//) to prevent conflicts
with the semicolon being used in the stored procedure.
Line 3: Identifies the name of the procedure along with an integer value that will be passed in (r_value).
Lines 4 and 12: The BEGIN ... END statements are used for stored procedures with multiple lines. On line 12,
the END is the last line that needs to be executed to create the stored procedure so the termination characters are
used (//).
Line 5: The variable v_count is declared and set to zero (0). This variable will be used to track the number of

Oracle University and BOS-it GmbH & Co.KG use only

repetitions the INSERT statement on line 8 will be executed.
Line 6: This is the WHILE statement that controls (along with the v_count variable) the number of repetitions
(r_value) that the INSERT statement on line 8 will execute.
Line 7: This statement advanced the counter (v_count) by one (1).
Lines 8 through 10: This is the INSERT statement that loads all of the records from the City table into the
City_Test table. The purpose of the NULL instead of the actual ID from the City table is to prevent duplicates.
Line 11: Terminates the WHILE statement
Line 13: Returns the SQL terminator back to a semicolon (;).

Step 3. Test loading data into MyISAM

Action: Execute the Load_Data script with 1000 repetitions by typing the following command in the mysql
client:

CALL Load_Data(1000);

Effect: This executes the script just created and loads approximately 4 Million rows of data into the
City_Test table. How long did it take to complete the script?

_________ seconds (should take approximately 30 seconds)

Action: Review the status of the table along with other important information by executing the following
command in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Effect: This displays valuable information about the City_Test table to include the engine type, the number
of rows and the length of the data. The Data_length field provides an educated guess of the size of the data
that is being stored.
NOTE: The SHOW TABLE STATUS is an approximation of the table information and multiple runs of this
same command may result in slightly different outputs.

2 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Action: Remove the data from the City_Test table by executing the following command in the mysql client:

TRUNCATE City_Test;

Effect: The TRUNCATE command when executed against a table is equivalent to dropping the table and recreating
it (which is faster than a row bye row delete). For InnoDB tables, the dropping and recreation of the table is not
performed if there are foreign key constraints referencing the table.
Step 4. Test loading data into ARCHIVE

Action: Change the storage engine for the City_Test table to ARCHIVE by typing the following command in the
mysql client:

Oracle University and BOS-it GmbH & Co.KG use only

ALTER TABLE City_Test ENGINE=ARCHIVE;

There are many fundamental differences between the different storage engines, with each having their advantages
and disadvantages. The next few steps will recreate the steps previously run on the City_Test table with the
ARCHIVE storage engine.
Action: Execute the Load_Data script with 1000 repetitions against the City_Table again by typing the following
command in the mysql client:

CALL Load_Data(1000);

Effect: This executes the script created earlier and loads approximately 4 Million rows of data into the
City_Test table. How long did it take to complete the script now that the City_Test table is using the
ARCHIVE storage engine?

_________ seconds (should take approximately 40 seconds)

Was there a big difference in the amount of time between the ARCHIVE storage engine and the MyISAM storage
engine?
Action: Review the status of the table along with other important information by executing the following command
in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Is there a difference in the Data_length field now that the ARCHIVE storage engine has been used?
Action: Attempt to remove all data from the City_Test table by executing the following command in the mysql
client:

TRUNCATE City_Test;

Effect: The ARCHIVE storage engine does not allow any deletion of the data (either by TRUNCATE or DELETE).

2 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Step 5. Test loading data into BLACKHOLE

Action: Change the storage engine for the City_Test table to BLACKHOLE by typing the following
command in the mysql client:

ALTER TABLE City_Test ENGINE=BLACKHOLE;

The data in the City_Test table could not be deleted while it was in the ARCHIVE storage engine; however,
the storage engine for the table could be altered. In this case, the storage engine chosen is the BLACKHOLE
storage engine.
Action: Review the status of the table along with other important information by executing the following

Oracle University and BOS-it GmbH & Co.KG use only

command in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Is there a difference in the Data_length field now that the BLACKHOLE storage engine has been used? ☺
Action: Execute the Load_Data script with 1000 repetitions against the City_Table again by typing the
following command in the mysql client:

CALL Load_Data(1000);

Effect: This executes the script created earlier and processes approximately 4 Million rows of data against the
City_Test table. How long did it take to complete the script now that the City_Test table is using the
BLACKHOLE storage engine?

_________ seconds (should take approximately 4 seconds)

Was there a big difference in the amount of time between the BLACKHOLE storage engine to the other storage
engines used up to this point?
Step 6. Test loading data into INNODB

Action: Change the storage engine for the City_Test table to INNODB by typing the following command in
the mysql client:

ALTER TABLE City_Test ENGINE=INNODB;

Effect: The City_Test table is now using the INNODB storage engine. The change to the new storage engine
should not have taken long due to the fact that there was no data in the table to process.
Action: In the mysql client type the following:

START TRANSACTION;

Effect: This will tell the client that we are starting the first steps of a transaction.

2 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Action: Execute the Load_Data script against the City_Table again by typing the following command in the
mysql client:

CALL Load_Data(1000);

Effect: This executes the script created earlier and processes approximately 4 Million rows of data against the
City_Test table. How long did it take to complete the script now that the City_Test table is using the InnoDB
storage engine?

_________ seconds (should take approximately 60 seconds)

Was there a big difference in the amount of time between the InnoDB storage engine to the other storage engines

Oracle University and BOS-it GmbH & Co.KG use only

used up to this point?
Action: Type the following command in the mysql client:

COMMIT;

Effect: This will commit all the INSERT transactions that were just issued.
Action: Review the status of the table along with other important information by executing the following command
in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Is there a difference in the Data_length field now that the InnoDB storage engine has been used?
In addition, the Data_free column identifies that there is an amount of capacity available in the InnoDB
tablespace (InnoDB Free: #### kb). This free amount identifies how much capacity is available in the InnoDB
tablespace and will increase as more data is added to tables connected to the InnoDB storage engine.
Action: Due to the fact that the InnoDB storage engine is much slower on inserts, it is necessary to use a smaller
data set to work with. At this point it is necessary to clean up the table and start fresh by typing the following in the
mysql client:

TRUNCATE City_Test;

The InnoDB storage engine processes the insertion of data differently due to it being ACID compliant. This effects
the time it takes to perform inserts; however, the advantage of ACID compliancy is definitely worth it for many
applications. There are steps to improve the insertion of data with tables that utilize the InnoDB storage engine that
will be discussed later in the course.
Step 7. Investigate Data and Index sizes

Action: Execute the Load_Data procedure (along with the transaction commands) against the City_Test again
but this time with only 250 repetitions by typing the following commands in the mysql client:

START TRANSACTION;
CALL Load_Data(250);
COMMIT;

2 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Effect: This executes the script that was just modified and loads approximately 1 Million rows of data into the
City_Test table.
Action: Review the status of the table along with other important information by executing the following
command in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

What is the size of the Data_length field and how many rows of data does the table contain?

______________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

Action: Add an additional index to the City_Test table on the Name column by typing the following in the
mysql client:

CREATE INDEX Name_Index ON City_Test(Name(2));

This index is based on the first two characters of the data in the Name column. The reason that only the first
two characters are being chosen for the index is the amount of time it takes to index any more characters in the
data. With that said, this process can still take a few minutes.
NOTE: For this lab, there may not be a large difference between indexing the whole column over the first two
characters; however, with larger data sets and more unique data this could make a big difference.

Action: Review the table of the City_Test table again to see how the index has affected the status of the
table by typing the following in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Has the size of the Data_length changed? Is there a field for the Index_length now? If so, what is its
size?

_________________________
Also, for the purposes of the next few steps, record the number of records (Rows) that the table contains:

__________________________
Action: Execute the following command in the mysql client to delete approximately 10% random records in
the City_Test table:

DELETE FROM City_Test WHERE RAND() < .10;

Effect: This process deletes approximately 10% of the records from the City_Test table. This process could
also take some time to complete.

2 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Action: Review the table of the City_Test table again to see how the deletion of the records has affected the
status of the table by typing the following in the mysql client:

SHOW TABLE STATUS LIKE 'City_Test'\G

Has the number of records (Rows) changed to reflect a deletion of records from the table? If so, what is the new
number of Rows?

__________________________________
If there was a deletion of records, did this affect the size of the Data_length or the Index_length?

Oracle University and BOS-it GmbH & Co.KG use only

NOTE: The Data_length and Index_length most likely did not reduce based on the deletions. In later
chapters we will discuss ways to regain physical space by optimizing the data structures after deletions of records.

2 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

2.5 Locks
When discussing locks it is important to see them as a synchronization mechanism for enforcing limits on access to a
53 resource in an environment where there are many threads of execution. Locks are one way of enforcing concurrency
control policies and maintaining data integrity. There are two types of locks:
● Shared (read) locks – many clients may read from the resource at the same time and not interfere with each
other. This type of lock allows other shared locks to be set.
● Exclusive (write) locks – only one client can write to the resource at a given time, and others are prevented
from reading while a client is writing. This type of lock prevents any other locks being set.

Locking Types

Oracle University and BOS-it GmbH & Co.KG use only

When discussing locks, there are two locking types that also need to be understood:
● Explicit locks – this type of locking is set explicitly set using a LOCK statement. It is important to be careful
with this locking type.
● Implicit locks – this type of locking is automatically set based on the operation that the MySQL server is
performing.

54 Locks granularity
In MySQL, there are two different levels of locking granularity implemented by different storage engines:
● Table locks – This is a when a table as a whole is locked on an all-or-nothing basis. When a client wants to
make a change to the data (INSERT, DELETE, UPDATE, etc.) the entire table is locked from write or reads
from any other client.
● Row locks – This is when an individual row is locked from reads or writes from the other clients; however,
due to the complexity of performing this type of lock it is the most computationally expensive, with the
greatest possible concurrency from the other clients perspectives.
NOTE: Storage engine dependent

The algorithms used by MySQL concerning locking methods are completely dependent upon the storage
engines that are being utilized. Each storage engine is very specific in its implementation of locks and will be
discussed in detail later in this course.

Locking Issues
Three locking issues that arise with applications using MySQL are:
● Blocking locks – Locks that prevent another thread from performing work, will be dependent on the locking
mechanism via the chosen storage engine.
● Deadlocks - This is a specific condition when two or more processes are each waiting for another to release
a resource, or more than two processes are waiting for resources in a circular chain. For every resource
request, the server sees if granting the request will mean that the system will enter an unsafe state, meaning a
state that could result in deadlock. The system then only grants requests that will lead to safe states. This
type of condition can not occur on storage engines that only support table level locking.

2 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

NOTE: Deadlock prevention

Deadlock prevention is not only a function of the storage engine. There is some application design that needs to be
considered also. For example, operations should always be done in the same order. (e.g.. If a transaction updates A
and then B, whereas another transaction updates B and then A, the system would be prone to deadlocks).
● Locks on higher level than necessary - Locks can sometimes be acquired at a higher level than necessary,
usually when working with explicit locks versus implicit locks. Generally, this is an application issue.

Oracle University and BOS-it GmbH & Co.KG use only

2 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

Further Practice
In this further practice, you will use different storage engines to determine the performance differences and size
differences.
1. In the world schema create a new table called city_huge that contains the structure and all the data from
City table.
55
2. Increase the number of records in the city_huge table by inserting all the records from the city_huge
table between four (4) and eight (8) times. The data in the city_huge table will grow exponentially as
you run the INSERT command against and from the city_huge table. The last two executions should
start to run slower. Depending on the speed of your hardware, adjust the number of repetitions to challenge

Oracle University and BOS-it GmbH & Co.KG use only

it.
3. Create a copy of the city_huge table using other storage engines: MEMORY, ARCHIVE, MYISAM (if not
the default) , INNODB (if not the default). For MEMORY, you will likely need to increase the
max_heap_table_size variable to fit the data in memory.
◦ Note: To maintain the index of the city_huge table, it is best to perform a three step process when
creating the tables: create the table, alter the engine of the table created, and then insert the records into
the table.
4. Search each table created for all records where the city name is “Paris” and compare the average response
times. If the response times differ, why?

__________________________________________________________________________________

__________________________________________________________________________________

5. Search each table created for all records where the city identification number (ID) is equal to 123456 and
compare the average response times. If the response times differ, why?

__________________________________________________________________________________

__________________________________________________________________________________

6. Using the SHOW TABLE STATUS, review the Data_length and the other information associated with
the tables created in step 3.

2 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 2 - MySQL Architecture

2.6 Chapter Summary

56 In this lesson, you have learned to:
● Describe the MySQL architecture at a high level
This included discussions on the core shared subsystems, storage engine abstraction, query parsing, optimization
and execution, and the query cache.
● Describe the client/server architecture
Client-server communication can be accomplished via: TCP/IP, Unix Socket, and Named Pipes
● Describe MySQL storage engines

Oracle University and BOS-it GmbH & Co.KG use only

MySQL supports several storage engines that act as handlers for different table types: MyISAM, InnoDB,
NDBCluster, MEMORY (formerly known as HEAP), MERGE, FEDERATED and ARCHIVE.
● Describe locks and how they affect performance
There are two types of locks: read (shared) and write (exclusive) that support table and row locks (depending on
the storage engine).

2 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

57
3 MYSQL PERFORMANCE TOOLS

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

58 3.1 Learning Objectives

This chapter explains the MySQL performance tools and how they relate to performance tuning. In this lesson, you
will learn to:
● Evaluate the most common benchmarking tools used with MySQL
● Utilize the monitoring tools available for MySQL
● Demonstrate knowledge of MySQL’s administration tools

Oracle University and BOS-it GmbH & Co.KG use only

3-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

3.2 Benchmarking Tools

There are a number of benchmarking tools that are compatible with MySQL and when taken as a whole can develop a
benchmark foundation. The following tools will be highlighted during this session:
● mysqlslap
● Super Smack
● MyBench
● SysBench
● Web Tools (ApacheBench and httperf)

Oracle University and BOS-it GmbH & Co.KG use only

59 3.2.1 mysqlslap
mysqlslap is a diagnostic program, created by MySQL AB, that is designed to emulate client load for a MySQL server
and to report the timing of each stage. This diagnostic progem can be configured to emulate multiple clients accessing the
server at one time. mysqlslap is a part of the MySQL 5.1 server binaries.

Load generation application

mysqlslap is a three stage application which includes the creation of a schema, the loading of the data needed to
perform the benchmark, and a general purpose query of the data. Any of the stages can be overwritten, such as using
existing data or executing queries specified directly from the command line (or provided in a file). If a file is specified, it
can only contain one statement per line and can not contain comments (mysqlslap does not know how to handle
them).

Minimum arguments
The minimum that is required to run the mysqlslap diagnostic program includes the executable itself (entered on the
operating system command line) along with a user identified that is allowed to create a database (and a password if one is
assigned to the user):

shell> mysqlslap -uusername [-p]

When executed with the minimum number of arguments, mysqlslap will create a schema called mysqlslap, load the
schema with data (by creating a table called t1 to contain the data) and then will execute a general purpose query against
that data. The result will look similar to the output listed below:

shell> mysqlslap -uroot

Benchmark
Average number of seconds to run all queries: 0.003 seconds
Minimum number of seconds to run all queries: 0.003 seconds
Maximum number of seconds to run all queries: 0.003 seconds
Number of clients running queries: 1
Average number of queries per client: 0

NOTE:: The mysqlslap database will need to be created prior to executing mysqlslap as described.

3-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

Command line options

60
mysqlslap is designed to provide flexibility in the processing of data to provide the most useful results for the end
user. This flexibility is based on the numerous command line options that mysqlslap supports:
● --help, -? - displays the complete list of command line options that are available for use with the
application
● --user=user_name, -u user_name - this option identifies the user that the application should
connect to the MySQL server as.
● --password[=password], -p[password] - this option is used to specify that a password is needed
for the user_name that is being used to connect to the MySQL server. Even though it is possible to add a

Oracle University and BOS-it GmbH & Co.KG use only

password to the command line, it is not a safe practice and should be avoided. If a password is not supplied
on the command line, the -p or --password option will prompt the user to enter a password (this is the
preferred and secure option).
● --auto-generate-sql, -a - this option generates a random SQL statement to be executed against the
data to provide a reasonable facsimile of what a real application would have to deal with.
● --iterations=N, -i N - this option identifies the number of times that the test should be run. If this
option is not provided, the default is 1 time. Only one numeric value can be placed in this option.

61 ● --concurrency=N, -c N - when using a SELECT statement, this option identifies the number of clients
that would be accessing the data at the same time. The N can be a single numeric entry or can be multiple
numeric entries by separating the numeric values with a comma as shown in the example below:

shell> mysqlslap -uusername -p -a -i 10 --concurrency=1,4,16,64,256

NOTE: It may be necessary to increase the size of max_connections prior to executing the
example described above.
● This will result in 5 runs (identified by --concurrency) of the application (and 5 result sets); the first run
will simulate only one client accessing the SQL, followed by a simulation with 4 concurrent clients
accessing the data, then 16, 64, and 256 concurrent clients. The following graph represents how the number
of concurrent clients, along with 10 iterations of each statement, may affect the response times of the
MySQL server:

3-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

Default Storage Engine

10 Iterations of random INSERT and SELECT statements against server
4.00
Response Time (Seconds)
3.50
3.00
2.50
Average
2.00
Minimum
1.50
Maximum
1.00

Oracle University and BOS-it GmbH & Co.KG use only

0.50
0.00
1 4 16 64 256
Concurrent Operations

NOTE: What is mysqlslap doing when there is no schema or query identified?

When a schema or query is not provided to the mysqlslap application, mysqlslap will create a database
called mysqlslap with a table called t1 which includes an INT column and VARCHAR(128) column. With
this database and table created, mysqlslap then performs multiple random inserts against the table (to fill the table
with data), followed by multiple random SELECT and INSERT statements.

62 ● --create-schema - this option identifies the schema that will be created to execute the mysqlslap
application against.
● --create - this option identifies the file or string that will be used to create the table that will utilize.
● --query, -q - this option identifies the query string or file that mysqlslap will execute against the data.
● --only-print - this option tells mysqlslap not to execute against the actual data but rather just print what
it would have done if it actually executed. The following example below identifies how the previous components
would work together:

shell> mysqlslap -uuser_name -p --create-schema="Dog"

--create="CREATE TABLE Puppy (age INT, hair_color CHAR(10))"
--query="INSERT INTO Puppy VALUES (7,'Black'),(4,'Golden')"
--only-print

The will result in the following being displayed as the output to the terminal window:

3-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

DROP SCHEMA IF EXISTS `Dog`;

CREATE SCHEMA `Dog`;
Use Dog;
CREATE TABLE Puppy (age INT, hair_color CHAR(10));
INSERT INTO Puppy VALUES (7,'Black'),(4,'Golden');
DROP SCHEMA IF EXISTS `Dog`;

● --csv[=file] - this option creates an output in the form of comma-separated values. The output can be
passed into an optional file or to the screen itself if no file is identified.
● --engine=engine_name, -e engine_name - this option can be used when creating tables by

Oracle University and BOS-it GmbH & Co.KG use only

providing the means to control which storage engine the table will utilize. Multiple storage engines can be
selected by separating the engine names with commas as shown in the example below:

shell> mysqlslap -uuser_name -p -a -i 10 --concurrency=1,4,16,64,256

--engine=MyISAM,ARCHIVE,InnoDB

● Similar to the previous example, 5 runs of the application will be executed but this time in three groups. The
first group will contain those results for the MyISAM storage engine. The second group will contain those
results for the ARCHIVE storage engine. The third, and last group, will contain the results using the InnoDB
storage engine. The following graph represents how the number of concurrent clients, storage engines and
number of iterations may affect the average response time of the MySQL server

63
Multiple Storage Engines
10 Iterations of random INSERT and SELECT statements against server

7.00
Average Response Time

6.00

5.00
(Seconds)

4.00
MyISAM
3.00
ARCHIVE
2.00
InnoDB
1.00

0.00
1 4 16 64 256
Concurrent Operations

3-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

Other command line options

64
There are numerous command line options that mysqlslap allows; some are more general while other are more
specific to the benchmark that is being performed This following is a listing of some of the other command line options
(along with a brief description) that mysqlslap allows:
● --compress, -c - this option supports compression between the client and the server if they both support this
feature.
● --delimiter=str, -F str - this option allows a delimiter to be selected for files read in that may not use
the standard delimiter.
● --host=host_name, -h host_name - this option will identify the host to connect to (host_name). By

Oracle University and BOS-it GmbH & Co.KG use only

default, the host_name is localhost.
● --number-char-cols=N, -x N - this option identifies the number of VARCHAR columns that will be used
with the --auto-generate-sql. By default, only one VARCHAR column is used.
● --number-int-cols=N, -y N - this option identifies the number of INT columns that will be used with the
--auto-generate-sql. By default, only one INT column is used.
● --number-of-queries=N - this option limits the number of queries that will be executed.
● --port=port_num, -P port_num - this option identifies the port number to use when connecting to the
mysql server.

65 ● --protocol={TCP|SOCKET|PIPE|MEMORY} - this option identifies the connection protocol to use.

● --silent, -s - this option will execute the mysqlslap application without any output to the terminal.
● --socket=path, -S path - this option identifies the UNIX socket file to use when connection to a MySQL
server running on the localhost. (In Windows, this would be the name of named pipe to use).
● --version, -V - this option displays the version of mysqlslap along with the version of the MySQL server.

3-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

66 3.2.2 Super Smack

Super Smack is a powerful, customizable benchmarking tool that provides load limitations, in terms of queries per
second, of the benchmark tests it is supplied. In relation to mysqlslap, Super Smack is the more robust
benchmarking tool due to the following advantages:
● Super Smack has the ability to generate data, either from existing data or built-in Super Smack data.
● Super Smack has the ability to use dictionary files with replacement of parameters (ie. Using SELECT *
FROM TBL WHERE ID = $NUM with Super Smack picking $NUM from a file at random or sequential to
use).

Oracle University and BOS-it GmbH & Co.KG use only

● Super Smack provides a break down of time spent per query set.
● With Super Smack it is easy to adjust the frequency of different query patterns.
● Super Smack has the ability to have different connections perform different sets of queries.
Super Smack does not come with MySQL. It must be downloaded and built on the individuals operating system prior
to testing. As of this writing, the current release is available from Tony Bourke at http://vegan.net/tony/supersmack/.
NOTE: Super Smack details

Appendix F gives a detailed step-by-step approach to configuring and executing the SuperSmack application.
Please refer to that section if you wish to pursue using the application.

3-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

67 3.2.3 MyBench
This benchmarking tool handles the details of spawning clients along with gathering and computing statistics. This is a
Perl-based system which is hard on memory and should be run on a different machine from the MySQL server that it is
testing.

Perl Modules
The MyBench.pm modules contain the common logic. To use MyBench, the following Perl modules will need to be
installed:
● DBI

Oracle University and BOS-it GmbH & Co.KG use only

● DBD:mysql
● Time:HiRes
NOTE: When is MyBench needed?

MyBench is useful for running a benchmark with complicated logic, such as running a script that processes a
number of statements inside a transaction or relying on SQL in line variables. MyBench is the more flexible
benchmarking tool to meet these needs.

3-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

68 3.2.4 SysBench
SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating O/S parameters that are
important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an
impression about system performance without setting up complex database benchmarks or even without installing a
database at all.
The SysBench current feature set allows for a testing of the following system parameters:
● File I/O performance
● Scheduler performance

Oracle University and BOS-it GmbH & Co.KG use only

● Memory allocation and transfer speed
● POSIX threads implementation performance
● Database server performance

Test Modes
SysBench can be utilized to test a number of different modes within MySQL. The following is a list of the modes that
can be benchmarked:
● cpu - In this mode each request consists in calculation of prime numbers using 64-bit integers. Each thread
executes the requests concurrently until either the total number of requests or the total execution time
exceeds the limits specified with the common command line options.
● threads - This test mode was written to benchmark scheduler performance, more specifically the cases when
a scheduler has a large number of threads competing for some set of mutexes.
● mutex - This test mode was written to emulate a situation when all threads run concurrently most of the
time, acquiring the mutex lock for only a short period of time (incrementing a global variable). So the
purpose of this benchmark is to examine the performance of mutex implementation.
● memory - This test mode can be used to benchmark sequential memory reads or writes. Depending on
command line options each thread can access either a global or a local block for all memory operations.
● fileio - This test mode can be used to produce various kinds of file I/O workloads. At the prepare stage
SysBench creates a specified number of files with a specified total size. Then at the run stage, each thread
performs specified I/O operations on this set of files.
● oltp - This test mode was written to benchmark real database performance. At the prepare stage a table is
created in the specified database (sbtest by default). Then this table is filled with a specified number of
rows. Then the benchmark is run with a number of client threads, limiting the total number of requests by a
defined number.

3 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

3.3 Administration Tools

MySQL provides many options for administering and monitoring the performance of the server. In the majority of these
tools, the key is not just to retrieve the information but the ability to interpret the information in relation to performance
improvement. The following are some of the most useful MySQL administration and monitoring tools:
● Server logs
● mysqladmin
● MySQL Administrator
● Specific SHOW commands

Oracle University and BOS-it GmbH & Co.KG use only

3.3.1 Server Logs
69
There are numerous logging options within the MySQL server to help database administrators and developers alike
monitor performance and administer the server. These logs can be viewed either through one of the operating systems
editor's or MySQL Administrator (discussed in the next section). The following is the list of logs available from the
MySQL server:

General Query Log

The general query log is beneficial to performance tuning because it tracks all queries that have been submitted on the
MySQL server. The general query log is a general record of what mysqld is doing. The server writes information to this
log when clients connect or disconnect, and it logs each SQL statement received from clients. The general query log can
be very useful when an error in a client is suspected to know exactly what the client sent to mysqld.
NOTE: Query log order

mysqld writes statements to the query log in the order that it receives them. This may be different from the order
in which they are executed. To enable the general query log, add --log[=file_name] to the startup
configuration file. If no file_name value is given, the default name is host_name.log in the data directory.

Slow Query Log

The slow query log consists of all SQL statements that took more than the long_query_time system variable value in
seconds to execute. The time to acquire initial table locks is not counted as execution time. The minimum and default
values of long_query_time are 1 and 10, respectively. The slow query log can be used to find queries that take a
long time to execute and are therefore candidates for optimization.
NOTE: Enabling slow query log

mysqld writes a statement to the slow query log after it has been executed and after all locks have been released.
Log order may be different from execution order. To enable the slow query log, add --log-slow-
queries[=file_name] to the startup configuration file. If no file_name value is given, the default is the
name of the host machine with a suffix of -slow.log. If a filename is given, but not as an absolute pathname,
the server writes the file in the data directory.

3 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

Queries that do not use indexes are logged in the slow query log if the --log-queries-not-using-indexes
option is specified. In addition, the --log-slow-admin-statements server option enables logging of slow
administrative statements such as OPTIMIZE TABLE, ANALYZE TABLE, and ALTER TABLE to the slow query
log.
Queries handled by the query cache are not added to the slow query log, nor are queries that would not benefit from
the presence of an index because the table has zero rows or one row.

Binary Log
The binary log, which replaces the earlier update log, is primarily used for recovery and replication; however, it also
helps performance tuning by providing a record of all changes that were made to the database.

Oracle University and BOS-it GmbH & Co.KG use only

Statements are stored in the form of “events” that describe the modifications. The binary log also contains
information about how long each statement that updated data took.
NOTE: Slower performance

Running the server with the binary log enabled makes performance about 1% slower; however, the advantages
to recovery, replication and performance monitoring far outweigh the performance drain. To enable the binary
log, add --log-bin[=base_name] to the startup configuration file. It is recommended that a basename
be specified.

Error Log
The error logs keeps a record of major events such as server start/stop, as well as any serious errors. In addition, if
mysqld notices a table that needs to be automatically checked or repaired, it writes a message to the error log.
NOTE: Naming the error Log

To specify where mysqld stores the error log, add --log-error[=file_name] to the configuration file.
If no file_name value is given, mysqld uses the name host_name.err and writes the file in the data
directory. If a FLUSH LOGS command is executed, the error log is renamed with the suffix -old and mysqld
creates a new empty log file. No renaming occurs if the --log-error option is not given.

3 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

70 3.3.2 mysqladmin
This monitoring tool provides a host of administrative and diagnostic functions for maintaining and tuning the MySQL
database server. mysqladmin is executed from the operating systems command prompt and is located in the bin
directory under the root path of the MySQL server. The following is a list of some of the more important performance
monitoring and management mysqladmin commands:
● extended-status – This command provides a listing of server status variables from the server. The list is
lengthy and provides a great deal of information that can be used for monitoring the performance of the server.
Within the MySQL command line client, SHOW STATUS is equivalent to mysqladmin extended-
status.

Oracle University and BOS-it GmbH & Co.KG use only

● Flush commands
Flush commands store any unsaved buffers to permanent storage. Each of the flush commands can be useful for
clearing the slate of their respective purposes, providing performance monitoring tasks a starting point to track
problems and/or the effect changes will have on the server. The RELOAD privilege is required for users who
wish to use the flush commands.
○ flush-hosts – This commands clears the cached hosts. This can be useful when there are a number of
hosts that have been blocked because of incomplete requests.
○ flush-logs – This command clears all the server logs. There are four types of logs that MySQL uses:
■ error log - Problems encountered starting, running, or stopping mysqld.
■ general query log - Established client connections and statements received from clients.
■ binary log - All statements that change data (also used for recovery and replication).
■ slow log - All queries that took more than long_query_time seconds to execute.
● flush-status – This command resets all status variables. This is most useful for debugging queries.
● Flush-tables – This command forces all tables to be closed. The main purpose of the flush tables command
is to ensure that new tables (altered outside of MySQL) are used by all threads. However, for the purpose of
performance monitoring, this can ensure that all table changes are flushed to disk prior to making any changes to
the server.
● flush-threads – This command clears all threads in the cache.
● flush-privileges (reload)– This command reloads the grant tables into server memory.
● processlist – This command shows which threads are currently running on the server. To see all the
threads, the SUPER privilege is required. Without the SUPER privilege, the user initiating the command will see
only their own threads.
● kill id, id, ... – This command releases (or kills) specific MySQL threads. The id to use comes from the
processlist command located under the ID column.
● refresh – This command flushes all tables and open log files.
● variables – This command displays the system variables and their values.

3 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

71 3.3.3 MySQL Administrator

The MySQL Administrator is a valuable tool in the database administrator's toolbox for monitoring the performance
and administration of the MySQL server. This is a graphical user interface (GUI) application and provides valuable
information and control over the server and its instances. There are a number of sections within the application and
each one is explained briefly below:
● Server Information - Information about the MySQL server that is running, MySQL Administrator, the
machine that it runs on, and the connection.
● Service Control - Start and stop the MySQL server. On Windows, the MySQL service is also configurable.

Oracle University and BOS-it GmbH & Co.KG use only

● Startup Variables - Configure the startup variables for the MySQL server.
● Server Connections - View or kill current connections to the MySQL server (also called threads).
● User Administration - Administer existing users, add new users, and delete existing users.
● Health - Graphical displays of a number of usage or hit rate values that affect server performance, and a
hierarchical view of system and status variables.
● Server Logs - View log file entries.
● Backup - Plan and administer backup projects, select the databases to be backed up, and start the backup
process.
● Restore - Restore databases from existing backups.
● Replication Status - View information about replication master and slave servers.
● Catalogs - View information about databases, tables, columns, indexes, and rows; optimize tables.

3 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

72 3.4 Specific SHOW commands

The following SHOW commands (which can be used in the mysql client) that have a bearing on performance, either for
specific storage engines or for the server in general, include the following:
● SHOW PROCESSLIST– The SHOW PROCESSLIST command provides a real-time list of all the connections
to the server and the actions that each one is performing. With the SUPER privilege, the SHOW PROCESSLIST
provides the connection identifier, the user, the host the user is coming from, the database the user is using, the
current command being run, along with the time and state for each open connection.
○ INFORMATION_SCHEMA.PROCESSLIST - This table, which is a non-standard table, is another way of
displaying information about which threads are running.

Oracle University and BOS-it GmbH & Co.KG use only

Using the SHOW PROCESSLIST is an extremely helpful tool in identifying the current processes running on
the server and if needed provide the information to terminate processes that are affecting the servers
performance as a whole. There are two levels of terminating (killing) a current process:
○ KILL <connection_id> - With the connection identifier obtained from the SHOW PROCESSLIST
listing, the command KILL followed by the connection identifier will terminate the connection altogether.
○ KILL QUERY <connection_id> - With the connection identifier obtained from the SHOW
PROCESSLIST listing, the command KILL QUERY followed by the connection identifier will terminate
the query that the connection is currently executing.
NOTE: Automated Monitoring

An alternative to running SHOW PROCESSLIST within the MySQL client is to use "mysqladmin
processlist" in the operating system command prompt. This can be useful for setting up some type of
automated monitoring of the MySQL server.
● SHOW STATUS – SHOW STATUS provides server status information. This information can also be obtained
using the "mysqladmin extended-status" command. With the GLOBAL modifier, SHOW STATUS
displays the status values for all connections to MySQL. With SESSION, it displays the status values for the
current connection.
● SHOW VARIABLES - SHOW VARIABLES provides system variables information. This information can also
be obtained using the "mysqladmin variables" command. With the GLOBAL modifier, SHOW
VARIABLES displays the values that are used for new connections to MySQL. With SESSION, it displays the
system variables values for the current connection.

3 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 3 - MySQL Performance Tools

73 3.5 Chapter Summary

In this lesson, you have learned:
● To evaluate the most common benchmarking tools used with MySQL
mysqlslap, Super Smack, MyBench and SysBench.
● To utilize the monitoring tools available for MySQL
The following are some of the most useful MySQL monitoring tools: server logs, mysqladmin, SHOW
ENGINES, SHOW VARIABLES, SET/SELECT @@ and other SHOW commands.
● To demonstrate knowledge of MySQL’s administration tools

Oracle University and BOS-it GmbH & Co.KG use only

The most common and most useful tools in the MySQL arsenal for administration are: MySQL
Administrator and MySQL Query Browser

3 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

74
4 SCHEMA/DATABASE DESIGN

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

75 4.1 Learning Objectives

This chapter explains how the design of a database (schema) can support performance. In this lesson, you will learn
to:
● Explain the purpose of data modeling
● Understand normalization and how to normalize the data in your databases
● Understand denormalization and why you would want to denormalize the data in your database
● Describe how MySQL data types affect performance
● Understand how partitioning can improve performance

Oracle University and BOS-it GmbH & Co.KG use only

4-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

76 4.2 Data Modeling

Data Modeling is the process of structuring and organizing data. Data models describe structured data for storage in data
management. The data modeling approach can not be understated in the design of a database and its future performance.
Data modeling is crucial to ensure:
● Redundancy control in stored data and in development and maintenance efforts.
● Multiple users being able to share data.
● Users with different needs are given the correct access and interface to the data.
● Referential integrity is maintained.

Oracle University and BOS-it GmbH & Co.KG use only

Data Modeling Tools
There are two basic types of data modeling tools that are most commonly used during the data modeling process to
describe information needs and/or the type of information that is to be stored in the database during the requirements
analysis: entity relationship diagrams (ERD) and unified modeling language (UML). Each of these bring value separately
or in conjunction with each other.
77 ● Entity relationship model - This model assists with laying out the architecture of a relational database by
defining the entities, attributes and relationships of the data.
○ Entity - This represents the information that provides the meaning for the database by answering the
questions that the end user is asking the database to provide.
○ Attribute - This represents the actual data components that give the entities its ability to answer questions
asked. If a question can not be answered, consider adding more attributes (or entities) to sufficiently answer
the question.
○ Relationship - This represents the connections between the entities that provide the structure for the
schema. If any entity does not have a relationship with another entity, consider if that entity is required.

Last First
Phone Name Length Level
Name Name

Instructor
Instructor teaches Course

nts Enough
de
Stu tend seats?
at

Students Classroom

Last First Operating Room

Phone Seats
Name Name System Number

An ERD will draw out possible many-to-many relationships, ambiguities in the data, the need for additional
entities or attributes, and the relationship level required to enforce business rules.

4-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

78 ● Unified modeling language - The unified modeling language consists of many components and structures
that are designed to display relationships between "data" in large systems. This data can be anything from
resources to processes along attributes in a database. For the purposes of relational database structuring, only
a small subset of the UML is utilized. The following is an example of a schema diagramed with UML:

Instructor
Instructor Course
Course
InstructorID
InstructorID CourseID
CourseID
LastName
LastName Teaches ► CourseCode
CourseCode
FirstName
FirstName 0..* 1..1 CourseName
CourseName
Phone Category

Oracle University and BOS-it GmbH & Co.KG use only

Phone Category
StartDate
StartDate SubCategory
SubCategory
TermDate
TermDate CourseMaterial
CourseMaterial
ExperienceLevel
ExperienceLevel Duration
Duration

NOTE: MySQL Workbench

MySQL Workbench is a visual database design tool that gives database users the ability to integrate database
design, modeling, creation and maintenance into a single, seamless environment. This tool can be downloaded
from http://dev.mysql.com/downloads/

4-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

79 4.3 Normalization
Normalization is a design technique that is widely used as a guide in designing relational databases. Normalization is
essentially a two step process that puts data into tabular form by removing repeating groups and then removing duplicated
data from the relational tables. The purposes of normalization are:
● Eliminate redundant data – Removing entries of repeated data is a key to the success of normalization and
ensures integrity by keeping individual data in one location, making it easier to update and manage.
● Eliminate columns not dependent on key – Columns that are not dependent on a key can get disconnected
from data and cause data corruption. By having all data connected to a key identifier, data integrity is ensured.
● Isolate independent multiple relationships – With relationships of data across multiple tables, normalization

Oracle University and BOS-it GmbH & Co.KG use only

provides a cleaner recognition of these relationships and ensures that all relationships are identified and applied.

Advantages
Data normalization can be an expensive process in time and resources; however, besides the advantages derived from the
purposes of normalization, there are other advantages:
● ER diagrams – Developing ERD’s are much easier when the data is normalized making the relationships easily
identifiable.
● Compact – By having the data normalized it is easier to modify a single object property. If an attribute of a table
had student genders labeled as boy or girl and this was identified as incorrect labeling of students, it would be
easy to change boy to male and girl to female if there is a normalized table called gender.
● Joins – Joins can be expensive, and proper normalization can improve relationships between tables and
minimize the amount of data having to be searched.
● Optimizer – With normalization, the optimizer choices for selection are limited and it is easier for the optimizer
to pick the best solutions.
● Updates - It is easier to update in one location versus multiple locations.

Disadvantages
Even with the numerous advantages associated with normalization, there are some disadvantages:
● Numerous tables - Multiple tables must be accessed for user reports and other user data.
● Maintenance - Maintenance may be in some conflict with the business processes.
80
4.3.1 Anomalies and Ideal Fields
In designing a database, there are times when anomalies will be evident within table structures based on the fields within
those tables. Some of the anomalies that can occur from poorly designed table structures are duplicate data, redundant
data and difficulties in using the data. When normalizing, it is imperative to ensure that any potential anomalies are
caught ahead of time. For example, redundancy in a database can cause what are called update anomalies. Update
anomalies are problems that arise when information is inserted, deleted, or updated.
● INSERT - There are times that the database design will force one table to have associated data in another table
to actually insert the desired data. Example: a customer could be prevented from being entered into a database
system without first ordering a product.

4-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

● DELETE - When deleting data from one table, associated data in other tables may be deleted. Example:
Inventory statistics and values could be affected when a supplier row is deleted (eg. supplier goes out of
business) and the associated tables containing inventory from that supplier are deleted.
● UPDATE - This occurs when one field, or multiple associated fields, require an update causing multiple
associated rows requiring updates as well. Example: A supplier moves to a larger facility and their address
needs to updated in multiple tables, thus causing the potential for one or more tables being missed. Proper
normalization prevents this from being a problem.
Eliminating these anomalies is a central advantage to normalization and a central idea in normalization is using the
correct fields in each table. The best way to detect fields that may lead to these anomalies is ensuring that each field
designed meets the following criteria:

Oracle University and BOS-it GmbH & Co.KG use only

● Represents a characteristic of the table subject - Tables should be designed to address a subject of the
complete database. The fields within those tables should represent the subject of the table they are in. For
example, a table representing a dinner menu would not include a shoe size of the guests.
● Contains only a single value - If a field contains more than one value (multi-valued fields), it will become
apparent that the data contained is more difficult to work with and the potential for data redundancy is high.
● Can not be deconstructed into smaller components - In a field that contains multiple potential separate
data elements (multipart field), it is best to deconstruct the field into smaller, separate fields. An example of
this would be an address field which could be broken down to Street_Number, Street_Direction (N, E, W, S,
etc.), Street_Name, Street Identifier (Drive, Court, Lane, etc.), rather than being one field that contains all
those elements.
● Does not contain a calculated or concatenated value - Fields should not have to depend on their value
being derived from other field data. If using calculated or concatenated fields, unnecessary business
processes have to be put into place to ensure that associated field modifications are passed to these fields.
● Unique within the entire database structure - Ensuring that a field in one table does not have a redundant
or duplicate field in another table eliminates the anomalies associated with modifying or deleting identical
data in more than one place.
● Identical field types in relationship - When a field is used in more than one table to create relationships
between the tables, it is crucial that they are identical down to every detail, including field type.

4.3.2 Normalization Forms

Normalization has many levels, with each successive level providing stronger guarantees about the data modification
anomalies being eliminated. Each level, or form, depends on the previous level and it is best practice to work through
each form individually to achieve the best results. There are numerous levels to normalization; however, the first three
levels are the most common and are explained in detail below:
81 ● First Normal Form (INF) – A table that is in the first level of normalization contains no repeating groups
within rows.

4-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

Inventory
sID sLoc sPostal pID1 pName1 pQty1 pID2 pName2 pQty2
1 Holtsville 00501 1 bed 15 2 chair 4
2 Waukesha 53146 1 bed 4 3 table 6
3 Waukesha 53146 2 chair 8 4 sofa 4
4 Ketchikan 99950 2 chair 24 4 sofa 10

1NF

Inventory_1NF

Oracle University and BOS-it GmbH & Co.KG use only

sID sLoc sPostal pID* pName pQty
1 Holtsville 00501 1 bed 15
1 Holtsville 00501 2 chair 4
2 Waukesha 53146 1 bed 4
2 Waukesha 53146 3 table 6
3 Waukesha 53146 2 chair 8
3 Waukesha 53146 4 sofa 4
4 Ketchikan 99950 2 chair 24
4 Ketchikan 99950 4 sofa 10

* This column would not usually appear in a first normal form;however,

it is here to demonstrate a connection to second normal form.

This table is now in first normal form; however, there is still a level of redundancy that can be removed through
greater normalization. Each row contains the supplier identification number (sID), the supplier location (sLoc)
and the supplier postal code (sPostal), when all that is really needed is just the suppliers identification number.
This normalization level can result in inconsistencies and data integrity issues during updates and deletes.
Deleting all the rows associated with a supplier can result in the supplier's information being completely erased.
When having to update supplier information, every record associated with the supplier needs to be updated.
82 ● Second Normal Form (2NF) – A table is in this level of normalization has been normalized at the first level
(1NF), plus every non-key (supporting) value is dependent on the primary key value. In addition, the latter
constraint means that a non-key value must depend on every column in a composite primary key.

4-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

Inventory_1NF
sID sLoc sPostal pID pName pQty
1 Holtsville 00501 1 bed 15
1 Holtsville 00501 2 chair 4
2 Waukesha 53146 1 bed 4
2 Waukesha 53146 3 table 6
3 Waukesha 53146 2 chair 8
3 Waukesha 53146 4 sofa 4
4 Ketchikan 99950 2 chair 24
4 Ketchikan 99950 4 sofa 10

Oracle University and BOS-it GmbH & Co.KG use only

Part_1NF
2NF
sID pID pName pQty
1NF 1 1 bed 15
1 2 chair 4
Supplier_2NF 2 1 bed 4
sID sLoc sPostal 2 3 table 6
1 Holtsville 00501 3 2 chair 8
2 Waukesha 53146 3 4 sofa 4
3 Waukesha 53146 4 2 chair 24
4 Ketchikan 99950 4 4 sofa 10

The Inventory_1NF table has been separated into two distinct tables: Supplier_2NF and Part_1NF. The
Supplier_2NF table has one primary key (sID) with two supporting columns (sLoc and sPostal) that are
dependent on the primary key, thus making it a table that is in second normal form. The Part_1NF, on the
other hand, has a composite primary key (made up of sID and pID) but the supporting column (pName) is
not dependent on both of the composite keys (dependent only on pID), thus making this table still in first
normal form.
83 ● Third Normal Form (3NF) – A table in this level of normalization has been normalized so that every non-
key (supporting) value depends solely on the primary key and not on some other non-key (supporting) value.

4-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

Part_3NF
sID pID Qty
1 1 15
1 2 4
Part_1NF 2 1 4
sID pID pName pQty 3NF
2 3 6
1 1 bed 15
3 2 8
1 2 chair 4 3 4 4
2 1 bed 4 4 2 24
2 3 table 6
4 4 10
3 2 chair 8

Oracle University and BOS-it GmbH & Co.KG use only

3 4 sofa 4 PartName_3NF
4 2 chair 24 3NF pID pName
4 4 sofa 10 1 bed
2 chair
3 table
4 sofa

84 In the case of the Part_1NF table, when an attempt is made to normalize it in the second normal form, it is clear
that it is truly in third normal form. Due to the lack of complexity in the initial table (Part_1NF), the second
normal form was essentially passed over and the two resulting tables (Part_3NF and PartName_3NF) met all the
criteria of the third normal form.

SupplierPostal_3NF
sPostal sLoc
00501 Holtsville
53146 Waukesha
Supplier_2NF 3NF
99950 Ketchikan
sID sLoc sPostal
1 Holtsville 00501
2 Waukesha 53146
3 Waukesha 53146 Supplier_3NF
4 Ketchikan 99950 sID sPostal sName
3NF
1 00501 KidzRooms
2 53146 Zurniture, Inc.
3 53146 MyRoom
4 99950 Furventure, Inc.

Due to the transitive dependency in the Supplier_2NF table (sLoc is dependent on both sID and sPostal), the
table could not be considered third normal form. By normalizing that table into two separate third normal form
tables (SupplierPostal_3NF and Supplier_3NF), updates in the future will be less problematic.
85 1 to Many Relationships
One of the positive results of normalizing a database, is that the resulting relationship between two tables can result in a 1
to Many () relationship. This relationship ensures an efficient approach to updates, deletes and alterations to existing data.
The following is the complete view of the normalization that was completed on the original Inventory table:

4-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

Inventory
sID sLoc sPostal pID1 pName1 pQty1 pID2 pName2 pQty2
1 Holtsville 00501 1 bed 15 2 chair 4
2 Waukesha 53146 1 bed 4 3 table 6
3 Waukesha 53146 2 chair 8 4 sofa 4
4 Ketchikan 99950 2 chair 24 4 sofa 10

Part_3NF PartName_3NF Supplier_3NF SupplierPostal_3NF

sID pID Qty pID pName sID sPostal sPostal sLoc

Oracle University and BOS-it GmbH & Co.KG use only

1 1 15 1 bed 1 00501 00501 Holtsville
1 2 4 2 chair 2 53146 53146 Waukesha
2 1 4 3 table 3 53146 99950 Ketchikan
2 3 6 4 sofa 4 99950 1
3 2 8 1 1
3 4 4
4 2 24
4 4 10

4 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

86 4.4 Denormalization
There are times when a normalized database is considerably slow due to a large number of tables having to be queried.
One of the most common ways to optimize the performance of a database that is normalized is by adding redundant data.
This process is known as denormalizing a database.

Denormalizing a Database
The process of denormalizing a database begins with completely normalizing the database, thus removing all redundant
data possibilities. Once a database is normalized, the backward process of denormalizing a database is easier, thus
ensuring that a database is only denormalized to the extent needed to improve performance. Examples of denormalization
techniques include:

Oracle University and BOS-it GmbH & Co.KG use only

● Combining one-to-many relationships – In the event that a single column is being connected to another table
through a one-to-many relationship, there are times when the single column should be added to the “many” table
to remove the need to have a join. Again, this should only be done to improve performance and only after the
database has been completely normalized and other performance improvements have not been effective.
● Adding Attributes – Similar to the previous technique, this technique adds additional columns to an existing
table to improve performance by minimizing the need for a join; however, this method does not cannibalize one
table to create the new attribute. This technique simply adds redundant data and does not interfere with other
existing tables that are normalized.
● Data Snapshots – One technique that does not interfere with the normalization of the database but can be an
effective method of avoiding extensive denormalization is creating a table that stores a snapshot of data that
needs to be retrieved through multiple methods. This can be useful for creating different views of a report from
the same base data or summarized data.
87 ● Schema Designs – There are multiple schema designs that can assist with denormalizing the design of the
database. The following are the most common schema designs:
○ Star Schemas – This is a hierarchical approach to designing a database by creating individual dimensional
tables (CourseDimension and ScheduleDimension) thus ensuring no additional tables are needed to hold
hierarchical information. Storing the hierarchy in a dimension table allows for the easiest browsing of the
dimensional data. This allows for drilling-down into the data by choosing an individual subcategory from
within the same table. This eliminates the need to join to an external table for any of the hierarchical
information.

CourseDimension ScheduleDimension
CourseID
CourseID ScheduleID
ScheduleID
SalesFact
CourseCode
CourseCode ScheduleCode
ScheduleCode
CourseName CourseID (FK) CostCode
CostCode
CourseName
Category ScheduleID (FK) StartDate
StartDate
Category
SubCategory
SubCategory SalesDollars Location
Location
CourseMaterial
CourseMaterial Instructor
Instructor
Duration
Duration

4 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

The fact table (SalesFact) that combines the two dimensional tables (Course and Schedule) is the center
of the “star” approach to schema design and provides the facts that could be used in general reports, thus
eliminating joining the two dimensional tables to retrieve that same information. In this case there is
only one attribute that is available in the fact table, but more could easily be added.
88 ● Snowflake Schemas – This is another hierarchical approach to designing a database, but breaking out the
dimensional tables (CourseDimension and ScheduleDimension) into more tables that normalize the database.
In the following example, the original Course dimension table is normalized with the addition of another
table.

CourseDimension ScheduleDimension
CourseID
CourseID ScheduleID
ScheduleID

Oracle University and BOS-it GmbH & Co.KG use only

SalesFact
Category CategoryID
CategoryID (FK)
(FK) ScheduleCode
ScheduleCode
CategoryID CourseCode
CourseCode CourseID (FK) CostCode
CostCode
CategoryID
CourseName
CourseName ScheduleID (FK) StartDate
StartDate
CategoryName
CategoryName CourseMaterial
CourseMaterial SalesDollars Location
Location
Duration
Duration Instructor
Instructor

This is a more normalized structure, but leads to more difficult queries and slower response times. This
normalization saves space, however, the dimension tables usually only hold about 1% of the records.
Therefore, space savings from normalizing, or snowflaking, is negligible.
NOTE: Dimension vs. fact tables

While the dimension tables are made up of multiple columns and hold large amounts of data in a minimal
number of rows, the fact tables are generally small in the number of columns but have large amounts of data in
a large number of rows.

4 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

89 4.5 Data Types

Data storage and retrieval is a large issue in the tuning of the MySQL server and ensuring that data integrity is
maintained. When designing the database, the following are some data type considerations that should be taken into
account:
● Use Appropriate Data Type – An example of this is storing numbers as strings. “09” and “9” are the same
number but different strings, and can lead not only to compromised data, but additional processing to convert the
string to a numeric value prior to performing calculations.
● Use Appropriate Length – Consider the max size of the column prior to setting the data type. For a person's
age, a tinyint unsigned data type (0 to 255) is appropriate. Using an integer unsigned data type (0 to
4,294,967,295) here would be a waste of memory and affect performance over the lifetime of the database.

Oracle University and BOS-it GmbH & Co.KG use only

● Use Not Null – Use not null if the column will never hold nulls. This will ensure the integrity of the data along
with minimizing the processes that have to be put into place to test for nulls or work with nulls, thus improving
performance.
● Use Appropriate Character Length – For columns that have the potential for holding varying length of strings,
CHAR and VARCHAR can be the best suitable data types. However, the max number of characters should be
taken into account and minimization should be taken into account. For example, a VARCHAR(64) is an
acceptable length for a name column. Anything more to compensate for the minimal number of names that could
exceed this value can have a negative effect on performance.
NOTE: Additional data type considerations

• A foreign key field type should be identical to its connecting primary key field (ex. If a primary key is an INT
field type, all foreign key field types associated with that primary key also should be INT).
• It is always best to consider which data types and character sets will minimize storage (disk I/O).
• Whether to use fixed or variable length strings depends on the length distribution of the values.
• For multi-byte character sets always consider variable length data types (fixed length columns always use the
maximum storage).
• Different server SQL modes will affect the data types that can be used and the validity of data entry for specific
data types.

4.5.1 MySQL Data Types

In MySQL (as in most database systems), the data types available can be broken down into three categories or types:
numeric, date/time and string. Within each category there are numerous specific data types that use varying amounts of
memory and thus have varying effects on performance. Choosing the best data type for a single column has a rather small
effect on performance in an individual record, but as the database grows these small effects can lead to larger effects and
should be taken into account prior to their becoming performance issues.

4 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

90 4.5.2 Numeric Data Types

The numeric category has the largest number of data types, and in comparison to the other categories, traditionally
these have been the most used data types in databases. The following is a chart of the numeric data types and their
associated attributes:
Data Type Storage Size Range
TINYINT 1 byte S: -128 to 127
U: 0 to 255
SMALLINT 2 bytes S: -32,768 to 32,767
U: 0 to 65,535

Oracle University and BOS-it GmbH & Co.KG use only

MEDIUMINT 3 bytes S: -8,388,608 to 8,388,607
U: 0 to 16,777,215
INT, INTEGER 4 bytes S: -2,147,483,648 to 2,147,483,647
U: 0 to 4,294,967,295
BIGINT 8 bytes S: -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807
U: 0 to 18,446,744,073,709,551,615

91 FLOAT(P) P<=25, 4 bytes S: -3.402823466E+38 to -1.175494351E-38 , 0,

P = precision in bits P>25, 8 bytes 1.175494351E-38 to 3.402823466E+38
U: 0, 1.175494351E-38 to 3.402823466E+38
(IEEE theoretical limits, hardware and O/S affect this range)
DOUBLE, REAL[(M,D)] 8 bytes S: -1.7976931348623157E+308 to -2.2250738585072014E-308,
M = Max # of decimal digits 0, 2.2250738585072014E-308 to 1.7976931348623157E+308
D = Decimal point digits U: 0, 2.2250738585072014E-308 to 1.7976931348623157E+308
(IEEE theoretical limits, hardware and O/S affect this range)
* Accurate to approximately 15 decimal places
DEC, DECIMAL, Multiples of 9 M can not exceed a value of 65 and defaults to 10 if omitted. D
NUMERIC[(M,D)] digits require 4 can not exceed 30.
bytes
S = Signed. U = Unsigned.

BIT Data Type

Previous to MySQL 5.0, the BIT data type was a synonym for the TINYINT data type. With the introduction of
MySQL 5.0, the BIT data type has taken on new meaning. A bit refers to a digit in the binary numeral system (base
2). The BIT data type is assigned using the following syntax: BIT(M)… where M indicates the number of bits per
value, from 1 to 64. The default is 1 if M is omitted. The memory storage required for the BIT data type is
approximately FLOOR((M+7)/8) bytes. So with a BIT(6) data type, the approximate storage required would be
FLOOR((6+7) / 8 ) => 1.625 or 1 byte.

4 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

NOTE: Assigning BIT values

To specify bit values, b'value' notation can be used. A bit value is a binary value written using zeroes and ones. For
example, b'111' and b'100000000' represent 7 and 128, respectively. If a value is assigned to a BIT (M) column
that is less than M bits long, the value is padded on the left with zeroes. For example, assigning a value of b'101' to
a BIT(6) column is, in effect, the same as assigning b'000101'.

92
4.5.3 Temporal Data Types
Date and time data types are referred to as temporal data types. The following is a chart of the temporal data types and
their associated attributes:

Oracle University and BOS-it GmbH & Co.KG use only

Data Type Storage Size Attributes
DATE 3 bytes Supported range: '1000-01-01' to '9999-12-31'
DATETIME 8 bytes Supported range: '1000-01-01 00:00:00' to '9999-12-31 23:59:59'
TIMESTAMP 4 bytes Supported range: '1970-01-01 00:00:00' to partway through 2037
TIME 3 bytes Supported range: '-838:59:59' to '838:59:59'
YEAR [(2|4)] 1 byte 4 digit year supported range: 1901 to 2155 (and 0000)
2 digit year supported range: 70 to 69
Temporal data types have some noteworthy limitations in MySQL:
● For the DATETIME and DATE range descriptions, “supported” means that although earlier values might work,
there is no guarantee. Example, using a date like 0325-04-04 (April 4th, 325 AD) may work, but there is no
guarantee on the part of MySQL.
● The SUM() and AVG() aggregate functions do not work with temporal values (they convert the values to
numbers, which loses the part after the first non-numeric character).
93
4.5.4 String Data Types
A string data type is a data type modeled on the idea of a formal string. Strings are such an important and useful data type
that they are implemented in nearly every programming language. The following is a chart of non-binary character string
data types and their associated attributes:
Data Type Storage Size Attributes
CHAR(M) M characters. Fixed-length right padded string. Supported range: 0 to 255
characters. The number of bytes required per character
depends on the character set used. (for all text types).
VARCHAR(M) Length of content in bytes, plus 1 Variable-length string. Supported range: 0 to 65,532
or 2 bytes to store the length. characters.
TINYTEXT Length of content in bytes, plus 1 Variable-length string. Max size: 255 characters.
byte to store the length.

4 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

TEXT Length of content in bytes, plus 2 Variable-length string. Max size: 65,535 characters. If a
bytes to store the length. maximum length (M) is entered, MySQL will use the smallest
TEXT data type that will support the values entered.
MEDIUMTEXT Length of content in bytes, plus 3 Variable-length string. Max size: 16,777,215 characters.
bytes to store the length.
LONGTEXT Length of content in bytes, plus 4 Variable-length string. Max size: 4,294,967,295 characters.
bytes to store the length.
ENUM('VALUE1', 1 byte for up to 255 values, 2 An enumeration column can have a maximum of 64k distinct
'VALUE2', ...) bytes for 256 to 64k values values. ENUM values are represented internally as integers.

Oracle University and BOS-it GmbH & Co.KG use only

SET('VALUE1', 1-8 elements occupies 1 byte, 9- A SET is a string object that can have zero or more values,
VALUE2', ...) 16 elements occupies 2 bytes, each of which must be chosen from a list of allowed values
17-24 elements occupies 3 bytes, specified when the table is created. A SET column can have a
25-32 elements occupies 4 bytes, maximum of 64 members. SET values are represented
and 33-64 elements occupies 8 internally as integers.
bytes.

94 4.5.5 Binary Data Types

Binary data types fall under the string data type due to their similarities in storage. Binary files are usually thought of
as being a sequence of bytes, which means the binary digits (bits) are grouped in eights. Binary files typically contain
bytes that are intended to be interpreted as something other than text characters. Compiled computer programs or
compiled applications are usually the first thing that comes to mind when talking about binary files; however, binary
files can also contain images, sounds, etc. The following is a chart of binary string data types and their associated
attributes:
Data Type Storage Size Attributes
BINARY(M) M bytes Fixed-length right padded binary byte string. Supported range:
0 to 255 bytes.
VARBINARY(M) Length of content in bytes, plus 1 Variable-length binary byte string. Supported range: 0 to
or 2 bytes to store the length. 65,532 bytes.
TINYBLOB Length of content in bytes, plus 1 Variable-length BLOB (binary large object). Max size: 255
byte to store the length. bytes.
BLOB[(M)] Length of content in bytes, plus 2 Variable-length BLOB. Max size: 65,535 bytes. If a maximum
bytes to store the length. length (M) is entered, MySQL will use the smallest BLOB
data type that will support the values entered.
MEDIUMBLOB Length of content in bytes, plus 3 Variable-length BLOB. Max size: 16,777,215 bytes.
bytes to store the length.
LONGBLOB Length of content in bytes, plus 4 Variable-length BLOB. Max size: 4,294,967,295 bytes.
bytes to store the length.

4 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

95 4.5.6 PROCEDURE ANALYSE

MySQL has a built-in procedure which provides feedback on the data types used by offering suggestions on which data
type would be most appropriate for the data stored. This procedure analyzes the columns in a given query and provides
feedback on each column returned. The procedure works by analyzing the minimum and maximum length of the columns
along with determining if a column can be assigned with NOT NULL. This procedure has a tendency to offer ENUM's
quite often as a solution; however, there are times when this column type should not be used. The following is an example
of how this procedure is used within the mysql client:

mysql> SELECT District, Population FROM City PROCEDURE ANALYSE()\G

Oracle University and BOS-it GmbH & Co.KG use only

The resulting output would display three records providing not only the recommended optimal column type but also
additional information that shows how the procedure came to that conclusion. The following is a sample of the output
that might be displayed using the above SQL statement:

******************************* 1. row ******************************

Field_name: world.City.District
Min_value: Abhasia [aphazeti]
Max_value: ?
Min_length: 1
Max_length: 20
Empties_or_zeros: 4
Nulls: 0
Avg_value_or_avg_length: 9.0194
Std: NULL
Optimal_fieldtype: VARCHAR(20) NOT NULL
******************************* 2. row ******************************
Field_name: world.City.Population
Min_value: 42
Max_value: 10500000
Min_length: 2
Max_length: 8
Empties_or_zeros: 0
Nulls: 0
Avg_value_or_avg_length: 350468.2236
Std: 723686.9870
Optimal_fieldtype: MEDIUMINT(8) UNSIGNED NOT NULL
2 rows in set (#.## sec)

4 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

How MySQL Handles Variable Length Strings

It is common to declare variable length string types to their maximum allowable length, on the off chance that one
day a field value might need to use all of that possible length. For example, on disk a VARCHAR(255) has the same
storage requirement as VARCHAR(32) if all field values will be less than 32 characters long anyway. So what's the
problem? The catch with this approach is that when the VARCHAR(255) field is moved into memory to be worked
with, it often behaves like a CHAR(255). In many cases MySQL needs to allocate enough memory to suit the
maximum possible size of the field.
One example of this behavior that can significantly and unexpectedly affect performance occurs with implicit
temporary tables that use the MEMORY storage engine (implicit temporary tables are used by MySQL to resolve some
queries; EXPLAIN will report Extra: Using temporary in this case). MEMORY supports only fixed length

Oracle University and BOS-it GmbH & Co.KG use only

records, so VARCHAR(255) becomes CHAR(255) when the data is loaded into the temporary table and all of a
sudden the 32 character strings are wasting over 200 bytes or memory!
Another example of memory wastage occurs when sorting data using ORDER BY. The sort buffer (controlled by the
sort_buffer_size variable) uses fixed length fields for fast lookup and reordering. If the sort buffer overflows,
MySQL must resort to doing multiple sort passes to sort the entire data set.
Consider this example using the standard world database.

mysql> show create table City\G

*************************** 1. row ***************************
Table: City
Create Table: CREATE TABLE `City` (
`ID` int(11) NOT NULL auto_increment,
`Name` varchar(32) NOT NULL default '',
`CountryCode` char(3) NOT NULL default '',
`District` char(20) NOT NULL default '',
`Population` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) ENGINE=MyISAM AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

mysql> flush status;

Query OK, 0 rows affected (0.00 sec)

Above, City uses a VARCHAR(32) for the Name field. If we execute a query that does an ORDER BY Name, we
can watch the Sort* status variables to see whether the sort buffer overflowed and if so, how many passes were
required:

4 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

mysql> select sql_no_cache Name from City order by Name;

...
4079 rows in set (0.02 sec)

mysql> show session status like 'sort%';

+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Sort_merge_passes | 0 |
| Sort_range | 0 |
| Sort_rows | 4079 |

Oracle University and BOS-it GmbH & Co.KG use only

| Sort_scan | 1 |
+-------------------+-------+
4 rows in set (0.00 sec)

Above, we see the simple ORDER BY Name operation required a single sort operation, with no extra pass. This is the
ideal situation (short of adding indexes to remove the sort step altogether). But, what happens if we make the Name field
size excessive? Make the VARCHAR(32) into a VARCHAR(1000):

mysql> alter table City change Name Name varchar(1000) not null default '';
Query OK, 4079 rows affected (0.06 sec)
Records: 4079 Duplicates: 0 Warnings: 0

Re-execute the exact same ORDER BY Name query:

mysql> flush status;

Query OK, 0 rows affected (0.00 sec)

mysql> select sql_no_cache Name from City order by Name;

...
4079 rows in set (0.02 sec)

mysql> show session status like 'sort%';

+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Sort_merge_passes | 1 | <-- sort buffer overflow
| Sort_range | 0 |
| Sort_rows | 4079 |
| Sort_scan | 1 |
+-------------------+-------+
4 rows in set (0.00 sec)

This time the same query caused a sort buffer overflow solely because we wasted space in the Name field. MySQL was
forced to allocate more memory to perform a second sort pass. In a production situation, this problem could easily
multiply until MySQL runs out of memory.

4 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

Unnecessary BLOB and TEXT Fields

Another common problem with database design is the unnecessary use of large data types when there is no real
requirement for them. Using a BLOB or TEXT type when a smaller CHAR or VARCHAR could be used instead will
again cause problems with the MEMORY storage engine and implicit temporary tables. Since MEMORY does not
support BLOB and TEXT, the potentially fast in-memory temporary table will be converted to a slow on-disk MyISAM
table, solely because of the data type, and not necessarily because of the real data size. This will cause unnecessarily
increased disk I/O and slow the MySQL server down.
Using PROCEDURE ANALYSE() is a good idea!

Oracle University and BOS-it GmbH & Co.KG use only

4 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

96 4.6 Partitioning
Controlling the physical aspects of data storage is an important aspect of performance tuning, because data access speed
is an important bottleneck for any database server. Traditionally, MySQL allows control of the storage method on a per-
table basis though choice of storage engines, and of course the Operating System and/or system hardware can be custom
configured. Partitioning refers to breaking a single table up into discrete segments which can then be distributed across
different file systems and/or hardware. It can lead to significant performance gains.

4.6.1 MySQL Built-in Partitioning

As of version 5.1, MySQL includes built-in partitioning. Specifically, it allows horizontal partitioning where a table may
be broken up into groups of rows (this is different to vertical partitioning which would allow a table to be broken up into

Oracle University and BOS-it GmbH & Co.KG use only

groups of columns. Only horizontal partitioning is supported in MySQL).
MySQL Partitioning includes:
● Partitioning a table is done at the database server level, without requiring the application know anything about it.
● Partitioning can be applied to any table regardless of the underlying storage engine. The only restriction is that a
single table must use the same storage engine for all partitions.
● The user supplies a partitioning function which governs how a table's data is spread between its partitions. This
can be tailored to suit the type of data and queries the table will be handling.
97 Some of the performance tuning related reasons for partitioning a table are:
● Avoid single disk or file system size and speed limitations by spreading data over multiple mediums. This is
somewhat storage engine dependent as, for example, MyISAM data files can easily be stored on different disks,
while InnoDB would need to run multiple table-spaces before this became beneficial.
● Query efficiency and performance can benefit:
○ Speed can improve by parallelizing multiple partition scans.
○ In certain cases, the MySQL Optimizer can use the partitioning function to prove in advance that certain
partitions will not contain any data required by a query, and so optimize those partitions away completely
from the execution plan, so they will not be touched (or locked) by the query at all. This is called partition
pruning, and it can improve query speed as less data is physically accessed, and query concurrency as less
data is locked.
○ Concurrency can improve when queries need to reference only a single or subset of partitions, as locking
conflicts are reduced.
● The impact of table maintenance tasks can be reduced by applying operations to a single partition at a time. For
example, archiving or removing old data might be a simple matter of altering or removing only the oldest
partition, rather than scanning the entire table.

4 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

98 Partition Types
The partitioning function for a table can divide data up between partitions in different ways:
● RANGE - Assigns rows to partitions based on column values falling within a given range.
○ Suits queries that access ranges of records, particularly contiguous ranges (WHERE …
BETWEEN ...AND...). A common example is web applications doing frequent display of paginated
data.
○ Suits things like date based partitioning, for example grouping records by year or month. This is
particularly useful for archiving old data from large tables: instead of issuing a slow and expensive
DELETE FROM … WHERE … < 2001, one might simply ALTER TABLE ... DROP

Oracle University and BOS-it GmbH & Co.KG use only

PARTITION P2001, which is potentially a much faster operation.
● LIST - Similar to partitioning by range, except that the partition is selected based on columns matching one
of a set of discrete values.
○ Suits queries that access records by the values used to partition data (WHERE … IN (…)).
● HASH - A partition is selected based on the value returned by a user-defined expression that operates on
column values in rows to be inserted into the table. The function may consist of any expression valid in
MySQL that yields a non-negative integer value.
○ This is a good method for ensuring even data distribution across partitions which can be important for
large data sets in terms of both access time and making the best use of available space.
○ There is a form of HASH partitioning called LINEAR HASH which is very fast at adding, dropping,
merging, and splitting partitions making it an excellent method for reducing the impact of maintenance
tasks on large partitioned tables.
● KEY - Similar to partitioning by hash, except that only one or more columns to be evaluated are supplied,
and the MySQL server provides its own hashing function. These columns can contain other than integer
values, since the hashing function supplied by MySQL guarantees an integer result regardless of the column
data type.
○ The benefits of HASH partitioning generally also apply here.

4 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

A Partition Pruning Example

As mentioned above, partition pruning refers to the MySQL Optimizer identifying which partitions in a table a certain
query needs, and ensuring the untouched partitions are skipped. This can have get impact on most query execution plans,
except perhaps eq_ref. MySQL supplies the EXPLAIN PARTITIONS SELECT... statement to allow the user to
investigate just how partition pruning is working.
For example, using the standard world database, create a version of the City table called City_Test, partitioned on
the Population field (if performing this example, you may need to remove the City_Test table prior to
performing):

Oracle University and BOS-it GmbH & Co.KG use only

mysql> CREATE TABLE City_Test (
-> `ID` int(11) NOT NULL,
-> `Name` char(35) NOT NULL DEFAULT '',
-> `CountryCode` char(3) NOT NULL DEFAULT '',
-> `District` char(20) NOT NULL DEFAULT '',
-> `Population` int(11) NOT NULL DEFAULT '0'
->) PARTITION BY RANGE(Population) (
-> PARTITION one VALUES LESS THAN (100000),
-> PARTITION two VALUES LESS THAN (1000000),
-> PARTITION three VALUES LESS THAN MAXVALUE);
Query OK, 0 rows affected (0.01 sec)

Populate it with the normal City data set:

mysql> INSERT INTO City_Test SELECT * FROM City;

Query OK, 4079 rows affected (0.01 sec)
Records: 4079 Duplicates: 0 Warnings: 0

Consider this table scan on the normal City table:

mysql> EXPLAIN SELECT * FROM City WHERE Population > 1000000\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
1 row in set (0.00 sec)

4 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

The query will touch all 4079 rows. Now review the same table scan on the partitioned City_Test table:

mysql> EXPLAIN PARTITIONS SELECT * FROM City_Test WHERE Population >

1000000\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City_Test
partitions: three
type: ALL
possible_keys: NULL

Oracle University and BOS-it GmbH & Co.KG use only

key: NULL
key_len: NULL
ref: NULL
rows: 238
Extra: Using where
1 row in set (0.00 sec)

Because the query filters by a column included in the partitioning function, the MySQL Optimizer knows that only
the third partition needs to be scanned, and that greatly reduces the number of rows that need to be touched.

4 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 4 - Schema/Database Design

99 4.7 Chapter Summary

In this lesson, you have learned to:
● Explain the purpose of data modeling
There are two basic types of data modeling tools that are most commonly used during the data modeling process
to describe information needs and/or the type of information that is to be stored in the database during the
requirements analysis: entity relationship diagrams (ERD) and unified modeling language (UML).
● Understand normalization and how to normalize the data in your databases
The purposes of normalization are: eliminate redundant data, eliminate columns not dependent on key and
isolate independent multiple relationships.

Oracle University and BOS-it GmbH & Co.KG use only

● Understand denormalization and why you would want to denormalize the data in your database
Examples of denormalization techniques include: combining one-to-many relationships, adding attributes and
creating data snapshots.
● Describe how MySQL data types affect performance
In MySQL, the data types available can be broken down into three categories or types: numeric, date/time
(temporal) and string.
● Understand how partitioning can improve performance
MySQL 5.1 introduces table partitioning which can improve performance as well as reduce the impact of
maintenance tasks.

4 - 25
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

100
5 INDEXING

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

101 5.1 Learning Objectives

This chapter describes how indexing can affect performance. In this lesson, you will learn to:
● Understand how MySQL uses indexes
● Describe the MySQL index types
● Explain special index characteristics
● Define index optimizations

Oracle University and BOS-it GmbH & Co.KG use only

5-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

102 5.2 Indexes

Indexing increases the performance of the database when it comes to read-access performance (executing SELECT
statements) by allowing quick access to the rows in a table. Indexes are created using one or more columns of a table. In
addition to the index being smaller than the original table (due to having fewer columns), indexes are optimized for quick
searching, usually via a balanced tree. When indexes are not used or are not matched by SQL statements submitted to the
database, then a full table scan is executed. A full table scan will read all the data in a table to find a specific row or set of
rows, which can be extremely inefficient.

Index Issues
Indexes are used to find rows with specific column values quickly. Without an index, MySQL must begin with the first

Oracle University and BOS-it GmbH & Co.KG use only

row and then read through the entire table to find the relevant rows. As a table grows in number of records, this search
becomes more costly. If the table has an index for the columns requested, MySQL can quickly determine the position to
seek to in the data file without having to look at all the data. If a table has 1,000 rows, this is at least 100 times faster than
reading sequentially. Conversely, the more rows of the table the query requests, the faster the read is sequentially, because
of the minimal disk seeks. The following list decribes some index issues to be aware of:

103 ● Speed versus maintenance – Indexes help to speed up data retrieval but are expensive to maintain.
● Slower writes – As indexes are added, the time to write to the data is increased to maintain index integrity.
● Index selectivity - The more selective an index is, the more benefit is obtained from using it.
● Duplicated data types - Highly duplicated data should not be indexed (for example, boolean data types, and
columns that represent gender, state abbreviations, or country codes). For indexes to be more efficient than a
complete table scan requires more than 3 different values in each index.

104 ● UNIQUE indexes – A UNIQUE index creates a constraint such that all values in the index must be distinct. An
error occurs if a new row of data is added with a key that matches an existing row. If an index is unique, use the
UNIQUE clause to force the column to be distinct which in turn also improves the efficiency of the indexing for
that column.
● Dead Indexes - Make sure to avoid indexes that are never used by any queries. These cause additional overhead
that is not necessary, and removing them will improve overall efficiency, especially during updates and inserts.
● Duplicate indexes - Avoid more than one index on the same column(s). The optimizer must determine which to
use. Also there is more maintenance as the data changes.

105 Composite Indexes

MySQL uses multiple-column indexes in such a way that queries are fast when a known quantity is specified for the first
column of the index in a WHERE clause, even if values are not specified for the other columns. The following list of
characteristics summarizes composite indexes:
● Indexes can be created on several columns.
● They can be used for searches on just the first column(s).
● An index on (a,b,c) can be used for searches on [(a), (a,b), (a,b,c)].

5-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

5.3 MySQL Index Types

MySQL offers a number of indexing types to use with the server; however, some storage engines will limit the type of
indexing that can be used and can provide additional indexing types for use with the specific storage engine. The
following are the most common index types used by the MySQL server:

106 B-Tree
This widely used data structure emulates a tree structure with a set of linked nodes. There are four common terms for
nodes that tree indexes can contain:
● Root node – This is the starting node for the tree, all nodes within a tree will be connected to this node,
either directly or indirectly.

Oracle University and BOS-it GmbH & Co.KG use only

● Parent node – This is a node that is connected to a node above it and contains nodes below it. This is
sometimes referred to as an internal node.
● Child node – This a node that is connected to a parent node. A child node must have at least one parent
node.
● Leaf node – This is a node that could be referred to as a child node but with one additional feature; leaf
nodes have no child nodes connected to them.
Due to each node having zero or more child nodes, the graphical representation looks like a tree growing down, not
up. The B-tree indexing type adds an additional requirement that keeps the tree balanced (there is some belief that the
‘B’ in B-Tree stands for balanced). A B-tree is kept balanced by requiring that all leaf nodes are at the same depth.
This depth will increase slowly as elements are added to the tree, but an increase in the overall depth is infrequent,
and results in all leaf nodes being one more hop further removed from the root. By maximizing the number of child
nodes for each parent node, the height of the tree decreases, balancing occurs less often, and efficiency increases. All
nodes have the same size (ie the "block" structure of the index). This is important as B-Trees are optimized for
minimizing disk reads.

5-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Root Node
(non-leaf)
Pointers to
child leaf
nodes
20 40 60
Leaf
Nodes

Keys Keys Keys Keys

1 19 21 39 41 59 61 79
2-19 22-39 42-59 62-79

Oracle University and BOS-it GmbH & Co.KG use only

Pointers to
Key data pages
Values

Actual
Data Pages

107 HASH
A hash is simply a key/value pair and a hash index is a collection of those key/value pairs. The hash index works
efficiently by taking a lookup value (the value portion of the key/value pair), obtaining the specific key associated with
the lookup value, then transforming the key using a hash function into a hash, and then using the hash number with the
associated hash table to locate the desired value.

5-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Keys Indexes Key/Value Pairs

(Records)
1
Robert Freed 2 Pierre Debok +1-213-555-4567
3

576 Gunter Greese +1-315-555-2314

Pierre Debok 577
Robert Freed +1-913-555-6786

Oracle University and BOS-it GmbH & Co.KG use only

578

990
Gunter Greese 991
992

Hash indexes have the following characteristics:

● This is the default index for the MEMORY storage engine.
● They are used only for equality comparisons that use the = or <=> operators (but are very fast). They are not
used for comparison operators such as < that find a range of values.
● The optimizer cannot use a hash index to speed up ORDER BY operations (this type of index cannot be used
to search for the next entry in order).
● MySQL cannot determine approximately how many rows there are between two values (this statistic is
sometimes used by the optimizer to decide which index to use). This may affect some queries, for instance, if
a MyISAM table is changed to a hash-indexed MEMORY table.
● Only whole keys can be used to search for a row. (With a B-tree index, any leftmost prefix of the key can be
used to find rows).
● A HASH index works within a small space and is extremely fast; however, can be very slow with many non-
unique values.

5-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

InLine Lab 5-A

In this exercise you will perform tests using the different types of indexes and their effect on performance. Install a fresh
copy of the world database to remove any indexes or other actions that may have altered the contents of the database.

Step 1. Prepare the database

Action: After logging into the MySQL server as root, type the following in the mysql client:

SET SESSION max_heap_table_size = 1073741824;

Effect: This variable sets the maximum size to which MEMORY tables are allowed to grow. Setting this variable has

Oracle University and BOS-it GmbH & Co.KG use only

no effect on any existing MEMORY table, unless the table is re-created.
Action: In the mysql client, type the following:

SOURCE /lab/scripts/pt_stored_procedures.sql

Effect: The pt_stored_procedures.sql file loads in all the stored procedures that will be used throughout
the PT course. Change the location of this file from your home directory (~/) to whichever directory this file is
located in.
Action: Create a table called city_memory_huge with approximately 20,000 records by executing the following
SQL statement:

CALL create_city_memory_huge(5);

Effect: This stored procedure will create a table called city_memory_huge that is based on the City table but
uses the MEMORY storage engine along with other indexing options. Once the city_memory_huge table is
created, the stored procedure will load in the records from the City table 50 times, creating approximately 20,000
records in the new table.
Action: Issue the following command to view the structure of the city_memory_huge table.

SHOW CREATE TABLE city_memory_huge;

Effect: The response shows that there is one B-Tree index and one HASH index on the Name column.
Step 2. Run A - Benchmark using a B-Tree index

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

mysqlslap -uroot -q /labs/scripts/btree-equal.sql --create-schema=world -i 5

If you wish to track how long it takes to actually run the mysqlslap executable, precede the executable with the
word time (such as time mysqlslap ...).
Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements using the B-Tree index,
against the city_memory_huge table.

5-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Record the number of seconds it takes to run all the SELECT statements (Run A):

Average Minimum Maximum

Step 3. Run B – Benchmark using a HASH index

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

Oracle University and BOS-it GmbH & Co.KG use only

mysqlslap -uroot -p /labs/scripts/hash-equal.sql
--create-schema=world -i 5

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements using the HASH index,
against the city_memory_huge table.
Record the number of seconds it takes to run all the SELECT statements (Run B):

Average Minimum Maximum

Step 4. Run C – Benchmark using no index at all

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

mysqlslap -uroot -p /labs/scripts/scan-equal.sql

--create-schema=world -i 5

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements not using any index,
against the city_memory_huge table.
Record the number of seconds it takes to run all the SELECT statements (Run C):

Average Minimum Maximum

Step 5. Run D – Benchmark using a B-Tree index with range searches

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

5-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

mysqlslap -uroot -p /labs/scripts/btree-nonequal.sql

--create-schema=world -i 5

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements using the B-Tree index
along with range searches, against the city_memory_huge table.
Record the number of seconds it takes to run all the SELECT statements (Run D):

Average Minimum Maximum

Oracle University and BOS-it GmbH & Co.KG use only

Step 6. Run E – Benchmark using a HASH index with range searches

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

mysqlslap -uroot -p /labs/scripts/hash-nonequal.sql

--create-schema=world -i 5

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements using the HASH index
along with range searches, against the city_memory_huge table.
Record the number of seconds it takes to run all the SELECT statements (Run E):

Average Minimum Maximum

Step 7. Run F – Benchmark using no index at all with range searches

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_memory_huge table:

mysqlslap -uroot -p /labs/scripts/scan-nonequal.sql

--create-schema=world -i 5

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements not using any index along
with range searches, against the city_memory_huge table.
Record the number of seconds it takes to run all the SELECT statements (Run F):

5-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Average Minimum Maximum

Step 8. Review and compare results

Action: Compare the performances of run a, b and c against each other.

Which run was the fastest?

Oracle University and BOS-it GmbH & Co.KG use only

Which run was the slowest?

Action: Compare the performances of run d, e and f against each other.

Which run was the fastest?

Which run was the slowest?

Action: Compare the performances of run c and f against each other.

Which run was the fastest?

Which run was the slowest?

Step 9. Clean up

Action: Clean up the world database by issuing the following command to delete the city_memory_huge
table:

DROP TABLE city_memory_huge;

Effect: The city_memory_huge table has been deleted from the world database.

5 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

B+TREE
108 Index Nodes
The B+Tree index is a data (i-nodes)
structure to store vast amounts
of information and is the next
3 5
level B-Tree. Typically
Leaf
B+Trees are used to store Key Nodes
amounts of data that will not fit Values
in main system memory. To do
this, secondary storage (usually 1 2 3 4 5 6 7
disk) is used to store the leaf

Oracle University and BOS-it GmbH & Co.KG use only

nodes of the tree. Only the
internal nodes of the tree are Pointers to
data pages
stored in computer memory. Links allow for
rapid in-order
In a B+Tree the leaf nodes are traversal
the only ones that actually store
data items. All other nodes are Actual
called index nodes or i-nodes Data Pages
and simply store ''guide'' values
which allow us to traverse the tree structure from the root down and arrive at the leaf node containing the data item we
seek. Because disk I/O is very slow in comparison to memory access these leaf nodes store more than one data item each.
In fact, the data structure will perform best when the size of the leaf nodes closely approximates the size of a disk sector
under most operating systems. Thus, when a search is performed against a B+Tree (by traversing from the root node
down to the proper data node) a read must take place against the data node from the disk to search its contents. Another
way to improve the speed of a query operation is to keep a memory cache of recently read leaf nodes.
A B+Tree is a rooted search tree satisfying the following properties:
● All paths from root to leaf are of the same length.
● The index node has ...
○ at least 2 children if not a leaf.
○ between 0 and (n–1) values (index entries), if a leaf.
● All other nodes are at least half-full, i.e. they have ...
○ between [n/2] and n children, if not a leaf.
○ between [(n–1)/2] and n–1 values (index entries) if a leaf.
● The fanout n (width of the index) depends on the size of the search-key and the size of disk blocks.

5 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Advantages of B+Tree indexes:

● Automatically reorganizes itself with small, local, changes, in the face of insertions and deletions.
● Reorganization of entire file is not required to maintain performance.
Disadvantage of B+Tree indexes:
● Extra insertion and deletion overhead, space overhead.

109 FULLTEXT
The MySQL server supports full-text indexing and searching. A full-text index in MySQL is an index of type
FULLTEXT and can be used only with MyISAM tables. Full-text indexes can be created for CHAR, VARCHAR, or

Oracle University and BOS-it GmbH & Co.KG use only

TEXT columns with CREATE TABLE. A FULLTEXT index may also be added later using ALTER TABLE or
CREATE INDEX. In large datasets, it is much faster to load the data into a table that has no FULLTEXT index and
then create the index subsequent to the load.
● MATCH() and AGAINST() functions - The MATCH() function performs a natural language search for a
string against a text collection, which is a set of one or more columns included in a FULLTEXT index. The
search string is given as the argument to AGAINST(). By default, the search is performed in case-
insensitive fashion. When MATCH() is used in a WHERE clause the rows returned are automatically sorted
with the highest relevance first.
The FULLTEXT index is stored as a kind of normalized, condensed version of the actual text and created as a B-Tree
index (words are used instead of column values). “Stop-words” (words that appear frequently in the data losing their
effectiveness as search terms) are removed and other words are grouped together.
NOTE: SHOW INDEX FROM ...

When displaying index status information for a specific table with SHOW INDEX FROM ..., the following
values will be displayed in the index_type field:
• BTREE - This value will be displayed for indexes that use B-Tree or B+Tree indexes.
• HASH - This value will be displayed for indexes that use Hash indexing.
• FULLTEXT - This value will be displayed for indexes that use Full-text indexing.

110 5.3.1 Index Overview

The following is an overview of which indexes can be used with which storage engine:

MyISAM
MyISAM InnoDB
InnoDB MEMORY
MEMORY NDB
NDB
Tree
TreeIndex
Index »«
»« »«
»« »«
»« »«
»«
Hash
HashIndex
Index »«
»« »«
»«
Full
Full-text
-textIndex
Index »«
»«

5 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

111 5.4 Index Optimizations

Due to the potential for indexes to be a source of great return on investment when it comes to MySQL performance, the
following “tricks” of index optimization are presented:
● Prefix Indexes - An index on a whole column is not always necessary. Instead, it may be best to create an index
on just a prefix of a column. Prefix indexes take less space and the operations are faster. Prefix index for data
selective by first few characters to help the optimizer determine statistically if it is useful for a given query.
● Composite Indexes - Composite indexes can be used for searches on the first column(s) in the index.
● Primary Key – Minimizing the size of the primary keys that are used as references in other tables can gain
incremental performance benefits. Additionally, using an AUTO_INCREMENT column can be more optimal for

Oracle University and BOS-it GmbH & Co.KG use only

the primary key column. Short keys are better; integer data types are best.
● FULLTEXT - A FULLTEXT index is useful for word searches in text and searches on several columns.
● Covering index - A covering index, which may be a composite index, includes all of the columns referenced in
the SELECT, JOIN, and WHERE clauses of a query. Because of this, the index contains the data being looked for
and MySQL server doesn't have to look up the actual data in the table, reducing logical and/or physical I/O, thus
boosting performance. While non-covering composite indexes can hinder performance, covering composite
indexes can be very useful, and in many cases, really boost the performance of a query by saving data reads.

5 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

Quiz
In this exercise you will answer the following questions pertaining to indexing.
1. What are the differences between B-Trees and B+Trees?
__________________________________________________________________________________
__________________________________________________________________________________
2. What are the differences between tree indexes and hash indexes?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

3. What are prefix indexes? What is their purpose?
__________________________________________________________________________________
__________________________________________________________________________________
4. How do FULLTEXT indexes differ from other indexes?
__________________________________________________________________________________
__________________________________________________________________________________
5. What are covering indexes? For which Storage Engines can they be useful?
__________________________________________________________________________________
__________________________________________________________________________________

5 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 5 - Indexing

5.5 Chapter Summary

112 In this lesson, you have learned to:
● Understand how MySQL uses indexes
Indexing increases the performance of the database when it comes to read access performance (executing
SELECT statements) by allowing quick access to the rows in a table.
● Describe the MySQL index types
MySQL offers a number of indexing types to use with the server; however, some storage engines will limit the
type of indexing that can be used and can provide additional indexing types for use with the specific storage

Oracle University and BOS-it GmbH & Co.KG use only

engine. The following are the most common index types used by the MySQL server: B-Tree and Hash. Other
index types include FULLTEXT, B+Tree, T-Tree, R-Tree and RB Tree.
● Explain special index characteristics
MySQL can create composite indexes (that is, indexes on multiple columns). An index may consist of up to 15
columns. For certain data types, MySQL can index a prefix of the column also. A multiple-column index can be
considered a sorted array containing values that are created by concatenating the values of the indexed columns.
● Define index optimizations
Due to the potential for indexes to be a source of great return on investment when it comes to MySQL
performance, the following “tricks” of index optimization were presented: prefix indexes, composite indexes,
use of primary key, FULLTEXT and covering indexes.

5 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

113
6 STATEMENT TUNING

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

114 6.1 Learning Objectives

This chapter explains practical steps to tuning MySQL statements. In this lesson, you will learn to:
● Explain general SQL tuning practical steps
● Use the EXPLAIN command
● Describe the MySQL optimizer
● List ways to find problematic queries

Oracle University and BOS-it GmbH & Co.KG use only

6-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

115 6.2 General SQL Tuning

Whether creating a new database or administering one that is quite mature, the database administrator must understand
how queries are executed within the database engine. The essence of database performance tuning is understanding how
the users of the database are querying and manipulating the data. The following is a list of practical steps to keep in mind
when thinking about MySQL statement tuning:
● Test on a realistic data set - Use a complete set of real-world data, not just a subset or sample of that data.
● Watch for changes to data and requirements – As the database and applications grow, be aware that
performance characteristics of the application will change.
● Do not make ported code assumptions - Porting applications from another RDBMS requires testing: do not

Oracle University and BOS-it GmbH & Co.KG use only

assume a query that executes quickly on other databases will do so on MySQL. Test all ported code thoroughly.
● Use efficient data types - Keep data types at the smallest possible size that meets requirements. This will ensure
minimal data storage requirements and will aid performance of the queries against the data. This is especially
critical for text mode queries (queries using text data types).
● Use temporary tables – When possible, temporary tables should be used for caching.
● Consider multiple queries – Break slow and complex queries into multiple, smaller queries.

6-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

116 6.3 EXPLAIN

The EXPLAIN command describes the plan which MySQL intends to utilize to execute a particular SQL statement,
but does not return any data from the data sets. This can be useful when a particular SQL query appears to take up a
significant amount of resources or causes frequent locking of the server. The EXPLAIN command can assist in
determining if MySQL has been able to choose an optimal pattern for data access. This command should be
considered an insider’s diagnostic view into how MySQL is determining a pathway to execute the SQL code.
When the EXPLAIN command is used preceding a SELECT statement, such as with the following example...

EXPLAIN SELECT * FROM world.City WHERE Name LIKE 'R%'\G

Oracle University and BOS-it GmbH & Co.KG use only

...it will produce a number of columns that provide insight on how MySQL will execute the statement. The following
list provides a quick overview of each output column that is produced from the EXPLAIN command:
117 ● id – This column is a simple identifier for the SELECT statement. This can be greater than 1 if there is a
UNION or sub-query.
● select_type – This column describes the type of SELECT being performed, and can have a number of
different results because the select_type has such a large part in how well the query will perform. Each
of the possible results will be discussed in greater detail later in this chapter.
● table – This column displays the name of the table used in the access strategy described by the row in the
EXPLAIN result. For DERIVED select_type’s, this output column will be labeled with the name
“derived” followed by a number.
NOTE: Derived tables

A derived table is a special type of subquery that appears in the FROM clause of a query, as opposed to the
SELECT or WHERE clauses. Derived tables can also be referred to as virtual tables or inline tables. When a
derived table is used in a query, it should be provided a unique name (this can be accomplished with the AS
clause) that will act as the table name:
SELECT … FROM ( subquery ) AS unique_name

● type – This column describes the access strategy deployed by MySQL to get at the data in the table or
index in the row. The access strategy type used by MySQL to execute the SELECT statement is a lengthy
and worthwhile discussion for improving performance on the MySQL server and will be presented in greater
detail later in the chapter.
● possible_keys – This column provides the available indexes (or NULL if there are none available) that
MySQL had to choose from in evaluating the access strategy for the table that the row describes. When this
field does not display any indexes, or minimal indexes, associated with the SELECT statement source data, it
is worthwhile to consider developing indexes to improve the performance of this query and additional
queries that will be executed on the source data.
118 ● key - This column displays the actual key chosen to perform the data access (or NULL if there was not one
available). Typically, when diagnosing a slow query, this should be the first place reviewed to ensure that
MySQL is using an appropriate index.

6-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

● key_len - This column provides the length, in bytes, of the key chosen. The length is NULL if the key column
says NULL. The number, if a unit is displayed, is often very useful in diagnosing whether a key’s length is
hindering a SELECT statement’s performance, and helps in determining how many parts of a multiple-part key
MySQL is using.
● ref - This column displays the columns within the key chosen that will be used to access data in the table, or a
constant, if the join has been optimized away with a single constant value. For instance, the following...

EXPLAIN SELECT * FROM Country INNER JOIN City

ON City.CountryCode = Country.Code WHERE City.Name='Paris'\G

Oracle University and BOS-it GmbH & Co.KG use only

...will be optimized away so that the constant City.CountryCode will be used instead of a comparison of
key values in the JOIN between City and Country.
● rows – This output column displays the number of rows that MySQL expects to find, based on the statistics it
keeps on the table or index (key) chosen to be used and any preliminary calculations it has done based on the
WHERE clause. This is a calculation MySQL does based on its knowledge of the distribution of key values in the
indexes. The freshness of these statistics is determined by how often an ANALYZE TABLE command is run on
the table, and, internally, how often MySQL updates its index statistics. Different storage engines supply the
optimizer with varying degrees of accuracy in determining estimated row count.
● Extra - This column contains extra information pertaining to this particular row’s access strategy. This field
can be thought of as an “any additional information that MySQL thinks might be helpful in understanding how it
is optimizing the SELECT statement executed” field.
NOTE: EXPLAIN EXTENDED

When a query is sent to MySQL , the optimizer sets about re-organizing it to match the best query execution plan.
This might include steps like rearranging the table join order. The optional EXTENDED keyword causes
EXPLAIN to output the final query form as an SQL warning which can then be viewed by SHOW WARNINGS.
This can sometimes be useful to gain insight into how the optimizer is treating problem queries.

119 6.3.1 EXPLAIN Output Column - select_type

This EXPLAIN output column describes the type of SELECT being performed. The results can be any of the following
values:
● SIMPLE- Identifies that the query is a normal, non-UNION, non-sub-query SELECT statement.
● PRIMARY- Identifies that the query is the outermost SELECT in any statement.
● UNION- Identifies that the query is a second or later SELECT in a UNION statement.
● DEPENDENT UNION- Identifies that the query is a second or later SELECT in a UNION statement that is
dependent on the results of an outer SELECT statement.
● UNION RESULT- Identifies that the query statement is the result of a UNION.
● SUBQUERY- Identifies that the query statement is the first SELECT in a sub-query.

6-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

● DEPENDENT SUBQUERY- Identifies a sub-query that contains a reference to a table that also appears in the
outer query (also known as a correlated sub-query).
● DERIVED- Identifies that the FROM clause contains a sub-query.
NOTE: Complicated select_type's

The more complicated select_type’s will affect the performance of the server as it attempts to execute the
statement. For select_type’s that are not identified as SIMPLE or some other less taxing select_type,
consider rewriting if they are having negative effects on performance of the server overall. If there is no other
way to produce the output being requested, this output column can give insight into exactly what the SELECT
statement is dealing with.

Oracle University and BOS-it GmbH & Co.KG use only

120 6.3.2 EXPLAIN Output Column - type
This EXPLAIN output column describes the type of access strategy deployed by MySQL to get at the data in the table
or index in the query components. When MySQL evaluates the best way to perform a SELECT query, it weighs all of
the strategies for accessing the various tables contained in the query using an optimization formula. Each access
strategy is assigned a sort of sliding performance scale that is compared using a number of statistics. The two most
important statistics used are:
● Selectivity of an index – This number identifies the cardinality (the number of unique values in the field)
within the index tree, thus giving MySQL a possible number of keys that an index will most likely match in
the WHERE or ON conditions of the query. This number of matching keys is the source for the rows output
column of the EXPLAIN command.
NOTE: Selectivity/key distribution

Selectivity is this number (cardinality) divided by the total number of records in the table; thus a unique index
has a selectivity of 1.0. Key distribution is a related concept describing how often each unique value occurs
within the index. So, an index with 100 unique values out of 1000 total records would have a selectivity of .10.
However, if 900 of the records contained a single value, and the other 100 records contained the other 9 unique
values, key distribution would be low, as the unique values would not be distributed evenly across the index.
Currently, the optimizer can only optimize for selectivity values, and not for key distribution values.
● Sequential reads versus random seeks – The relative speed of doing sequential reads for data on disk is
high versus reading index keys and accessing the table data using random seeks from the index row pointers
to the actual data location. MySQL uses a threshold value to determine whether repeated seek operations will
be faster than a sequential read. The threshold value depends on the WHERE or ON conditions of the query
along with specific storage engine values.
121 This EXPLAIN output column is a key performance tuning column to ensure that MySQL has chosen an optimal path
for joining the various data sets or if the query itself needs additional fine-tuning. The following list of values are
those that can appear in the type column of the EXPLAIN output (ordered from most efficient access to least
efficient).

6-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

system
This refers to a special type of access strategy that MySQL can deploy when the SELECT statement requests data from an
in-memory table and the table has only one row of data. For example:

mysql> CREATE TABLE systest (i INT);

Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO systest VALUES (1);

Query OK, 1 row affected (0.00 sec)

mysql> EXPLAIN SELECT * FROM systest\G

Oracle University and BOS-it GmbH & Co.KG use only

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: systest
type: system
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 1
Extra:
1 row in set (0.00 sec)

const
This access strategy type is used when the optimizer knows that it can get at most one row from the table and the values
from that row can be considered constants when referenced elsewhere in the query. This could be because there is a
PRIMARY KEY or UNIQUE constraint on a column that has a WHERE clause or because there is at most one row.
For example, using a fresh copy of the world database do a simple WHERE field comparison with a constant value:

mysql> EXPLAIN SELECT * FROM City WHERE ID=3803\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: const
rows: 1
Extra:
1 row in set (0.00 sec)

6-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

A calculated constant expression is reduced to a single constant also:

mysql> EXPLAIN SELECT * FROM City WHERE id=3800+3\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: const
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: const

Oracle University and BOS-it GmbH & Co.KG use only

rows: 1
Extra:
1 row in set (0.00 sec)

But any expression that does not consist entirely of constant values known before query execution, won't use const:

mysql> EXPLAIN SELECT * FROM City WHERE id-3=3800\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
1 row in set (0.00 sec)

NOTE: Impossible WHERE

Due the fact that MySQL performs a lookup for constant conditions on a unique key before the query
execution begins, if no rows match the WHERE expression the process will be stopped and the extra column
will contain the following statement:
Extra: Impossible WHERE noticed after reading const tables
In this case the access strategy type will contain a NULL value.

6-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

122 eq_ref
This access strategy type refers to a query that contains a single row read from the table for each combination of rows
returned from a previous data set. An eq_ref access strategy type can be used only if both of the following conditions
can be met:
● Keys used – All parts of a key must be used by the join in the query. This can include expressions that use
columns from tables that are read before this table or a constant.
● Unique key(s) are present – The table contains a unique, non-nullable key for which a WHERE condition
contains a single value which is present for each column in the key.
For example, using a fresh copy of the world database:

Oracle University and BOS-it GmbH & Co.KG use only

mysql> EXPLAIN SELECT Country.Name, City.Name FROM Country, City WHERE
City.CountryCode=Country.Code\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra:
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: world.City.CountryCode
rows: 1
Extra:
2 rows in set (0.00 sec)

The Country table's PRIMARY KEY is the Code field. Since the primary key is also UNIQUE by definition, MySQL
knows that only a single row will be returned from the Country table for each value of CountryCode found in the
City table.

6-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

ref
This access strategy type is the same as the eq_ref access strategy type with the exception that one or more rows
that match rows returned from previous table retrieval will be read from the current table (thus making this less
efficient than the eq_ref access strategy). The ref access strategy type is selected when either of the following
occurs:
● Leftmost part of unique index is used – The JOIN (or WHERE clause) condition uses only the leftmost part
of a multicolumn key.
● Non-unique and non-null key – The key used is not unique but does not contain any null values.
For example, using a fresh copy of the world database:

Oracle University and BOS-it GmbH & Co.KG use only

mysql> ALTER TABLE City ADD INDEX (District);
Query OK, 4079 rows affected (0.05 sec)
Records: 4079 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT * FROM City WHERE District='California'\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ref
possible_keys: District
key: District
key_len: 20
ref: const
rows: 62
Extra: Using where
1 row in set (0.00 sec)

6 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

123 ref_or_null
This access strategy type is identical to the ref access strategy type with the following exceptions:
● Null values – The key used can contain NULL values.
● OR null condition – The query contains a WHERE expression where an OR key_column is NULL.
For example, using a fresh copy of the world database:

mysql> ALTER TABLE Country ADD INDEX (IndepYear);

Query OK, 239 rows affected (0.01 sec)
Records: 239 Duplicates: 0 Warnings: 0

Oracle University and BOS-it GmbH & Co.KG use only

mysql> EXPLAIN SELECT IndepYear FROM Country WHERE IndepYear = 1905 OR
IndepYear IS NULL\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Country
type: ref_or_null
possible_keys: IndepYear
key: IndepYear
key_len: 3
ref: const
rows: 31
Extra: Using where; Using index
1 row in set (0.00 sec)

6 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

index_merge
This retrieval method uses more than one index for a single referenced table in the query. In an index_merge
access strategy type, multiple executions of ref, ref_or_null, or range accesses are used to retrieve key values
matching various WHERE conditions, and the results of these various retrievals are combined together to form a single
data set.
For example, using a fresh copy of the world database:

mysql> ALTER TABLE City ADD INDEX (District);

Query OK, 4079 rows affected (0.02 sec)
Records: 4079 Duplicates: 0 Warnings: 0

Oracle University and BOS-it GmbH & Co.KG use only

mysql> EXPLAIN SELECT * FROM City WHERE ID = 50 OR District = 'Michigan'\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: index_merge
possible_keys: PRIMARY,District
key: PRIMARY,District
key_len: 4,20
ref: NULL
rows: 9
Extra: Using union(PRIMARY,District); Using where
1 row in set (0.00 sec)

6 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

124 unique_subquery
This access strategy type is using a sub-query, which can be described as a child query that returns a set of values in an
IN clause in the WHERE condition. MySQL will use this access strategy type when the sub-query will return a list of
unique values due to the sub-query's SELECT statement using a unique, non-nullable index.
For example, using a fresh copy of the world database:

mysql> EXPLAIN SELECT * FROM City WHERE CountryCode IN (SELECT Code FROM
Country WHERE Continent='Asia')\G
*************************** 1. row ***************************
id: 1

Oracle University and BOS-it GmbH & Co.KG use only

select_type: PRIMARY
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
*************************** 2. row ***************************
id: 2
select_type: DEPENDENT SUBQUERY
table: Country
type: unique_subquery
possible_keys: PRIMARY
key: PRIMARY
key_len: 3
ref: func
rows: 1
Extra: Using where
2 rows in set (0.00 sec)

index_subquery
This access strategy type is identical to the unique_subquery access strategy type except that MySQL has
determined that the values returned by the sub-query will not be unique.
NOTE: Correlated sub-queries

A correlated sub-query is a sub-query that contains a reference to a table that also appears in the outer query. Those
sub-queries that are not reduced to a list of values are called correlated sub-queries, which means that the sub-
query (inner query) contains a reference in its WHERE clause to a table in the outer query, and it will be executed
for each row in the PRIMARY result set. When this occurs, derived tables should be considered as another option
to remedy this potentially severe performance problem.

6 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

range
This access strategy type refers to a query that contains a SELECT statement involving a WHERE clause (or IN
conditions) that uses any of the following operators: >, >=, <, <=, IN, LIKE or BETWEEN. The LIKE operator can
utilize the range access strategy type only if the first character of the comparison expression is not a wild card. LIKE
'A%' can utilize the range access strategy type, but LIKE '%A', can not. Range operations are not possible with
HASH indexes.
For example, using a fresh copy of the world database:

mysql> ALTER TABLE City ADD INDEX (Population);

Query OK, 4079 rows affected (0.02 sec)

Oracle University and BOS-it GmbH & Co.KG use only

Records: 4079 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT * FROM City WHERE Population>1000000\G

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: range
possible_keys: Population
key: Population
key_len: 4
ref: NULL
rows: 238
Extra: Using where
1 row in set (0.00 sec)

6 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

index
This access strategy type is chosen when MySQL does a sequential scan of all the key entries of an index. The access
type is usually seen when both the following conditions exist:
● Slow data retrieval – No WHERE clause is specified or the table utilized does not have an index that would
speed up data retrieval.
● Covering index – All columns in the SELECT list for this statement are available in the index.
For example, using a fresh copy of the world database:

mysql> EXPLAIN SELECT ID FROM City\G

Oracle University and BOS-it GmbH & Co.KG use only

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: index
possible_keys: NULL
key: PRIMARY
key_len: 4
ref: NULL
rows: 4079
Extra: Using index
1 row in set (0.00 sec)

NOTE: "Using index" versus "index" type

"Using Index" appears in the Extra column of the EXPLAIN output when the server was able to use the
index data pages to retrieve all the information that it needed for the table in the query. "Using Index" will
appear with many of the access strategy types, but it will always appear when the type column of the EXPLAIN
output is used to execute a query. However, the "index" access strategy is not always a good thing because all
values of the index are being read. Many times, a sequential index scan can be quicker than a table scan, but not
always. Regardless, both an index scan and a table scan are sub-optimal query execution plans.

6 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

ALL
125
This access strategy type is used when a sequential scan of the entire table’s data is necessary. This access type is used
if either of the following conditions exist in the query:
● No conditions on keys – No WHERE or ON condition is specified for any columns of the table’s keys.
● Poor index selectivity – When index selectivity is poor, a sequential scan is considered more efficient than
numerous index lookups.
For example, using a fresh copy of the world database:

mysql> EXPLAIN SELECT * FROM City WHERE Population>100000\G

Oracle University and BOS-it GmbH & Co.KG use only

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where
1 row in set (0.00 sec)

6 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

6.3.3 EXPLAIN Output Column - key

126
When reviewing the actual key chosen, from the key column in the EXPLAIN output, there are times the index being
used by the MySQL server is not the index expected. As a last defense, use join and index hints, but keep in mind that as
the dataset changes, previous join hints may produce a sub-optimal query execution plan.
A SELECT statement can include the following statement options to give the optimizer hints on how to use indexes:
● USE INDEX (key_list) – This SELECT statement option advises MySQL to use only one of the possible
indexes from the key_list to find rows in the table.
● IGNORE INDEX (key_list) – This SELECT statement option can be used to notify MySQL to not use

Oracle University and BOS-it GmbH & Co.KG use only

some particular index from the key_list.
● FORCE INDEX (key_list) – This SELECT statement option which acts like USE INDEX but with the
addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no
way to use one of the given indexes to find rows in the table.
NOTE: ORDER BY or GROUP BY

USE INDEX, IGNORE INDEX, and FORCE INDEX affect only which indexes are used when MySQL decides
how to find rows in the table and how to do the join. They do not affect whether an index is used when resolving
an ORDER BY or GROUP BY.

STRAIGHT_JOIN
Another option for assisting MySQL with running a query in a more efficient way is to use the STRAIGHT_JOIN clause.
This clause is used when the developer believes that executing a join in a different order than MySQL intends to would
provide more efficiency. STRAIGHT_JOIN tells the server to access tables in order from left to right as given in the
SELECT statement. This means that the first table in the FROM clause will be accessed first, then its resulting values
joined to the first joined table, and so on. In most cases, MySQL chooses the most optimal join method and any change in
order will diminish efficiency; however, having this ability to control how MySQL runs a query at this level is impressive
and, in rare times, effective.
NOTE: ANALYZE TABLE

Before using a STRAIGHT_JOIN, it is best to ensure that MySQL is up to date with statistics on the tables to be
used. After running a baseline EXPLAIN to view MySQL’s chosen access strategy for the query, execute an
ANALYZE TABLE against the tables. By doing this, the tables analyzed will have their statistics updated to ensure
that MySQL uses the most efficient access strategy. Execute the EXPLAIN command again to see if any changes
took place. If not, then a STRAIGHT_JOIN may be warranted.

6 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

6.3.4 EXPLAIN Output Column - Extra

127
There are numerous comments that can be placed in this output column to provide additional information that
MySQL thinks might be helpful in understanding how it is optimizing the SELECT statement executed.
● Using where - This Extra comment states that a WHERE clause is used to restrict the rows to match
against the next table or send to the client. Unless the intent was to specifically fetch or examine all rows
from the table, there may be something wrong in the query if the Extra value is not Using where and the
table join type is ALL or index.
● Using filesort – This Extra comment states that a manual sorting-pass is needed. There is no extra

Oracle University and BOS-it GmbH & Co.KG use only

information given to tell if will fit in memory or use file-merge sort.
● Using temporary – This Extra comment states that MySQL will use a temporary table to store the
result set at some stage of the execution. There is no way to determine from the EXPLAIN output if the
temporary table will use the MEMORY or MyISAM storage engine.
128 ● Using index – This Extra comment states that all operations on this table reference are completed in
the index tree, thus no row lookups are needed.
● Using index for group-by - Similar to the Using index way of accessing a table, this indicates
that MySQL found an index that can be used to retrieve all columns of a GROUP BY or DISTINCT query
without any extra disk access to the actual table. Additionally, the index is used in the most efficient way so
that for each group, only a few index entries are read.
● Using sort_union(...), Using union(...), Using intersect(...) - These Extra
comments indicate how index scans are merged for the index_merge join type.
129 ● Distinct - This Extra comment states that MySQL is looking for distinct values, so it stops searching
for more rows for the current row combination after it has found the first matching row.
● Not exists - This Extra comment states that MySQL was able to do a LEFT JOIN optimization on
the query, and does not examine more rows in this table for the previous row combination after it finds one
row that matches the LEFT JOIN criteria.
● range checked for each record (index map: N) - This Extra comment states that
MySQL found no good index to use, but found that some indexes might be used after column values from
preceding tables are known. For each row combination in the preceding tables, MySQL checks whether it is
possible to use a range or index_merge access method to retrieve rows. This is not very fast, but is faster than
performing a join with no index at all.

6 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

6.3.5 Tuning Example

Lets investigate how to handle tuning the first four Extra column values: Using where, Using filesort,
Using temporary and Using index. Start with a fresh copy of the world database and run:

mysql> EXPLAIN SELECT Name FROM City WHERE CountryCode = 'USA'

-> ORDER BY Population DESC LIMIT 5\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City

Oracle University and BOS-it GmbH & Co.KG use only

type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where; Using filesort
1 row in set (0.00 sec)

Above, we can see:

● There will be a table scan (type: ALL) when locating the rows, as there is no index available to satisfy the
WHERE CountryCode = 'usa' clause.
● There will be a file sort (Extra: Using filesort) before returning the rows to the client, as there is no
index to return the rows pre-sorted for the ORDER BY Population DESC clause.
To remove the requirement for a table scan, we can add an index on CountryCode:

mysql> ALTER TABLE City ADD INDEX (CountryCode);

Query OK, 4079 rows affected (0.03 sec)
Records: 4079 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT Name FROM City WHERE CountryCode = 'USA'

-> ORDER BY Population DESC LIMIT 5\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ref
possible_keys: CountryCode
key: CountryCode
key_len: 3
ref: const
rows: 267
Extra: Using where; Using filesort
1 row in set (0.00 sec)

6 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

The table scan is gone.

We need to do more to prevent the file sort step. We can try adding an index on both CountryCode and
Population as these are the fields used in the WHERE and ORDER BY clauses. This should allow MySQL to both
locate and sort the rows in a single step (as the index is pre-sorted):

mysql> ALTER TABLE City ADD INDEX Code_Pop (CountryCode,Population);

Query OK, 4079 rows affected (0.03 sec)
Records: 4079 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT Name FROM City WHERE CountryCode = 'USA'

-> ORDER BY Population DESC LIMIT 5\G

Oracle University and BOS-it GmbH & Co.KG use only

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ref
possible_keys: CountryCode,Code_Pop
key: Code_Pop
key_len: 3
ref: const
rows: 267
Extra: Using where
1 row in set (0.00 sec)

This time both the table scan and the file sort are skipped by using the new index on both CountryCode and
Population to locate and sort the rows in a single step.

6 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

This query is now quite efficient and here we could stop, but we can take this one step further and add an index across all
three fields referenced in the query:

mysql> ALTER TABLE City ADD INDEX Code_Pop_Name

-> (CountryCode, Population, Name);
Query OK, 4079 rows affected (0.03 sec)
Records: 4079 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT Name FROM City WHERE CountryCode = 'USA'

-> ORDER BY Population DESC LIMIT 5\G
*************************** 1. row ***************************
id: 1

Oracle University and BOS-it GmbH & Co.KG use only

select_type: SIMPLE
table: City
type: ref
possible_keys: CountryCode,Code_Pop,Code_Pop_Name
key: Code_Pop_Name
key_len: 3
ref: const
rows: 188
Extra: Using where; Using index
1 row in set (0.00 sec)

Now, not only have we still prevented the table scan and the file sort, but we have allowed MySQL to skip touching the
table itself and draw all data from the index on CountryCode, Population and Name.

6 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

Quiz
In this exercise connect the definition with the best EXPLAIN output terminology.

Term Definition

____ const A. Refers to a non-unique key that may not have

NULL values

____ eq_ref B. Refers to an unique key

____ ref C. The “fastest” explain type

Oracle University and BOS-it GmbH & Co.KG use only

____ ref_or_null D. Identifies the outer query uses a non-unique key
to match against values returned from the nested query

____ unique_subquery E. A non-unique key that may have NULL values

____ index_subquery F. Identifies the outer query uses a unique index to

match against values returned from the nested query

6 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

130 6.4 MySQL Optimizer

The task of the query optimizer in MySQL is to find the best or optimal plan for executing the SQL query that is
submitted to the server. The query optimizer searches for the best plan among all the possible query evaluation plans. A
process called greedy optimization enables the optimization engine to significantly reduce the amount of time it spends
calculating optimal execution paths by a process of intelligent best-path reductions. For larger more complex queries, this
became a bottleneck in the server’s performance. The following representation demonstrates the Query Process:

MySQL DBMS
Query Compiler

Oracle University and BOS-it GmbH & Co.KG use only

Parser
Analyze Syntax

Preprocessor –
Semantic checking, name resolution

Parse
Optimizer
Tree
Generate
generate
Optimal
optimalQuery
QueryExecution
ExecutionPlans
Plans(Q(EP’s)
Query Transformations

Search for optimal execution plan

Plan refinement Query

Execution
QEP Plan

Query execution engine

6 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

131 6.5 Finding Problematic Queries

The need to search for problems in the server can sometimes be brought to the forefront by comments from the end
users; however, in most cases this process should be initiated as preventive maintenance, instead of a reactive "putting
out fires" situation. The following informal guidelines can assist in developing a preventive maintenance approach to
performance tuning:
● Run EXPLAIN - By monitoring the SQL statements being sent to the server and running those SQL
statements through the EXPLAIN command, many performance issues can be headed off. EXPLAIN is very
easy to run and with the steps taught up to this point, fairly uncomplicated to read. This command gives an
in-depth understanding of how MySQL views SQL statements and can be the first step in preventing
performance issues. For new databases, this process should be done extensively during the development

Oracle University and BOS-it GmbH & Co.KG use only

process and then put into a regular preventive maintenance schedule to ensure that as data grows, SQL
statements are optimized to handle such growth.
● Slow query log - Query logs can be costly on both performance and hardware but the slow query log is
worth the minimal overhead. The slow query log consists of all SQL statements that took more than the
long_query_time system variable value in seconds to execute. The time to acquire the initial table locks
is not counted as execution time. The minimum and default values of long_query_time are 1 and 10,
respectively. The slow query log can be used to find queries that take a long time to execute and are therefore
candidates for optimization.
NOTE: Enabling the slow query log

mysqld writes a statement to the slow query log after it has been executed and after all locks have been
released. Log order may be different from execution order. To enable the slow query log, start mysqld with
the --log-slow-queries[=file_name] option. If no file_name value is given, the default is the
name of the host machine with a suffix of -slow.log. If a filename is given, but not as an absolute
pathname, the server writes the file in the data directory.
○ Queries that do not use indexes are logged in the slow query log if the --log-queries-not-
using-indexes option is specified. In addition, the --log-slow-admin-statements server
option enables logging of slow administrative statements such as OPTIMIZE TABLE, ANALYZE
TABLE, and ALTER TABLE to the slow query log.
○ Queries handled by the query cache are not added to the slow query log, nor are queries that would not
benefit from the presence of an index because the table has zero rows or one row.
● General query log - The general query log is beneficial to performance tuning because it tracks all queries
that have been submitted on the MySQL server. The general query log is a general record of what mysqld
is doing and because of the additional overhead should probably be used only on development boxes. The
server writes information to this log when clients connect or disconnect, and it logs each SQL statement
received from clients. The general query log can be very useful when an error in a client is suspected and it is
beneficial to know exactly what the client sent to mysqld.

6 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

NOTE: Statements not in order

mysqld writes statements to the query log in the order that it receives them. This may be different from the order
in which they are executed. To enable the general query log, start mysqld with the --log[=file_name] or
-l [file_name] option. If no file_name value is given, the default name is host_name.log in the data
d irectory.

132 ● SHOW PROCESSLIST - The SHOW PROCESSLIST command provides a real-time list of all the connections
to the server and the actions that each one is performing. With the SUPER privilege, the SHOW PROCESSLIST
provides the connection identifier, the user, the host the user is coming from, the database the user is using, the
current command being run, along with the time and state for each open connection. Equivalent to executing

Oracle University and BOS-it GmbH & Co.KG use only

mysqladmin processlist on the O/S command line.
● Query Execution - And last but not least, if there is a query that is taking an extra long amount of time to
execute, it is a good sign that it needs to be tuned. These queries can be found through the other processes
discussed or through the end users.

InLine Lab 6-A

In this exercise you will perform tests using the different types of indexes and there effect on performance.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered the
contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the labs
for this chapter. The SQL to accomplish this is below:

SOURCE /labs/scripts/pt_stored_procedures.sql

Action: In the mysql client, create the city_huge table with approximately 800,000 records by executing the
following SQL statement:

CALL create_city_huge(200);

Effect: This stored procedure will create a table called city_huge that is based on the City table. Once the
city_huge table is created, the stored procedure will load in the records from the City table 200 times, creating
approximately 800,000 records in the new table.
Note: A warning may appear due to the script attempting to drop a table that does not exist, this is not a problem
and should be ignored.
Action: Terminate the mysql client by executing the following SQL statement:

QUIT;

Effect: This will ensure that the MySQL server can be shut down cleanly in the next step (nothing accessing the
server while it is shutting down).

6 - 25
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

Step 2. Modify MySQL server settings in my.cnf

Action: Shut-down the MySQL server by entering the following in an O/S terminal window:

/etc/init.d/mysql stop

Effect: This will terminate the current running of the MySQL server on your system. This an important step to
ensure that the changes to the my.cnf file will take place.
Action: Create the /etc/my.cnf file using the vi editor (or an available editor you are familiar with) to
create the content below:

Oracle University and BOS-it GmbH & Co.KG use only

[mysqld]
log_output=FILE
log-slow-queries=/tmp/mysql_slow.log
long_query_time=1
log-queries-not-using-indexes

Effect: These configuration options will be used to record information that will be used in the lab from this
point on.
Note: In some versions, the default for the log_output is TABLE. In addition,

Action: Restart the MySQL server by entering the following in an O/S terminal window:

/etc/init.d/mysql start

Effect: This will restart the MySQL server with the configuration options added to my.cnf.
Step 3. Setup a second O/S terminal for monitoring

Action: In a new O/S terminal window, execute the following command to continually monitor the
mysql_slow.log:

tail -f /tmp/mysql_slow.log

Effect: Executing this command in a separate terminal window will allow for you to switch back and forth to
monitor the current activity of the multiple processes that will be running.
Step 4. Execute mysqlslap

Action: Issue the following command in an O/S terminal window to run the mysqlslap benchmarking tool
against the city_huge table:

mysqlslap -uroot -q /labs/scripts/tuning-queries.sql

--create-schema=world -i 2

Effect: This will execute a pre-defined SQL file, containing multiple SELECT statements using, against the
city_huge table. Continue to the next step while mysqlslap is still running.

6 - 26
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

Step 5. Monitor slow query log

Action: Examine the window with the tail -f running while the mysqlslap benchmark in is running. This
may display queries that are running slow and provide information similar to that which is displayed below:

# Time: ###### ##:##:##

# User@Host: root[root] @ localhost []
# Query_time: # Lock_time: # Rows_sent: # Rows_examined: #
SET timestamp=###########;
SELECT * FROM city_huge WHERE Name = 'Hamilton';

Oracle University and BOS-it GmbH & Co.KG use only

Effect: There most likely will be a few queries that took over 1 second to run (that can be a small criteria), but even
then there should not be a very large number of queries that ran slow.
Step 6. Examine the slow query log in detail

Action: Issue the following command in an O/S terminal window to run the mysqldumpslow tool against the
/tmp/mysql_slow.log table:

mysqldumpslow /tmp/mysql_slow.log

Effect: This will display a summary of the content of the mysql_slow.log to the O/S window.

6 - 27
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

Further Practice
In this further practice you will optimize queries that were slow in lab 6-A.
1. Create a clean version of the world database to remove any index structures that have been created.
2. Load the stored procedures that were deleted when creating a fresh copy of the world database by issuing
the following command:

SOURCE /labs/scripts/pt_stored_procedures.sql

3. Create the city_huge table needs to be recreated again also by issuing the following command:

Oracle University and BOS-it GmbH & Co.KG use only

133 CALL create_city_huge(200)

4. In an O/S terminal window, review the SQL statements that were slow. The following query was listed as
slow:

SELECT CountryCode, MAX(Population) FROM city_huge GROUP BY

CountryCode;

5. Improve the query in step 4 by including a combined index on the CountryCode and Population
columns in the city_huge table.
6. In the mysql client, execute the SQL statement from step 4 to see if the query execution time was
improved.
7. In the O/S terminal window, review another SQL statement that was slow. The following query was listed as
slow:

SELECT Country.Name FROM Country WHERE Code IN (SELECT CountryCode

FROM city_huge WHERE Population > 8000000);

8. Improve the query in step 7 by rewriting the query as a JOIN statement.

9. In the mysql client, execute the SQL statement improved in step 11 to see if the query execution time was
improved.

6 - 28
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 6 - Statement Tuning

6.6 Chapter Summary

134 In this lesson, you have learned to:
● Explain General SQL Tuning practical steps
There are numerous practical steps to keep in mind when thinking about MySQL statement tuning: Test on a
realistic data set, watch for changes to data and requirements, do not make ported code assumptions, use
efficient data types, use temporary tables and consider multiple queries.
● Use the EXPLAIN command
The EXPLAIN command describes how MySQL intends to execute a particular SQL statement, but does not

Oracle University and BOS-it GmbH & Co.KG use only

return any data from the data sets. This can be useful when a particular SQL query appears to take up a
significant amount of resources or causes frequent locking of the server.
● Describe the MySQL optimizer
The query optimizer searches for the best plan among all the possible query evaluation plans. Prior to MySQL
5.0, the optimizer was determined to give the best possible plan without little thought to the time that it was
taking to determine the best evaluation plan. For larger more complex queries, this became a bottleneck in the
server’s performance.
● List ways to find problematic queries
The need to search for problems in the server can sometimes be brought to the forefront by comments from the
end users; however, in most cases this process should be initiated as preventive maintenance and not as an
exercise in putting out fires.

6 - 29
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

135
7 CACHING AND MYSQL

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

136 7.1 Learning Objectives

This chapter describes how caching works in MySQL and how to utilize caching to improve performance. In this
lesson, you will learn to:
● Discuss the issues related to choosing an architecture for the database application
● Explain caching and the associated benefits
● Understand how to utilize the query cache to improve performance
● Describe using MEMORY/MYISAM tables as cache tools

Oracle University and BOS-it GmbH & Co.KG use only

7-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

137 7.2 Choosing an Architecture

Throughout the many discussions that have been presented up to this point, the overriding thought that comes to many
peoples minds is how to put this all together to meet the needs of the end-users and the sponsors of the application. Thus,
the first step in this seemingly overwhelming process is to answer the question of what type of architecture is needed for
the database to meet the requirements. (This is assuming that the requirements have already been decided on: if not, start
there.) Once an architecture design is chosen it is hard to change or modify. On the other hand, if an architecture is
developed to be scalable from the beginning, the design complexity may be difficult and time consuming to implement.
Thus, designing the database architecture correctly from the beginning is crucial to the success of any application. With
this as the forethought, the following questions may assist in choosing the proper architecture design:

How much performance is needed?

Oracle University and BOS-it GmbH & Co.KG use only

Performance is definitely a large concern for end users and sponsors; however, it may not be the overriding concern in
reality. The requirements may instead stress extensibility, ease of maintenance, reliability, availability and the possible
integration with other applications. These are not necessarily counter to performance, but they can have an affect on
performance. Shooting only for a high-performance system is not always the best architecture design, it is better to
determine what the minimal performance level can be in light of other considerations and work from there.

How much scalability is required?

Again, if one of the requirements of the application is to be scalable almost to a fault, then other issues may have to be
put on a lower priority level. This can affect not only performance but implementation time as well. Being realistic with
sponsors and end users about each requirement is going to be essential to answering many of these questions and
explaining the consequences for rigid requirements in time and functionality.

What is the expected life span of the application?

If the application is a temporary solution until a larger solution can be implemented, the time and effort put into designing
an architecture that meets the needs of the final solution is foolish and a waste of resources. If, on the other hand, the
application is the final solution and is planned to have an extensive shelf life, the time, energy and resources put into the
application upfront are critical to the success of the application long term.
NOTE: What does this have to do with caching?

The architecture design decisions will have a huge influence on the types of caching to be implemented and
utilized. Having a proper understanding of the type of architecture that is planned to be implemented will provide a
better understanding of the caching that can and should be used. Not every architecture design with utilize all the
caching features or the same types of caching features; thus, it is important to have the architectural needs finalized
prior to determining which types of caches to utilize.

138 7.2.1 Architecture Design Best Practices

Now that the importance of determining the architecture required is established conceptually, it is important to review
some architecture best practices:

7-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

Review local database operations

When a requirement comes up, the first thought should be on seeing if the database operation can be handled locally
on the server. If a problem arises with the initial solution pointing to the application (“To change this we need to fix
15000 queries”), step back and determine if the problem can be resolved with a database solution (“This can really be
fixed by adding an index to the tables”). This requires less resources and the developers may not even be aware that a
database solution is possible.

Write code in “black boxes”

While this is more in the area of application development, DBAs should be aware of this as well. In this approach the
user is not able to see or review the code as they work, thus keeping the code constant and allowing the developers to
control possible side effects, and the DBA the ability to do local re-architecting as needed.

Oracle University and BOS-it GmbH & Co.KG use only

Think strategically
Ask the questions about implications of a change and whether it may fix performance today, but cause problems in
the future. Something that may take a short time now to implement may take much longer in the future to implement.

Use empirical data to plan

Even though there is a lot to be said about experience, the idea that it should be an overriding factor for making
changes is not reasonable. Before making any changes, review the empirical data to ensure that the data supports the
need for the change. Running benchmarks early is a good way to determine if the possible changes will support the
end result any better than the current stable application.

Scale out
Scale-out is the ability to distribute tasks across multiple machines. It is best to answer the question early which is
better: thirty-two (32) single CPU boxes or twenty (20) two CPU boxes.
NOTE: Layered architecture

MySQL architecture is more layered than hierarchical. Subsystems in MySQL do not depend on each other it
order to function, although some subsystems themselves do function in a tree-like fashion.

139 7.2.2 Architecture - The Big Picture?

Discussing the application involves not only the database but of course the interface to the database. Working in
concert with the application developers is an important task for DBAs. This includes not only the requirements and
design phases of the application process, but every stage up until the retirement of the application. This involves
answering the following refining questions:

What functionality does this system use and provide?

What is the overall purpose of the application and what is the database's part in that purpose. A database is just a tool
to meet a need in the organization, and the part it plays may not be as crucial as DBAs would like to believe. Seeing
where the database fits in and the part it can play can help the DBA see what is most crucial and what is not so
crucial.

7-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

Can changing this affect performance?

This question addresses the fact that minor use case behavior changes can sometimes give great boosts. Do not be limited
to the database server, since the architecture may allow opportunities for storing large data in files, especially static web
content, caching data (web proxies, memory cache, dedicated database), or external Full-text indexing.

Is the proper MySQL extension being used?

Custom MySQL extensions using user defined functions (UDF), or the various storage engines supported by MySQL
should also be considered in performance.

Should non-SQL data processing be used?

Oracle University and BOS-it GmbH & Co.KG use only

Non-SQL data processing occurs when data is processed in an application instead of using complex queries. Use of a
middle-tier approach for this type of application can focus performance on that tier rather than the database server.

Is normalization improving performance or making it worse?

Denormalizing a database may be necessary to improve performance. However, the process of denormalizing a database
begins with completely normalizing the database, thus removing all redundant data possibilities. Once a database is
normalized, the backward process of denormalizing is easier, thus ensuring that a database is only denormalized to the
extent needed to improve performance.

7-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

140 7.3 Caching

Proxy Cache
A cache is a block of memory used as a
temporary storage area where frequently
accessed data can be stored for quick access. Application Cache
The CPU and hard drive frequently use a cache,
as do web browsers. The following graphic Query Cache
represents the cost of caching using an inverted
triangle. As a system uses the caches farther
Table Cache
down the triangle, the costs associated with the
request are more expensive (ex. a proxy cache
hit is much cheaper than a key cache hit). The

Oracle University and BOS-it GmbH & Co.KG use only

Block/Data/Index Cache
design of an application should work to get the
cache hit as high as possible in the triangle, so
each layer will satisfy some requests, and
eventually move deeper and deeper into the triangle only if required. The only aspects of caching that are directly
related to MySQL are at the bottom of the triangle. All the higher level caches are independent and separate from the
database used.
NOTE: Buffer versus Cache

Buffers allow processes to temporarily store input in memory until the process can execute it. Buffers are used
when there is a difference between the rate at which data is received and the rate at which it can be processed,
or in the case that these rates are variable, for example in a printer spooler. Buffers are usually handled through
software versus hardware. Caches, on other hand, contain the results of a previous process or the process itself.
This allows the cache to provide the result to a frequent request without having to interact with the hardware,
thus speeding up the response time. For example, multiple processes access the same files, much of those files
can be cached to improve performance (RAM being much faster than hard drives).

Proxy cache
A proxy server is a computer that offers a computer network service to allow clients to make indirect network
connections to other network services. A client connects to the proxy server, then requests a connection, file, or other
resource available on a different server. The proxy provides the resource either by connecting to the specified server
or by serving it from a cache, which is referred to as the proxy cache. The proxy cache is the highest level of caching
and the most efficient caching in relation to performance.

Application cache
Application caches are components of the application that cache data to reduce the number of times that files and
databases have to be directly accessed. The majority of application caches have a defined expiration parameter for the
data cached to ensure that outdated information is not retrieved. In addition, many of the application caches create
dependencies between data in the cache and the data in the files the cache was populated with (such as application
caches that remove data from the cache when the original data is modified).

7-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

NOTE: Simplifying an application with query cache

The MySQL query cache simplifies the execution of the application by allowing the MySQL server to manage the
caching process. However, even though the MySQL server has simplified the application by determining which
query results can be reused, the performance of the server can be affected.

Query cache
The MySQL query cache caches SQL query results and allows duplicate queries (which must be absolutely the same) to
see the same query results from memory. It caches in MySQL server process memory and is fully transparent to the
applications accessing it. There is no invalidation control, as the query is invalidated only when any involved tables are
updated. This methodology works great for many read intensive web applications where data is often only used from

Oracle University and BOS-it GmbH & Co.KG use only

cache. The increased cache hit ratio in a read heavy environment is indicative of better performance in many situations.
The query cache does not work with prepared statements.

Table cache
Due to MySQL being a multi-threaded application, there may be many clients issuing queries for a given table
simultaneously. To minimize the problem with multiple client threads having different states on the same table, the table
is opened independently by each concurrent thread. This uses additional memory but normally increases performance.
The table cache’s purpose is to maintain a thread for each connection to a table that is open. If three connections are using
the table, the table cache will maintain a minimum of three separate threads in memory.

Block/data/index cache
The implementation of the caches at the bottom of the triangle is dependent on the storage engine being used; however,
the purpose of the caches remain the same. These caches are designed to keep the most frequently accessed table blocks
in memory. In the case of an index cache, the structures usually contain a number of block buffers where the most-used
index blocks are placed. These cache types are the most resource heavy but can provide the most benefits to improve
performance.

7-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

7.4 Caching Benefits

Most applications can benefit from some form of caching. For many applications caching is the only optimization
needed. There are many ways of caching in MySQL.
141 ● hostname cache – facilitates the quick lookup of hostnames.
This cache is most useful where there are slow DNS servers which results in repeated lookups that are time
consuming.
● privilege cache – access control list (ACL) privilege (grant) information for user accounts.
● table cache – represents a MySQL table in a simple hash structure.

Oracle University and BOS-it GmbH & Co.KG use only

The structures that make up this cache allow other objects to retrieve the table's structure meta information.
● key cache – repository for frequently used B-tree index data blocks for all MyISAM tables.
The key cache uses a form of least recently used (LRU) method of index tracking to purge unused indexes
and promote actively used indexes while at the same time keeping indexes that are needed for the B-tree
index.
142 ● query cache – store queries and data to return the data if the same query is issued.
This cache is made up of a separate set of classes that work together to cache not only the SQL statements
but the results of those SQL executions.
● buffer pool - contains cached index pages for InnoDB tables.
This buffer contains both cached leaf and non-leaf index pages. This buffer can be controlled by configuring
the innodb_buffer_pool_size server variable.

7-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

7.5 Non-MySQL caches to consider

143 HTTP server-side proxy cache
In an HTTP Server-Side Proxy Cache, first-time requests are passed through by the cache process to an origin server that
produces the page to be returned to the requestor. Subsequent requests for the same page are processed and returned by
the cache process. The origin server will only receive new requests for the page if the version in the cache process is
invalidated or has expired. This is referred to as an external request-response cache. It is useful when data does not
change and is a must have for static, semi-static web sites. Some issues related with this cache:
● Dynamic sites - There can be significant overhead for use with dynamic sites only, as there must be mechanisms
for expiration of data as it changes.

Oracle University and BOS-it GmbH & Co.KG use only

● Cache invalidation - Problems with cache invalidation can also occur causing either poor performance or stale
pages being issued.
● Sensitive data - Security issues may arise from storing sensitive data on the disk or the cache disclosing data to
a wrong user. These issues are resolved through the proxy cache itself.
144 Response component caching
Component caching is a spin-off of HTTP server-side proxy caching. Instead of caching the entire response, only
components are cached. Generally these are static or infrequently changing components, such as menus or other static
components that form the basis of the response. This is not the same as template language cache. There are many
different variants and tools available:
● PHP Smarty, < http://smarty.php.net/ >
● Perl Mason, < http://www.masonhq.com >
● Java JCS, < http://jakarta.apache.org/jcs/ >
With component caching the biggest need is to identify which data is “static” and which data is not.
145 Network Distributed Caching
A network distributed cache ensures that the same key-value mapping will exist in all the copies of the same mapping
area on every server participating in the cache and will update the caches everywhere in a lockstep. This eliminates the
need to use process or file cache on multiple nodes. The overall benefits of this type of cache include: no size limits, no
double caching, utilization of spare resources and working with local short term caches. The disadvantage is a possibility
of network latency issues that can lead to performance issues.
● memcached (http://www.danga.com/memcached/) - is a high-performance, distributed memory object caching
system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database
load.

7-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

146 Disk Cache

A disk cache is a mechanism for improving the time it takes to read from or write to a hard disk. Today, the disk cache
is usually included as part of the hard disk. A disk cache can also be a specified portion of random access memory
(RAM). The disk cache holds data that has recently been read and, in some cases, adjacent data areas that are likely to
be accessed next. Write caching is also provided with some disk caches. The following benefits are realized with disk
caching:
● Uses the file system cache.
● Well known “file” type interface.
● Storage space is cheap.

Oracle University and BOS-it GmbH & Co.KG use only

● Good latency if cached in the O/S file cache.
● Good for large objects and objects costly to generate.

7 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

147 7.6 Query Cache

The following simplified MySQL server diagram depicts the query cache subsystem being prior to the parser subsystem
due to its primary function of caching SQL statements and query results.

Client1 Client2 ClientN

MySQL Server
Connection Thread Pool

Oracle University and BOS-it GmbH & Co.KG use only

Query Cache
 InnoDB
Storage Engines  MyISAM
Parser Query 101101  MERGE
 MEMORY
 Federated
Optimizer  ARCHIVE

The query cache stores the text of a SELECT statement together with the corresponding result that was sent to the client.
If an identical statement is received later, the server retrieves the results from the query cache rather than parsing and
executing the statement again. The following list addresses additional query cache matters:
● Caches in MySQL server process memory.
● Fully transparent for applications.
● No invalidation control.
● Does not work with prepared statements!
● Works great for many read intensive web applications.
● The query cache can be emptied with RESET QUERY CACHE.
148 MySQL Query Cache Specifics
Of course, there are prerequisites and limitations regarding MySQL query cache usage, with the most important being:
● Only identical queries may be serviced from the cache. This includes spacing, text case, etc.
● Any modification to a table used by a query in the cache causes the query to be invalidated and removed from
the cache. InnoDB only causes query ejection when data is committed.
● Many functions, such as CURRENT_DATE, NOW, RAND and others, negate the use of the cache.
● No query that uses bind variables can be reused.
● No query that makes use of user defined functions can be cached.

7 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

149 Query Cache Scaling

In the case of the Query Cache, more memory does not always equal better performance. As the size of the cache
grows, some considerations are:
● Scanning the cache for a matching query gets slower.
● Processing the cache to invalidate queries gets slower.
● Mutex contention can increase.
Since access to the cache must be restricted during the update process (ie, for both inserting and invalidating queries),
as traffic load increases a cache that is too large may start to incur a performance penalty outweighing any
performance benefit it actually provides. In this case, it is advisable to set the cache to DEMAND only (see section

Oracle University and BOS-it GmbH & Co.KG use only

1.6.1 below) and specifically use SELECT SQL_CACHE for queries that are known to be worthwhile caching.

7.6.1 Query Cache Variables

To make sure MySQL uses the query cache, there are a few variables that need to be set in the configuration file:
150 ● query_cache_type – This server variable sets the query cache type. Setting the GLOBAL value sets the
type for all clients that connect thereafter. Individual clients can set the SESSION value to affect their own
use of the query cache. The following values are allowed:
○ 0 (OFF) - This setting tells the server to not cache results in or retrieve results from the query cache.
Note that this does not deallocate the query cache buffer. To do that, the query_cache_size should
be set to zero (0).
○ 1 (ON) – This default setting tells the query cache to cache all query results except for those that begin
with SELECT SQL_NO_CACHE.
○ 2 (DEMAND) – This setting tells the query cache to cache results only for queries that begin with
SELECT SQL_CACHE.
151 ● query_cache_size – This server variable determines the amount of memory allocated for caching query
results. The default value is 0, which disables the query cache. Note that query_cache_size bytes of
memory are allocated even if query_cache_type is set to zero (0). If this variable is set too small, less
than 40960, the server will issue a warning and automatically set the variable to zero (0)
○ Qcache_free_memory – This status variable displays the amount of free memory for the query
cache. Low amounts observed vs. total allocated for the cache may indicate an undersized cache, which
can be remedied by altering the global variable query_cache_size.
○ Qcache_not_cached - This status variable displays the number of non-cached queries. The
Qcache_not_cached counter provides insight into the number of statements executed against
MySQL that were not cacheable, due to either being a non-SELECT statement or being explicitly barred
from entry with a SQL_NO_CACHE hint.
○ Qcache_queries_in_cache - This status variable displays the number of queries registered in the
query cache.
○ Qcache_total_blocks - This status variable displays the total number of blocks in the query
cache.

7 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

152 ● query_cache_limit – This server variable establishes the maximum size result set that the query cache will
store. The default is 1M.
○ Qcache_hits – This status variable displays the number of query cache hits.
NOTE: Query Cache Utilization

If a query result is returned from query cache, the server increments the Qcache_hits status variable, not
Com_select. Thus the total number of select statements that were successfully executed can be obtained by
adding the Qcache_hits and Com_select numbers together. This would allow the following formula to
provide the query cache utilization: QC_Utilization = Qcache_hits/(Qcache_hits+Com_select)

Oracle University and BOS-it GmbH & Co.KG use only

○ Qcache_inserts – This status variable displays the number of queries added to the query cache.
Qcache_hits and Qcache_inserts shows the number of times a query was serviced from the cache
and how many queries have been inserted into the cache. Low ratios of hits to inserts indicate little query
reuse or a too-low setting of the query_cache_limit, which serves to govern the RAM devoted to each
individual query cache entry. Large query result sets will require larger settings of this variable.
● query_cache_min_res_unit – This server variable determines the minimum size (in bytes) for blocks
allocated by the query cache. The default value is 4096 (4KB). If there are a lot of queries with small results, the
default block size may lead to memory fragmentation, as indicated by a large number of free blocks.
Fragmentation can force the query cache to prune (delete) queries from the cache due to lack of memory. In this
case, decrease the value of query_cache_min_res_unit.
○ Qcache_free_blocks - The number of free memory blocks in the query cach.
○ Qcache_lowmem_prunes – This status variable displays the number of queries that were deleted from
the query cache because of low memory. This indicates how often MySQL had to remove queries from the
cache to make room for incoming statements. Other reasons for an increasing number of
Qcache_lowmem_prunes are an undersized cache, which can't hold the needed amount of SQL
statements and result sets, and memory fragmentation in the cache.
NOTE: Query cache defragmentation

In the event that there is suspicion that the query cache has become fragmented, you can execute the FLUSH
QUERY CACHE statement to defragment the query cache thus ensuring better utilization of memory. This
command does not remove queries from the cache, but coalesces memory free space chunks. Remove all queries
from the cache with the RESET QUERY CACHE command.

7 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

InLine Lab 7-A

In this exercise you will tune performance using the query cache.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered
the contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.
Action: In the mysql client, create the city_huge table with approximately 200,000 records by executing the

Oracle University and BOS-it GmbH & Co.KG use only

following SQL statement:

CALL create_city_huge(50);

Effect: This stored procedure will create a table called city_huge that is based on the City table. Once the
city_huge table is created, the stored procedure will load in the records from the City table 50 times,
creating approximately 200,000 records in the new table.

Action: Execute the following SQL statement to enable the query cache:

SET GLOBAL query_cache_size = 4 * 1024 * 1024;

Effect: This activates the query cache by setting the size of the query cache to a non-zero value. Setting it to 0
disables the query cache. The default size is 0, so the query cache is disabled by default.
Step 2. Insert a query into the query cache

Action: In the world database, execute the following command:

SELECT * FROM Country WHERE Code = 'FIN';

Effect: A single record from the world database is displayed.

7 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

Action: Execute the following SQL statements to review the state of the query cache

SHOW GLOBAL STATUS LIKE 'qcache%';

Effect: The query cache status is displayed, showing an output similar to the one below:

+-------------------------+---------+
| Variable_name | Value |
+-------------------------+---------+
| Qcache_free_blocks | 1 |
| Qcache_free_memory | 4185568 |

Oracle University and BOS-it GmbH & Co.KG use only

| Qcache_hits | 0 |
| Qcache_inserts | 0 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 0 |
| Qcache_free_blocks | 0 |
| Qcache_queries_in_cache | 0 |
| Qcache_total_blocks | 1 |
+-------------------------+---------+
8 rows in set (#.##)

Step 3. Test the query cache

Action: Execute the following SQL statement (from step 3) again:

SELECT * FROM Country WHERE Code = 'FIN';

Effect: The same record from the world database is displayed.

Action: Execute the following SQL statement again to review the query cache state:

SHOW GLOBAL STATUS LIKE 'qcache%';

Effect: The query cache status is displayed, is there a change in any of the variables?

_______________________________________

_______________________________________

7 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

Step 4. Test the query cache again

Action: Execute the following SQL statement (from step 3) again:

SELECT * FROM Country WHERE Code = 'FIN';

Effect: The same record from the world database is displayed.

Action: Execute the following SQL statement again to review the query cache state:

SHOW GLOBAL STATUS LIKE 'qcache%';

Oracle University and BOS-it GmbH & Co.KG use only

Effect: The query cache status is displayed, is there a change in any of the variables?

_______________________________________

_______________________________________

Step 5. Try a different query

Action: Execute the following SQL statement:

SELECT * FROM Country WHERE Code = 'SWE';

Effect: A new record from the world database is displayed.

Action: Execute the following SQL statement again to review the query cache state:

SHOW GLOBAL STATUS LIKE 'qcache%';

Effect: The query cache status is displayed, is there a change in any of the variables?

_______________________________________

_______________________________________

7 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

153 7.7 MEMORY/MyISAM Tables as Cache

Developing a caching approach would not be complete without investigating the possibility of using the MEMORY or
MyISAM tables as caches themselves for the applications. These two tables provide multiple benefits that make them
candidates for such strategies as creating tables that are essentially a subset of data from large data sets for caching
“subqueries”.

MEMORY
The MEMORY storage engine stores all data in memory; once the MySQL server has been shut down any information
stored in a MEMORY database will have been lost. However, the format of the individual tables is kept thus enabling the
creation of temporary tables that can be used to store information for quick access without having to recreate the tables

Oracle University and BOS-it GmbH & Co.KG use only

each time the database server is started. In reference to caching, this storage engine is exceptionally useful for developing
an internal caching for the following reasons:
● Performance - MEMORY tables are very fast as the data is fully in memory.
● Size limitation - The size is limited only by the size of the memory available.
● Table level locking – The MEMORY table objects utilize table level locking which is good for static data.
● Data population – Even though the data in MEMORY tables are lost on server shut down, the table structure
remains and can be populated on restart by using --init-file option.
● Indexing – MEMORY tables can utilize HASH indexes which can increase the performance even more.
154 MyISAM
This engine provides a combination of performance and functionality while providing high-speed storage and retrieval, as
well as full-text searching capabilities. Utilizing this storage engine in a caching strategy is very practical for the
following reasons:
● Fast disk based table access - A very fast engine for applications with a high read ratio from disk.
● FULLTEXT indexing – The MyISAM storage engine is the only engine that supports FULLTEXT indexing.

TEMPORARY
A TEMPORARY table is visible only to the current connection, and is dropped automatically when the connection is
closed. This means that two different connections can use the same temporary table name without conflicting with each
other or with an existing non-TEMPORARY table of the same name. (The existing table is hidden until the temporary table
is dropped.) By being only visible to the current connection, this table type can be used for result caching for a single
session.

7 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

Further Practice
In this further practice you will size the query cache followed by steps in creating a manual cache.
1. In the mysql client, execute the following SQL statement to reset the query cache statistics:

FLUSH STATUS;

155 2. Verify the query cache size is set to 4M. If there is a different value, set the query cache size to 4M.
3. Execute the Qcache-queries.sql file using the following mysqlslap benchmarking tool with one iteration:

Oracle University and BOS-it GmbH & Co.KG use only

mysqlslap -uroot -q /labs/scripts/Qcache-queries.sql -i 1
-–create-schema=world

4. Execute the same O/S command in step 3 again and then examine query cache statistics.
5. Reset the query cache statistics.
6. Set the query cache size to 16M.
7. Execute the same O/S command in step 3 two times and then examine the query cache statistics.
8. Reset the query cache statistics.
9. Set the query cache size to 32M.
10. Execute the same O/S command in step 3 two times and then examine the query cache statistics.
11. Reset the query cache statistics.
12. Set the query cache size to 64M.
13. Execute the same O/S command in step 3 two times and then examine the query cache statistics.
14. Reset the query cache statistics and then turn off query caching.
15. Of the different query cache sizes (4M, 16M, 32M or 64M), which size seems the best? Why do you think
this is?
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
16. Execute the following query in the mysql client and record the time it took to execute:

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,

Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

7 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

17. Execute the query from step 16 to obtain the next 5 records (6-10) and record the time it took to execute.
How many seconds did the query take to execute: _____________
18. Execute the query from step 16 to obtain the next 5 records (11-15) and record the time it took to execute.
How many seconds did the query take to execute: _____________
19. Execute the query from step 16 to obtain the next 5 records (16-20) and record the time it took to execute.
How many seconds did the query take to execute: _____________
20. Execute the query from step 16 to obtain the next 5 records (21-25) and record the time it took to execute.
How many seconds did the query take to execute: _____________

Oracle University and BOS-it GmbH & Co.KG use only

21. Sum up the total time of all the previous 5 queries executed. How many seconds total did it take to run all 5
queries: _______________________
22. Using the query from step 16, create a MEMORY table called foo that contains the first 25 records using the
following SQL and record how many seconds it took to execute the query:

CREATE TABLE foo ENGINE=MEMORY SELECT MAX(city_huge.Population) AS

city_pop, COUNT(*) AS num, Continent, Region FROM Country, city_huge
WHERE Country.Code = city_huge.CountryCode GROUP BY Continent, Region
LIMIT 25;

How many seconds did the query take to execute: _____________

23. Select the first 5 records from the table created in step 22 and record how many seconds it took the query to
execute.
How many seconds did the query take to execute: _____________
24. Select the next 5 records (6-10) from the table created in step 22 and record how many seconds it took the query
to execute.
How many seconds did the query take to execute: _____________
25. Select the next 5 records (11-15) from the table created in step 22 and record how many seconds it took the
query to execute.
How many seconds did the query take to execute: _____________
26. Select the next 5 records (16-20) from the table created in step 22 and record how many seconds it took the
query to execute.
How many seconds did the query take to execute: _____________
27. Select the next 5 records (21-25) from the table created in step 22 and record how many seconds it took the
query to execute.
How many seconds did the query take to execute: _____________

7 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

28. Sum up the total time of all the previous 5 queries executed. How many seconds total did it take to run all 5
queries: _______________________.
How often would you need to have people go to extra pages to be useful?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

7 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 7 - Caching and MySQL

7.8 Chapter Summary

156 In this chapter, you have learned:
● Discuss the issues related to choosing an architecture for the database application
The first step in the performance tuning process is to answer the question of what type of architecture is needed
for the database to meet the requirements of the end-users and sponsors.
● Explain caching and the associated benefits
A cache is a block of memory used as a temporary storage area where frequently accessed data can be stored for
quick access. The six caches specifically addressed were the proxy cache, application cache, network cache,
query cache, table cache and the block/data/index cache.

Oracle University and BOS-it GmbH & Co.KG use only

● Understand how to utilize the query cache to improve performance
The query cache stores the text of a SELECT statement together with the corresponding result that was sent to
the client. If an identical statement is received later, the server retrieves the results from the query cache rather
than parsing and executing the statement again.
● Describe using MEMORY/MYISAM tables as cache tools
Developing a caching approach would not be complete without investigating the possibility of using the
MEMORY or MyISAM tables as caches themselves for the applications. These two tables provide multiple
benefits that make them candidates for such strategies as creating tables that are essentially a subset of data from
large data sets for caching “subqueries”.

7 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

157
8 MYSQL SERVER CONFIGURATION

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

158 8.1 Learning Objectives

This chapter explains the details of configuring the server and using the associated status variables used to monitor
these settings. In this lesson, you will learn to:
● Describe the server connection parameters
● List the ways to deal with table cache and multi-threading issues
● Explain the individual thread parameters
● Describe the query cache and the associated server and status variables
● Describe the general monitoring status variables

Oracle University and BOS-it GmbH & Co.KG use only

● List the numerous handler status variables

8-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

159 8.2 Server Connection Parameters

The client-to-server connection is the vital link between the end user and the data contained in the server. When the
client-to-server connection is not running at optimal performance, often the server itself is blamed for the poor
performance when in reality the problem was improperly configured connection parameters. MySQL considers a
connection as any attempt by a client to access the server, no matter if the connection was successful or not. The sections
contains connection parameters that have a direct effect on the client-to-server connection.

8.2.1 The table cache server variables (table_open_cache and table_definition_cache)

The table_open_cache server connection setting tells the server the number of open tables for all threads that the

Oracle University and BOS-it GmbH & Co.KG use only

server should allow. Increasing this value (default is 64) increases the number of file descriptors that mysqld requires.
When setting the table_open_cache it is important to make sure the operating system can handle the number of
open file descriptors implied by the table_open_cache setting. If the table_open_cache is set too high,
MySQL may run out of file descriptors and refuse connections, fail to perform queries, and be very unreliable.
NOTE: What is a file descriptor?

A file descriptor is an index for an entry in a kernel-resident data structure containing the details of all open files.
Applications, such as MySQL, that need to access data that is maintained by the operating system, must issue
system calls to the kernel that in turn accesses the files on behalf of the applications, based on the index keys. The
application itself cannot read or write the file descriptor table directly.
The table_open_cache uses little RAM and it is generally fine to increase to values in the 10,000 range and even
higher to 100,000 or more if the operating system is configured to support that (ulimit in Linux). MyISAM uses
approximately one for each use of the table in all queries currently running on the server plus one for the first open.
NOTE: Separating table_cache

The table_cache server variable has been separated in MySQL 5.1 into two separate server variables:
table_open_cache and table_definition_cache. table_open_cache is similar to the original
table_cache variable by providing a cache for internal table objects but is no longer used for table definition
lookups. table_definition_cache provides a cache of the .frm file without using file descriptors. This
variable is ultimately responsible for handling the definition lookups by describing the metadata of the tables.
● Open_table_definitions - This status variable displays the number of cached .frm files.
● Open_tables – This status variable displays the number of currently open tables. A single table opened two
times is counted as two open tables. If the Open_tables status variable returns a large number of tables being
open and there are performance issues, it is a good sign that the table_cache server variable needs to be
increased. Even if the database has only 64 tables there may be more than 64 open tables. MySQL, being multi-
threaded, may be running many queries on the table at one time, and each of these will open a table.
● Opened_tables – This status variable displays the number of tables that have been opened.

8-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

NOTE: Is Opened_tables too big?

The simplest approach to determining if the Opened_tables status variable is prompting an increase in the
table_open_cache is by monitoring the server over time. If after starting the MySQL server the
Opened_tables status variable is constantly going up, then the table_open_cache is too low.
However, if it's only going up slowly per hour (<10), then the table_open_cache is configured well. The
purpose in monitoring the Opened_tables status variable is to avoid unnecessary disk I/O requests. If the
MySQL server has to open a table for each query, then the server will be bogged down with performance
issues.

InLine Lab 8-A

Oracle University and BOS-it GmbH & Co.KG use only

In this exercise you will perform tests against table cache server connection parameters and the associated status
variables.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered
the contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.
Action: After logging into the MySQL server as root, type the following in the mysql client:

FLUSH TABLES;

Effect: This SQL command will reset the table cache.

Step 2. Review the initial state of the table cache

Action: Issue the following SQL command to view the status of the table cache:

SHOW GLOBAL STATUS LIKE 'open%table%';

Effect: The response will include the following status variables:

• Open_table_definitions which displays the number of cached .frm files.
• Open_tables which displays the number of currently open tables.
• Opened_tables which displays the number of tables that have been opened.
Record the values for each of these status variables:

__________________________________________________

8-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Step 3. Open some tables

Action: In the world database, issue the following SQL command to lock the Country table to reads only for
any other clients accessing the database:

LOCK TABLES Country READ;

Effect: The current MySQL client connection is the only connection that can perform non-read SQL statements
against the Country table.

Oracle University and BOS-it GmbH & Co.KG use only

Action: Start up a second mysql client and perform the following in the world database:

LOCK TABLES Country READ;

Effect: A lock is issued against the Country table in this instance.

Step 4. Review the table cache again

Action: Issue the following SQL command in the second mysql client started to view the status of the table cache:

SHOW GLOBAL STATUS LIKE 'open%table%';

Effect: Did the values change for any of the status variables from those that were recorded previously? If so, which
ones and why?
___________________________________________________

___________________________________________________
Step 5. Open and close a table

Action: Start up a third mysql client and perform the following in the world database:

SELECT COUNT(*) FROM Country;

Effect: The current record count (similar to the listing below) for the Country table will be displayed:

+--------- +
| COUNT(*) |
+----------+
| 239 |
+--------- +
1 row in set (#.## sec)

8-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Step 6. Review the table cache again

Action: Issue the following SQL command in the third mysql client started to view the status of the table cache:

SHOW GLOBAL STATUS LIKE 'open%table%';

Effect: Did the values change for any of the status variables from those that were recorded in step 3. If so, which
ones and why?
___________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

___________________________________________________
Step 7. Review the table cache using SHOW OPEN TABLES

Action: Shut down all instances of the mysql clients currently running. This will ensure there are no longer any
connections to the MySQL server.
Action: Start up a single mysql client instance and issue the following command:

LOCK TABLES Country READ;

Effect: The Country table is locked allowing only this connection to be able to perform non-read SQL
statements against the table.

Action: View the contents of the table cache by issuing the following SQL command in the mysql client:

SHOW OPEN TABLES LIKE 'C%';

Effect: A response similar to the listing below will be displayed in the mysql client:

+----------+-----------------+--------+-------------|
| Database | Table | In_use | Name_locked |
+----------+-----------------+--------+-------------|
| world | City | 0 | 0 |
| world | Country | 1 | 0 |
| world | CountryLanguage | 0 | 0 |
+----------+-----------------+--------+-------------+
3 rows in set (#.## sec)

Effect: Why is the Country table showing that it is in use?:

___________________________________________________

8-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Step 8. Clean up

Action: Issue the following command to unlock any tables that were locked during this exercise:

UNLOCK TABLES;

Effect: Additional labs performed may have problems if any tables remain locked.

Oracle University and BOS-it GmbH & Co.KG use only

8-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

160 8.2.2 max_connections

The max_connections setting determines the number of simultaneous client connections allowed at one time. The
value should not be unnecessarily high (default is 100), since each connection requires memory and a file descriptor.
However, persistent connections will benefit from a large number of allowed connections, since a larger number of
max connections allow connections to remain opened rather than having to close and reopen frequently.
NOTE: Maximum number of connections

Static Linux binaries provided by Sun Microsystems can support up to 4000 connections; however, the actual
number of connections is dependent upon the quality of the thread library on a given platform, how much

Oracle University and BOS-it GmbH & Co.KG use only

RAM is available and the processes that the clients are running. For most Linux and Solaris systems, 500 –
1000 simultaneous connections is a more realistic number for mysqld to support. mysqld will limit the
number of threads that connect prior to limiting the amount of memory usage (read_buffer,
sort_buffer, etc.) for the threads already connected.

● Max_used_connections – This status variable displays the number of connections that were opened
simultaneously up to a particular time. Monitoring this variable will assist in determining if the
max_connections server variable needs to be increased.
NOTE: Extra reserved connection

When mysqld is executed, MySQL adds an additional reserved connection to the max_connections
variable for access by users who have the SUPER privilege. This can be useful when MySQL rejects any
connections with a “Too Many Connections” error when an attempt is made to connect when all available
connections are in use.

8.2.3 open_files_limit
This server variable defines the number of open file descriptors to reserve (or open). When viewing the
open_files_limit value with SHOW VARIABLES, the number may be different to the number that was
assigned to the --open-files-limit option. This is because the server determines the open_files_limit
value by running the two formulas below and picking the largest of the two:
max_connections * 5
… or …
max_connections + table_cache * 2
In the event that this value has been entered into the mysqld_safe section of the my.cnf file, the number entered
will be used if it is larger than either of these calculated numbers. For systems where MySQL is unable to change the
number of open files, the number will always be zero (0). Due to these equations, it is easy to see that this variable is
important both for opening files and connections. If the MySQL server prevents connections with the error ‘Too many
open files’, it is a good indication that the open_files_limit server variable needs to be increased.
● Open_files – This status variable displays the number of open file descriptors that are currently active.

8-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

161 8.2.4 thread_cache_size

This server variable defines the amount of threads “cached” after a connection disconnects for the purpose of maintaining
memory for new connections. Creating a thread costs time, so rather than killing the thread when a connection closes, the
server can keep the thread in its thread cache, so it can be used for new connection later. Requests for threads are satisfied
by threads taken from the cache if possible, and only when the cache is empty is a new thread created.
● Threads_cached – This status variable displays the number of threads in the thread cache.
● Connections - The number of connection attempts (successful or not) to the MySQL server.
● Threads_connected – This status variable displays the number of currently open connections.

Oracle University and BOS-it GmbH & Co.KG use only

● SHOW PROCESSLIST – This statement displays the details of the current threads running on the servers. With
the PROCESS privilege, a user can see all processes running. Without the PROCESS privilege, a user can only
see their running processes.
● Threads_created – This status variable displays the number of threads created to handle connections. If the
Threads_created status variable is close to the number of the thread_cache_size server variable
(default 8), then it will be necessary to increase thread_cache_size. Continue to increase the size of
thread_cache_size until this status variable increases at only a low rate. The thread connection overhead
varies by operating system and is quite low on Linux, higher on some others.
● Threads_running – This status variable displays the number of threads that are currently active and not
sleeping.
NOTE: Threads versus processes

A process has been given an address space on the operating system for executing a set of instructions with full
control over various resources (files, devices, etc.) to complete its task. When a new process is created, the
operating system has to create a new address space for it. Threads, on the other hand, work under the address
space given to its parent process (with all the control given to its parent process) and can change state without
having to create a new process. MySQL utilizes a thread-based server architecture which allows the various
executing threads (or lightweight processes) to access key shared resources

8-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

162 8.3 Table Cache and Multi-threading Issues

MySQL runs as a multi-threaded application, so there may be many clients issuing queries for a given table
simultaneously. The table_open_cache and max_connections system variables affect the maximum number
of files the server keeps open. If either of these server variables are increased, the operating system may impose a
limit on the per-process number of open file descriptors. It is best to consult the operating system documentation on
how to increase the per-process limits if that is absolutely necessary.

Determining table_cache size

table_open_cache is related to max_connections, and each of these system variables should be taken into
account when increasing either one. In the case of the table_open_cache, the following formula should be

Oracle University and BOS-it GmbH & Co.KG use only

considered when calculating its size:
(Concurrent Running Connections) *
(Maximum Number of Tables per Join) + 5
The additional five (5) is to reserve extra file descriptors for temporary tables and files. The operating system must be
able to handle the number of open file descriptors implied by the table_open_cache setting. If
table_open_cache is set too high, MySQL may run out of file descriptors and refuse connections, fail to perform
queries, and be very unreliable. The number of file descriptors available to MySQL can be increased by using the
--open-files-limit startup option to mysqld.

Table Cache Level

The cache of open tables is kept at a level of table_open_cache entries. MySQL closes an unused table and
removes it from the table cache under the following circumstances:
● When the cache is full and a thread tries to open a table that is not in the cache.
● When the cache contains more than table_open_cache entries and a table in the cache is no longer
being used by any threads.
● When a table flushing operation occurs. This happens when someone issues a FLUSH TABLES statement or
executes a mysqladmin flush-tables or mysqladmin refresh command.
When the table cache fills up, the server uses the following procedure to locate a cache entry to use:
● Tables that are not currently in use are released, beginning with the table least recently used.
● If a new table needs to be opened, but the cache is full and no tables can be released, the cache is temporarily
extended as necessary.
● When the cache is in a temporarily extended state and a table goes from a used to unused state, the table is
closed and released from the cache.

HANDLER and the table cache

If a table is opened with the HANDLER tbl_name OPEN statement, a dedicated table object is allocated for the
thread. This table object is not shared by other threads and is not closed until the thread calls HANDLER tbl_name
CLOSE or the thread terminates. When this happens, the table is put back in the table cache (if the cache is not full).

8 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Further Practice
In this further practice you will benchmark the table cache server connection parameters.
1. Create 500 tables by passing the number 500 into the single parameter of the create_many_tables()
163 stored procedure.
2. View the status of the table cache by looking at the associated status variables.
3. Set the two table cache server connection system variables to 256 each.
4. In the mysql client, source the qry_many_tables.sql file to execute a query against each of the tables
created.

Oracle University and BOS-it GmbH & Co.KG use only

5. Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the tables created:

mysqlslap -uroot -q /labs/scripts/qry_many_tables.sql -i 10

-–create-schema=world

6. In the mysql client, monitor the change of the table cache statistics.
7. Set the two table cache server connection system variables to 1024 each.
8. In the mysql client, source the qry_many_tables.sql file to execute a query against each of the tables
created again.
9. Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the tables created again:

mysqlslap -uroot -q /labs/scripts/qry_many_tables.sql -i 10

-–create-schema=world

10. Compare execution time by monitoring the change of the table cache statistics in the mysql client. Is there a
noticeable difference?
11. Remove the tables created by calling the drop_many_tables() stored procedure.

8 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

164 8.4 Individual Thread Parameters

When an individual thread is running there are many factors that are used by MySQL to determine the amount of
memory assigned to it. It is best that if only some queries need large values, they should be set per session just before
running those queries. This avoids larger than necessary allocations for queries that don't need very big sizes. Over-
allocating global settings is a common tuning mistake. The following is a list of controllable local variables that assist
in maintaining these parameters:
● sort_buffer_size – This system variable determines the amount of memory to allocate for sort
operations. Each thread that needs to do a sort allocates a buffer of this size. This value can be increased to
ensure ORDER BY and GROUP BY operations perform faster. If there are too many records for the sort to be
performed in memory, then a disk based sort takes place instead, which takes longer to do.

Oracle University and BOS-it GmbH & Co.KG use only

Each of the status variables below should be monitored to determine the size that the sort_buffer_size
should be set to:
○ Sort_merge_passes – This status variable displays the number of merge passes that the sort
algorithm has had to perform. If this value is large, the sort_buffer_size system variable may
need to be increased.
○ Sort_range – This status variable displays the number of sorts that were done using ranges.
○ Sort_rows – This status variable displays the number of sorted rows.
○ Sort_scan - This status variable displays the number of sorts that were done by scanning the table.
● join_buffer_size – This system variable determines the size of the buffer for joins that do not use
indexes and thus must perform full table scans. Normally, the best way to get fast joins is to add indexes.
However, if this is not possible, increase the value of join_buffer_size to get a faster full join. One
join buffer is allocated for each full join between two tables. Complex joins between several tables for which
indexes are not used should be avoided at all costs because multiple join buffers are generally necessary.
select_full_join, and other similar status variables, measure the number of joins not using indexes.
165 ● read_buffer_size – This system variable determine the size of the buffer used for catching row data
during full table scans. Each thread that does a sequential scan allocates a buffer of this size (in bytes) for
each table it scans. If many sequential scans are carried out, it is best to increase this value, which defaults to
131072. Typically the benefit stops increasing once the size reaches 1M to 2M, but up to about 8M has the
potential for being useful (more than this is rarely useful).
● tmp_table_size – This system variable determines the maximum size allowed for a temporary table
stored in memory tables before it is converted to an on-disk MyISAM table. This variable should be
increased if GROUP BY queries involving many rows are performed and there is enough memory to support
them.

8 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

NOTE: max_heap_table_size

It it important to realize that since the MEMORY storage engine is used for implicit in-memory temporary tables, the
max_heap_table_size variable places a maximum limit on tmp_table_size. For example, if
tmp_table_size is 32M but max_heap_table_size is 16M, then implicit MEMORY temporary tables will
convert to on-disk MyISAM tables when they grow past 16M. Thus both these variables need to be considered
when dealing with implicit temporary tables.
○ Created_tmp_disk_tables – This status variable displays the number of temporary tables on disk
created automatically by the server while executing statements. If BLOB/TEXT fields are requested in the
statement executed, disk based temporary tables are automatic. The number of these statements (containing

Oracle University and BOS-it GmbH & Co.KG use only

BLOB/TEXT fields) run on the server has to be taken into account before worrying about the size of this
status variable.
○ Created_tmp_files – This status variable displays the number of temporary files mysqld has
created.
○ Created_tmp_tables – This status variable displays the number of in-memory temporary tables
created automatically by the server while executing statements. If the ratio of
Created_tmp_disk_tables to Created_tmp_tables is high, consider increasing
tmp_table_size, if it's low, consider decreasing it. One or two creates per second is reasonable, perhaps
fewer if RAM is plentiful.

8 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Further Practice
In this further practice you will tune the tmp_table_size system variable and determine its impact on
performance.
1. Display the current values of the variables associated with the tmp_table_size system variable by
166 issuing the following command in the mysql client:

SHOW GLOBAL STATUS LIKE 'created%tmp%tables';

2. Execute the vmstat command in the O/S terminal window to display the current memory usage recorded.

Oracle University and BOS-it GmbH & Co.KG use only

3. Issue the following mysql and mysqlslap benchmark commands in the O/S terminal window to have a
noticeable load applied against the server and give us information for monitoring any future changes to
status variables:

mysql -uroot -e "CALL world.create_city_huge(200);"

mysqlslap -uroot -q "SELECT name, SUM(population) FROM city_huge
GROUP BY name ORDER BY name DESC;" -i 3 –-create-schema=world

4. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?
5. Increase the size of the tmp_table_size system variable to 8 *1024.
6. Execute the benchmark command from step 3 again.
7. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?
8. Increase the size of the tmp_table_size system variable to 8 *1024 * 1024.
9. Execute the benchmark command from step 3 again.
10. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?

8 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

167 8.5 General Monitoring

MySQL has a large number of status variables that can be monitored and analyzed for identifying possible performance
issues within the server. A number of these have already been mentioned up to this point; however, the following are a list
of other variables that are noteworthy and have a bearing on database performance.

Query Status Variables

The following status variables provide insight into the types of queries that are being executed on the server, thus giving a
better insight into the needs of the end users.
● Questions – This status variable represents the total number of queries executed on the server from all
clients, including malformed queries. This variable can be used to develop a rough load indicator for stable load

Oracle University and BOS-it GmbH & Co.KG use only

mix.
● Select_full_join – This status variable displays the number of joins that perform table scans because they
do not use indexes. If this value is not 0, the indexes of the tables on the server should be checked.
● Select_full_range_join – This status variable displays the number of joins that used a range search on
a reference table. Due to the queries that are being counted in this status variable being a drain on the server,
they are good optimization candidates.
● Select_range – This status variable displays the number of joins that used ranges on the first table. This is
normally not a critical issue even if the value is quite large.
● Select_range_check – This status variable displays the number of joins without keys that check for key
usage after each row. If this value is not 0, the indexes of the tables on the server should be checked.
● Select_scan - This status variable displays the number of queries that did a full scan of the first table. Due to
the first table likely being scanned, it should be evaluated for proper indexing.
168 Table Locks Variables
The amount of time that a table lock is in place can cause serious repercussions on the performance of the server. The
following variables provide an insight into the table lock process on the server:
● Table_locks_immediate – This status variable displays the number of times that a table lock was
acquired immediately. Table locks are recorded even for InnoDB tables although they are quite rare.
● Table_locks_waited – This status variable displays the number of times that a table lock could not be
acquired immediately and a wait was needed. If this is high and there are performance problems, it could
indicate a need to optimize the queries executing on the server. If that does not solve the problem, the table or
tables involved may need to be split up, or replication may be required. Other resolution steps could include
using InnoDB tables, concurrent inserts or adjusting lock settings.

8 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

169 Other Status Variables

The following status variables do not fit under any one heading but are valuable enough to be discussed in any
performance tuning environment and are listed here for the benefits they will provide in that endeavor.
● Aborted_clients – This status variable represents the total number of connections that were aborted
because the client died without closing the connection properly. This counter is incremented when any of the
following occur:
○ The client program did not call mysql_close() before exiting.
○ The client had been sleeping more than wait_timeout or interactive_timeout seconds
without issuing any requests to the server.

Oracle University and BOS-it GmbH & Co.KG use only

○ The client program ended abruptly in the middle of a data transfer.
● Aborted_connects – This status variable displays the number of failed attempts to connect to the
MySQL server. This counter is incremented when any of the following occur:
○ The client doesn't have privileges to connect to a database.
○ A client uses an incorrect password.
○ A connection packet doesn't contain the right information.
○ It takes more than connect_timeout seconds to get a connect packet.
● Binlog_cache_disk_use – This status variable displays the number of transactions that used the
temporary binary log cache but that exceeded the value of binlog_cache_size and used a temporary
file to store statements from the transaction.
● Binlog_cache_use - This status variable displays the number of transactions that used the temporary
binary log cache.
NOTE: Binlog Cache Size

The binlog_cache_size server variable tells the server how big the size of the cache to hold the SQL
statements for the binary log during a transaction should be. A binary log cache is allocated for each client if
the server supports any transactional storage engines and if the server has the binary log enabled (--log-bin
option). If a transaction is larger than the binary log cache, the server must open a temporary file to store the
transaction and then delete that temporary file when the transaction ends. The Binlog_cache_disk_use
and the Binlog_cache_use can be used to tune the server variable binlog_cache_size to a large
enough value that avoids the use of these temporary files.

8 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

170 ● Bytes_received - This status variable displays the number of bytes received from all clients.
● Bytes_sent - This status variable displays the number of bytes sent to all clients. The Bytes_received
and Bytes_sent can be used together to determine the traffic to and from the server.
● Com_xxx – These status statement counter variables indicate the number of times each xxx statement has been
executed. There is one status variable for each type of statement. For example, Com_delete and
Com_insert count DELETE and INSERT statements, respectively. All of the Com_xxx variables are
increased even if a prepared statement argument is unknown or an error occurred during execution. In other
words, their values correspond to the number of requests issued, not to the number of requests successfully
completed. There are numerous Com_xxx variables and each brings valuable insight into the working of the
server and should be monitored and analyzed in any performance tuning exercise.

Oracle University and BOS-it GmbH & Co.KG use only

171
8.5.1 SHOW VARIABLES
MySQL is highly configurable and supports a variety of variable types:
● Ordinary Variables (User Variables) – These variables are session only variables and will lose their value and
their identity at the end of the user session that created them. They are not accessible by other users' sessions. To
declare or reference a session variable, the variable name chosen must be prefixed with an @ symbol:

SET @foo = 100;.

There is no SHOW command that displays all the user variables, thus making it imperative that user variables are
memorized (or retained in some external fashion) for later use within the session.
NOTE: Duplicating global variable names

Even though it is possible to give a user variable the same name as a global variable (ex. wait_timeout), it is best
not to duplicate global variable names to eliminate confusion.
● System and Server Variables – These are the variables that will be most beneficial to performance tuning
because they contain states or attributes of the MySQL server. Many of the system and server variables can exist
in two states:
○ Global – These variable are the same throughout all the sessions that are running on the MySQL server and
can only be modified by using the SET GLOBAL command. For example:

SET GLOBAL thread_cache_size = 20;.

● Current Connection – These variables are specific to the current users connection and revert to their global
value when the users session is terminated. These variables names must be prefixed with two @@ symbols:

SET @@wait_timeout = 36000;

When SHOW VARIABLES is executed, the system variables are the variables that are displayed. In the course of
performance tuning, these variables and their values will be crucial to tuning the server properly. Many of the
variables listed in SHOW VARIABLES will be discussed in greater detail throughout this course.

8 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

NOTE: Viewing system and server variables

SELECT variable_name can be used to view specific global or current connection system and server
variables (eg. SELECT @@wait_timeout; ). To ensure that global system or server variable is being
viewed, use SELECT @@global.variable_name (eg. SELECT @@global.wait_timeout; )
User, system and server variables are case insensitive thus making @foo, @FOO and @Foo the same variable
(@thread_cache_size, @Thread_Cache_Size and @THREAD_CACHE_SIZE identify the same
server variable).
● Stored Routines Variables and Parameters – These types of variables are declared within stored routines
and are valid within the stored routines only. These variables can not be called outside of the stored routine

Oracle University and BOS-it GmbH & Co.KG use only

and lose their value and identity at the end of executing the stored routine.
172
8.5.2 SET/SELECT @@
The SHOW VARIABLES command is extremely useful in performance tuning but even it is not complete. There are
other variables that can be called up using the SELECT @@ command and set using the SET @@ command. The
following is the list of the most useful variables that fall under this category:
● AUTOCOMMIT
In InnoDB, all user activity occurs inside a transaction. If the AUTOCOMMIT mode is enabled
(AUTOCOMMIT = 1), each SQL statement forms a single transaction on its own. By default, MySQL starts
new connections with AUTOCOMMIT enabled.
○ Disabled - When AUTOCOMMIT is set to 0 (switched off), MySQL considers that the user always has
a transaction open. A SQL COMMIT or ROLLBACK statement ends the current transaction and a new one
starts.
■ COMMIT – All changes made in the current transaction are made permanent and become visible to
other users.
■ ROLLBACK - Cancels all modifications made by the current transaction.
Both statements release all InnoDB locks that were set during the current transaction.
○ Enabled – When AUTOCOMMIT is set to 1 (switched on) multiple-statement transactions can be
performed by starting them with an explicit START TRANSACTION or BEGIN statement and ending it
with COMMIT or ROLLBACK.

8 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

173 ● BIG_TABLES
When select queries are complex in nature (such as JOIN queries), MySQL may create a temporary table in
memory to process them. When these select queries get large, this can lead to problems with memory. By
enabling BIG_TABLES prior to running these complex queries, the server is told to create all temporary tables
on disk. This can lead to relatively slower query results; however, freeing memory for other processes, and the
time difference may be well worth it.
A typical approach to using the BIG_TABLES variable is below:

SET @@BIG_TABLES = 1;
SELECT problematic_query;

Oracle University and BOS-it GmbH & Co.KG use only

SET @@BIG_TABLES = 0;

● FOREIGN_KEY_CHECKS
With storage engines that support foreign keys, there can be problems with loading data or altering table
definitions and/or table data. The FOREIGN_KEY_CHECKS variable, which performs a check on each
transaction to ensure that foreign keys are not being violated, can be a performance issue when performing
certain functions.
○ Disabled – By setting FOREIGN_KEY_CHECKS to 0 (switched off), transactions will not be tested upon
foreign key constraints. This is useful when loading data and performing table alterations.
○ Enabled – By setting FOREIGN_KEY_CHECKS to 1 (switched on), each transaction will be tested for
foreign key constraint violations. This setting can not only affect large amounts of data being loaded or table
alterations but can also affect performance. However, the purpose of foreign key constraints, when needed,
is worth the performance issue.
A typical approach to modifying the FOREIGN_KEY_CHECKS variable is shown below with a previous
dumped file being restored with a SOURCE command:

SET FOREIGN_KEY_CHECKS = 0;
SOURCE dump_file_name;
SET FOREIGN_KEY_CHECKS = 1;

8 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

● SQL_BIG_SELECTS
MySQL is designed to run every JOIN select statement that is sent to it and is structured correctly. However,
this can be a bad thing for those SELECT statements that are structured correctly but will result in execution
times that are quite long.
○ Disabled – By setting SQL_BIG_SELECTS to 0 (switched off), queries that will result in a number
of rows greater than the max_join_size variable will be terminated. This setting can prevent server
delays and excessive memory loads.
○ Enabled – By setting SQL_BIG_SELECTS to 1 (switched on), all SELECT statements are allowed
and there is no testing the query execution against the max_join_size variable.

Oracle University and BOS-it GmbH & Co.KG use only

If the max_join_size system variable is set to a value other than DEFAULT, SQL_BIG_SELECTS is
automatically set to 0.
174 ● SQL_BUFFER_RESULT
When SQL_BUFFER_RESULT is set to 1 (switched on), results from SELECT statements are forced into
temporary tables. This helps MySQL free the table locks early and can be beneficial in cases where it takes a
long time to send results to the client.
● SQL_LOG_OFF
This variable provides control over which queries are logged to the general query log and which are not. By
setting this variable to 1, any queries submitted after setting this variable are effectively ignored by the log.
Thus, the value 1 turns off logging to the general query log. Setting this variable to its default value of 0,
turns back on logging to the general query log. This can be beneficial for the following reasons:
○ Minimizes the size of the general query log (which can become quite large).
○ To track only the clients suspected of causing bottlenecks and not those that are acting within
performance standards (benchmarks).
● SQL_SELECT_LIMIT
This variable tells the MySQL server the maximum number of rows to allow from any SELECT statements.
If a SELECT has a LIMIT clause, the LIMIT takes precedence over the value of SQL_SELECT_LIMIT.
To return the SQL_SELECT_LIMIT size to its original value, the following command should be executed:

SET @@SQL_SELECT_LIMIT = DEFAULT;

8 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

175 ● UNIQUE_CHECKS
This variables, when set to 1, tells the MySQL server to perform uniqueness checks for secondary indexes in
InnoDB tables. If set to 0, uniqueness checks are not done for index entries inserted into InnoDB's insert buffer.
This variable should be set to 0 when the data has been verified to contain no uniqueness violations when
performing large table imports into InnoDB tables.
● SQL_LOG_BIN
This variable disables or enables binary logging for the current connection (SQL_LOG_BIN is a session
variable) if the client has the SUPER privilege. The statement is refused with an error if the client does not have
that privilege.

Oracle University and BOS-it GmbH & Co.KG use only

176
8.5.3 Other SHOW commands
Other SHOW commands that have a bearing on performance, either for specific storage engines or for the server in
general, include the following:
● SHOW TABLE STATUS – This command gives an extensive listing of the tables and their attributes (meta-data)
for an individual database. It is best to use the terminating command (\G) rather than the semicolon (;) to provide
an easier view of the information displayed.
● SHOW INDEX – This command, followed by the table name, will deliver a listing of indexes and associated
attributes (meta-data) for the indexes in the said table. This command is beneficial when tuning the MySQL
optimizer and uses the following syntax:

SHOW INDEX FROM table_name;

● SHOW OPEN TABLES - This command lists the non-TEMPORARY tables that are currently open in the table
cache. The LIKE clause, if present, indicates which table names to match. The WHERE clause can be given to
select rows using more general conditions.
● SHOW ENGINE {INNODB | NDB} STATUS – Some storage engines can report custom status listings to
assist with management, monitoring and performance tuning.

8 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

177 8.6 Table Handlers

In this discussion, the word handler refers to a portion of code within MySQL, which performs the task of
communicating with the storage engines or the tables utilizing those storage engines. The associated handler status
variables can provide valuable information on the operation of the server and assist in determining the performance of
the server.
● Handler_commit – This status variable displays the number of internal COMMIT statements.
● Handler_discover - The MySQL server can ask the NDBCluster storage engine if it knows about a
table with a given name. This is called discovery. Handler_discover is the status variable that
represents the number of times that tables have been discovered via this mechanism.

Oracle University and BOS-it GmbH & Co.KG use only

● Handler_delete - This status variable displays the number of times that rows have been deleted from
tables.
● Handler_read_first - This status variable displays the number of times the first entry was read from
an index. If this value is high, it suggests that the server is doing a lot of full index scans; for example,
SELECT col1 FROM foo, assuming that col1 is indexed.
● Handler_read_key - This status variable displays the number of requests to read a row based on a key.
If this value is high, it is a good indication that the tables are properly indexed for queries.
● Handler_read_next - This status variable displays the number of requests to read the next row in key
order. This value is incremented if an index column is being queried with a range constraint or if an index
scan is being performed.
178 ● Handler_read_prev - This status variable displays the number of requests to read the previous row in
key order. This read method is mainly used to optimize ORDER BY ... DESC.
● Handler_read_rnd - This status variable displays the number of requests to read a row based on a fixed
position. This value is high if there are a lot of queries that require sorting of the result. This indicates that
are probably a lot of queries that require MySQL to scan entire tables or there are joins that don't use keys
properly.
● Handler_read_rnd_next - This status variable displays the number of requests to read the next row in
the data file. This value is high if there are a lot of table scans being performed. Generally this suggests that
the tables are not properly indexed or that queries are not written to take advantage of the indexes available.
● Handler_rollback - This status variable displays the number of internal ROLLBACK statements.
● Handler_update - This status variable displays the number of requests to update a row in a table.
● Handler_write - This status variable displays the number of requests to insert a row in a table.

8 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

Quiz
In this exercise you will answer the following questions pertaining to MySQL server configurations.

6. How can you monitor if there are joins not using indexes being executed on the server?
__________________________________________________________________________________
__________________________________________________________________________________
7. How can you monitor if you might be having problems with lock contention on the server?
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

__________________________________________________________________________________
8. 3.Which handler_* status variables show efficient use of indexes and which show inefficient use (or no
use)?
_________________________________________________________________________________
__________________________________________________________________________________
9. Do all the handler_* status variable counters correspond to the number of executed queries? If not, which
of them do?
__________________________________________________________________________________
__________________________________________________________________________________

8 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 8 - MySQL Server Configuration

8.7 Chapter Summary

179 In this lesson, you have learned to:
● Describe the server connection parameters
The client-to-server connection is the vital link between the end user and the data contained in the server.
The following is a list of connection parameters that have a direct effect on the client-to-server connection:
table_open_cache, max_connection, thread_cache_size and open_files_limit.
● List the ways to deal with table cache and multi-threading issues
MySQL runs as a multi-threaded application, so there may be many clients issuing queries for a given table

Oracle University and BOS-it GmbH & Co.KG use only

simultaneously.
● Explain the individual thread parameters
When an individual thread is running there are many factors that are used by MySQL to determine the
amount of memory assigned to it. The following is a list of controllable local variables that assist in
maintaining these parameters: sort_buffer_size, join_buffer_size, read_buffer_size,
tmp_table_size
● Describe the query cache and the associated server and status variables
The MySQL query cache stores the SELECT queries issued by clients to the MySQL database server. and the
query's result set.
● Describe the general monitoring status variable
MySQL has a large number of status variables that can be monitored and analyzed for identifying possible
performance issues within the server.
● List the numerous handler status variables
Handler refers to a portion of code within MySQL, which performs the task of communicating with the
storage engines or the tables utilizing those storage engines. The associated handler status variables can
provide valuable information on the operation of the server and assist in determining the performance of the
server.

8 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

180
9 MYISAM STORAGE ENGINE

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

181 9.1 Learning Objectives

This chapter explains the details of the MyISAM storage engine and how it relates to performance tuning. In this
lesson, you will learn to:
● List the characteristics of the MyISAM storage engine
● Explain where MyISAM tables are most effective
● Describe the different MyISAM row storage formats
● Explain the MyISAM indexing features
● Utilize the MyISAM locking characteristics

Oracle University and BOS-it GmbH & Co.KG use only

● Describe the MyISAM key cache
● List additional MyISAM tuning aspects
● Describe other MySQL optimization tools
● List the characteristics of the MERGE storage engine

9-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

182 9.2 MyISAM Characteristics

ISAM stands for indexed sequential access method. MyISAM is an improved storage engine based on the original
MySQL implementation of an ISAM storage engine, which is now deprecated. MyISAM allows fast retrieval of data via
a non-clustered index and data organization. MyISAM is the default storage engine for all versions of MySQL.

Storage Medium
MyISAM, like all MySQL storage engines, uses one or more files to handle operations within the data sets that are
located under the data_dir directory. The data_dir directory contains one subdirectory for each schema (database) residing
on the server. The MyISAM storage engine creates three separate files for each table within the associated schema
directory:

Oracle University and BOS-it GmbH & Co.KG use only

● table_name.frm – this file contains all the meta information about the MyISAM table definition.
● table_name.MYD – this file contains the actual table row data.
● table_name.MYI – this file contains the indexing data.

Transaction Capabilities
MyISAM does not provide atomicity, consistency or durability of multiple statements executed within a transaction.
Statements executed against MyISAM can not be guaranteed to successfully process according to the applications (or
more appropriately the designers) intent in the event of a server crash.

Locking
The MyISAM storage engine protects against corruption of data by different threads updating in conflicting ways by
using table-level locking. This has been seen as a “deficiency” in the MyISAM storage engine, but for the majority of
applications this is not an issue due to the processes in place and the speed of MyISAM, including in very high
concurrency situations.
183 Backup and Recovery
Of all the standard MySQL backup capabilities, the ability to stop the server and copy the actual files is the safest and
most complete backup scenario. MyISAM supports this through the three files mentioned earlier. By copying each of the
files associated with a table (*.frm, *.MYD and *.MYI), the user can move or restore the associated data through
simple O/S commands with no break in data or design integrity. Copying applies even between dissimilar operating
systems and CPUs, like 32 bit Windows on Intel to 64 bit Linux on SPARC.

Optimization
With multiple indexing options and specific features of MyISAM, there are some simple approaches to optimizing
MyISAM data that will improve performance and provide noticeable results with little to no overhead. This the major
push of the remainder of this chapter.

9-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Special Features
Some of the special features of the MyISAM storage engine are:
● FULLTEXT indexing – When a record is inserted into a MyISAM table containing a FULLTEXT index, the
data for the indexed fields are analyzed and split into “words”.
● Concurrent inserts - For a MyISAM table, concurrent inserts can be used to add rows at the same time that
SELECT statements are running if there are no deleted rows in middle of the table.
● Prefix compression on indexes - String indexes are space compressed. If the first index part is a string, it is
also prefix compressed. Space compression makes the index file smaller than the worst-case figure if a string
column has a lot of trailing space or is a VARCHAR column that is not always used to the full length. Prefix

Oracle University and BOS-it GmbH & Co.KG use only

compression is used on keys that start with a string. Prefix compression helps if there are many strings with
an identical prefix.
● R-TREE index - This index works with the MyISAM storage engine only and is based on the same tree
structure as the B-Tree. It is an index that works with GIS data and contains indexes optimized for proximity
searches and spatial data. R-Tree indexes have a positive effect performance wise on multi dimension
lookups.

9-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

184 9.3 Where To Use MyISAM

The most typical uses of the MyISAM storage engine include the following applications:
● Logging applications – Due to the small footprint and the speed with which the MyISAM storage engine
performs when performing simple SELECT statements, MyISAM is a perfect choice for tables that are used
primarily for logging purposes.
● Read only/read mostly applications – With the specifics of the MyISAM storage engine and its ability to
perform well with simple SELECT statements, applications that have very few inserts and are mostly read-only
applications are ideal for MyISAM.
● Full table scan reporting – With MyISAM’s FULLTEXT capability, this feature is a huge plus for applications

Oracle University and BOS-it GmbH & Co.KG use only

that perform full text searches.
● Bulk data loads, data crunching – Due to MyISAM’s outstanding response time and indexing features, bulk
data loads using the correct process can be improved with the use of the MyISAM storage engine. In addition,
crunching existing data that utilize the MyISAM storage engine is an effective use of this storage engine and can
improve upon complex applications using aggregate functions.
● Read/write with no transactions low concurrency – Where transactional integrity is not needed, MyISAM is a
perfect choice for the storage engine, due to its low overhead (small footprint and high speed) which is needed
by many organizations today.
● Data warehouses - Many of the features of MyISAM lend themselves nicely to data warehousing applications.
It is an extremely cost-effective and viable tool for this purpose.
185
9.3.1 Where Not to Use MyISAM
In reality, the MyISAM storage engine has limitations and should not be considered when the following issues need to be
addressed:
● Where reliable storage of data is required if there is a risk of a crash or power; this applies to most business
and critical data in high availability web uses.
● Where consistent crash recovery and repair times are required for high availability. This is particularly
applicable to large data sets requiring high availability, because the MyISAM repair time increases with the
amount of data, while the InnoDB recovery time is consistent. There's a big difference between a 20 minute
InnoDB recovery for 200GB and two days for MyISAM doing the same thing.
● Where reliable unattended crash recovery and repair is required. InnoDB automatic repair and recovery is
more reliable than the fastest MyISAM repairs, while the slower more reliable repairs can take far too long in
practice. So, InnoDB can be helpful for the DBA who doesn't want to be woken up in the middle of the night to
help MyISAM get going again or because a recovery is taking too long.
● Where there is a performance benefit from having records clustered in primary key order automatically -
this is available in InnoDB but not MyISAM.

9-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

186 ● Where there is a performance benefit from the database caching writes itself, particularly applicable if
there may be many updates to the same data in a short time.
● Where high concurrency and/or comparatively long application lock times are required.
● Where foreign keys are needed for referential integrity.
● Where transaction support is required - Almost all activities involving money moving around.

Oracle University and BOS-it GmbH & Co.KG use only

9-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

187 9.4 MyISAM Row Storage Format

There are two ways that a table can be assigned the MyISAM storage engine: CREATE TABLE and ALTER TABLE.
When MyISAM is the selected storage engine, the storage engine must determine which record format to utilize to gain
the most advantage to performance of the table. This process of choosing the best record format directly affects the
*.MYD file and ultimately the end user. MyISAM has two possible choices of record format: fixed (static) and dynamic
(row to row variations).
NOTE: Forcing a specific row format

MyISAM tables can be forced to use a specific row format by using either ROW_FORMAT=FIXED or
ROW_FORMAT=DYNAMIC in the CREATE TABLE syntax.

Oracle University and BOS-it GmbH & Co.KG use only

Fixed
Fixed-length format is the default for MyISAM tables and is used when the table contains no variable-length columns
(VARCHAR, VARBINARY, BLOB, or TEXT). CHAR columns will be space-padded to the column width and BINARY
columns are padded with 0x00 bytes. Each row is stored using a fixed number of bytes and is the simplest and most
secure format due to it being the most stable (minimal possibility of corruption). Other advantages of the fixed-length
format:
● Performance – This format is the fastest of the on-disk formats due to the ease with which rows in the data file
can be found on disk: Rows can be looked up based on a row number in the index, then multiplying the row
number by the row length to calculate the row position. This also makes the fixed-length format easy to cache.
In addition, when scanning a table, it is very easy to read a constant number of rows with each disk read
operation.
● Crash recovery – If a computer crashes while the MySQL server is writing to a fixed-format MyISAM file,
myisamchk can easily determine where each row starts and ends, so it can usually reclaim all rows except the
partially written one.
● Reorganization – Reorganizing the data to improve disk usage is unnecessary unless a large number of rows
have been deleted. In this event, utilize OPTIMIZE TABLE or myisamchk –r.
● Character set issues - When multi-byte and variable-byte character sets are used together with CHAR columns,
the maximum size will always be used.
NOTE: Fixed-length disadvantage

The disadvantage to using the fixed-length format is its need to pad certain column types, thus causing the disk
space required to be larger then for identical data stored in a dynamic format.

9-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Dynamic
Dynamic storage format is used if a MyISAM table contains any variable-length columns (VARCHAR, VARBINARY,
BLOB, or TEXT). The one advantage that the dynamic format has over the fixed-length format is the ability for each
row to use only as much space as is required, thus taking up less disk space. This advantage may be all that is needed
to determine the row format to use; however, the following is a list of the associated disadvantages:
● Row fragmentation - If a row becomes larger than the allotted space, dynamic rows are split into as many
pieces as are required, resulting in row fragmentation. This may require running OPTIMIZE TABLE or
myisamchk -r from time to time to improve performance.
● Crash recovery – Dynamic row format tables are more difficult than fixed-length format tables to
reconstruct after a crash, because rows may be fragmented into many pieces and links (fragments) may be

Oracle University and BOS-it GmbH & Co.KG use only

missing.
NOTE: Fixed-length fields and fragmentation

To minimize the MyISAM table fragmentation that can occur, it is best to use fixed-length fields with tables
that are frequently updated or deleted. If this is not possible, large table definitions should be split into two or
more tables separating the variable-length fields from the fixed-length fields.

Compressed Record Format

There is another row format option available in MyISAM that allows for an entire table to be assigned a read-only
status and compressed up to 70% of its original size. This is done using the following syntax on the operating system
command line:

myisampack tablename.MYI

myisampack compresses each column in the table separately. When the table is used later, the server reads into
memory the information needed to decompress columns. MyISAM compressed is faster for RANDOM access to
rows because only the required rows must be read and decompressed. To list all the options that can be used with
myisampack, use the following syntax:

myisampack --help

NOTE: Rebuild indexes

After a table is compressed with myisampack, use myisamchk -rq to rebuild its indexes. Compressed
tables can be uncompressed with myisamchk -u tablename.MYI.

9-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

188 9.5 Indexing

The MyISAM storage engine utilizes sequential access (storing records one after the other) in a single file (*.MYD). This
file is a repository of all the data in the table it represents but has very little interaction with the MyISAM storage engine
beyond its storage function. The key to MyISAM is the associated index file (*.MYI) for each of these data repositories.
The MyISAM storage engines main responsibility is to manage the index file to provide the best performance.

tablename.MYI tablename.MYD

Oracle University and BOS-it GmbH & Co.KG use only

189 The index file (*.MYI) file contains the disk copy of all MyISAM B-tree and R-tree indexes built on a single MyISAM
table. The index file (*.MYI) is broken into two separate sections:
● Header - The .MYI header section is responsible for supporting the MyISAM storage engine by providing an
outline of the index structure, thus providing the means to traverse the index tree.
● Index Blocks – The MyISAM index blocks make up the largest part and the remainder of the index file
(*.MYI). The index blocks are usually made up of 1KB of on-disk pages of data, sometimes 4k or a size
specified with myisam_block_size. For FULLTEXT indexes 2k is used. A single index block contains only key
values pertaining to one index in the table.
NOTE: B-Tree Leaf and Non-Leaf Nodes

The idea behind B-trees is that non-leaf nodes (also known as internal nodes) can have a variable number of child
nodes within some pre-defined range. The size of the index blocks in a table is normally set by the server when the
MYI index file is created, depending on the size of the keys in the indexes present in the table definition. In most
cases, it is set equal to the I/O buffer size.
As data is inserted or removed from the data structure, the number of child nodes varies within a node and so
internal nodes are coalesced or split so as to maintain the designed range. A B-tree is kept balanced by requiring
that all leaf nodes (nodes that have zero child nodes and are the farthest from the root node) are at the same depth.
This depth will increase slowly as elements are added to the tree, but an increase in the overall depth is infrequent,
and results in all leaf nodes being one more hop further removed from the root.

9-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Quiz
In this exercise you will answer the following questions pertaining to the MyISAM storage engine.
10. What is the disk footprint of MyISAM tables? What are the purposes of the different files?
__________________________________________________________________________________
__________________________________________________________________________________
11. What uses are best suited to MyISAM?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

12. 3.What types of indexes are supported by MyISAM?
__________________________________________________________________________________
__________________________________________________________________________________
13. What is the difference between the different MyISAM row formats? What are their respective advantages?
Do they have implications to the index file?
__________________________________________________________________________________
__________________________________________________________________________________
14. Why are MyISAM indexes organized into blocks?
__________________________________________________________________________________
__________________________________________________________________________________
15. 6.How many key values are in each index block?
__________________________________________________________________________________
__________________________________________________________________________________

9 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

190 9.6 Locking

The MyISAM storage engine protects against corruption of data by different threads updating in conflicting ways by
using table-level locking. The MyISAM storage engine utilizes one of three types of locks depending on the request
issued by the connecting thread:
● READ LOCAL (SHARED)– This type of lock allows an INSERT to append only to the end of the data file. If
the INSERT requires a push of data anywhere else but to the end of the data file (beginning or middle of the
data file), the INSERT statement is delayed until the READ LOCAL LOCK is released.
● READ – This type of lock is activated when the actual data file (*.MYD), rather than the in-memory cache, is
used to get information. When this lock is activated, all UPDATE, INSERT and DELETE statements are delayed

Oracle University and BOS-it GmbH & Co.KG use only

until the lock is released. READ locks are shared locks.
● WRITE – This type of lock is activated whenever an UPDATE or DELETE request is submitted, or an INSERT is
received that would require adding the data anywhere but to the end of the data file. This lock prevents any
activity on the table until the lock is released. WRITE locks are exclusive locks.
NOTE: Tables designed to Log Activity

With the MyISAM storage engine's capability to perform INSERT operations while SELECT queries are running
against the data (through the READ LOCAL lock type), it makes an excellent choice for applications that are
responsible for logging continual activities while also giving end users the ability to run reports or other query
operations on the data contained. This is made possible by the fact that MyISAM knows that all new records will
always occur at the end of the data file and thus there is no need to hold up SELECT statements that request data
from anywhere else in the table.

191 Concurrent Inserts

With the MyISAM storage engine, it is possible to allow an INSERT statement to execute concurrently with SELECT
statements by appending new records to the end of the data file, even if there are other gaps that could be filled first.
Essentially, it trades using (temporarily) a little more storage space to achieve better read/write concurrency. Whether an
INSERT will be a concurrent insert is determined by the concurrent-insert global status variable. This variable can be set
using mysqld --concurrent_insert=x or SET GLOBAL concurrent_insert=x. The concurrent-insert
variable can take any one of the following values:
● 0 - (Off) This tells the server to not allow any concurrent inserts.
● 1 - (Default) Enables concurrent insert for MyISAM tables that don't have holes.
● 2 - Enables concurrent inserts for all MyISAM tables. If the table has a hole and is in use by another thread the
new row will be inserted at the end of the table. If the table is not in use, MySQL obtains a normal WRITE lock
and inserts the new row into the hole.

9 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

192 Priorities
When running a SQL statement that interacts with a MyISAM table, an individual user can determine the importance
level that their individual statements have in regards to priority execution on the MySQL server. The following is a
list of the priority commands that can be used:
● HIGH_PRIORITY – This command gives the SQL statement being executed the ability to move to the top
of the list of any waiting SQL statements (from all threads) and can be used with the following SQL
statements:
○ SELECT - When SELECT HIGH_PRIORITY ... is utilized, MySQL gives the SELECT a higher
priority than a statement that updates a table. This should only be used for queries that are very fast and
must be done at once. A SELECT HIGH_PRIORITY query that is issued while the table is locked for

Oracle University and BOS-it GmbH & Co.KG use only

reading runs even if there is an update statement waiting for the table to be free. HIGH_PRIORITY
cannot be used with SELECT statements that are part of a UNION.
○ INSERT – When INSERT HIGH_PRIORITY ... is utilized, MySQL stops any concurrent insert
capabilities and gives the INSERT statement complete control of the table until the statement has
executed completely.
NOTE: Priority levels

When using the INSERT HIGH_PRIORITY statement, the server automatically overrides any existing
low_priority_updates setting. The low_priority_updates setting, when set to 1, forces all
INSERT, UPDATE, DELETE, and LOCK TABLE WRITE statements to wait until there is no pending
SELECT or LOCK TABLE READ on the affected table. This variable previously was named
sql_low_priority_updates. In addition, by default writing queries have a higher priority than reading
queries, so by default there are two priority queues.
● LOW_PRIORITY – This command places the SQL statement being executed at the bottom of the list of any
waiting and new SQL statements (from all threads) and will only run when all waiting and new SQL
statements have completed. This command can be used with the following SQL statements:
○ INSERT – When INSERT LOW_PRIORITY ... is utilized, MySQL delays the execution of the
statement (and stops any other work from the particular client) until no other clients are reading from the
table. This includes any other clients that may access the table while the INSERT LOW_PRIORITY
statement is waiting to execute. In an environment where there is a large number of clients that can be
reading from the server at any one time this could cause the client to wait a very long time. This should
normally not be used with MyISAM tables because doing so disables concurrent inserts.
○ UPDATE – The UPDATE LOW_PRIORITY ... works identical to the INSERT LOW_PRIORITY
statement and should also be avoided unless absolutely necessary.
○ DELETE – The DELETE LOW_PRIORITY ... also works the same as the INSERT LOW_PRIORITY
by not executing until all clients are done reading from the table. This can cause integrity issues in that
another client can be viewing data that should be deleted immediately.

9 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

NOTE: Lost in the queue

Using LOW_PRIORITY with UPDATE, INSERT and DELETE can cause the queries to wait in the queue forever,
if there is no time of day when clients aren't reading from the table.

NOTE: INSERT DELAYED

The use of the INSERT DELAYED statement in SQL statements is not directly related to locking issues in
MyISAM, but has a bearing on many of the discussions in the locking section. When the INSERT DELAYED
statement is executed, the server puts the row or rows to be inserted into a buffer, and the client issuing the

Oracle University and BOS-it GmbH & Co.KG use only

INSERT DELAYED statement can then continue immediately. If the table is in use, the server holds the rows.
When the table is free, the server begins inserting rows, checking periodically to see whether there are any new
read requests for the table. If there are, the delayed row queue is suspended until the table becomes free again.
Another major benefit of using INSERT DELAYED is that inserts from many clients are bundled together and
written in one block. This is much faster than performing many separate inserts.
INSERT DELAYED is slower than a normal INSERT if the table is not otherwise in use. There is also the
additional overhead for the server to handle a separate thread for each table for which there are delayed rows. This
means that INSERT DELAYED should be used only when it is absolutely necessary.

9 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

193 9.7 Key Cache

The key cache is a repository for frequently used B-tree index data blocks for all MyISAM tables. When a client
attempts to access index (key) data from a MyISAM table, the key cache is checked to determine whether the
required index block is contained within it. If the requested index is in the key cache, the key cache returns the
requested block from its internal buffers. If the key cache does not contain the requested block, the block is read from
the index file (*.MYI) and stored in the key cache memory for present use and any possible future use. Any future
requests will utilize the index in the key cache versus having to read the index file.

Key Cache

Oracle University and BOS-it GmbH & Co.KG use only

Block Not
Key Found,
MySQL Is requested
Cache Pull Index from
Server index in Key
Memory Index File and
Cache?
store to Key
Cache
Tablename.MYI
Found Block,
Return Index
Key to Server

NOTE: “Cold” versus “warm” data blocks

The KEY_CACHE structure contains a number of linked lists of accessed index data blocks. These blocks each
represent a single, fixed size block of data read from an index file, and are kept in memory until they are
determined to be “cold” index blocks (not being used often enough to be kept in memory). When a block of
data is determined to be “cold”, purged from memory. Even though the KEY_CACHE uses a least recently used
(LRU) approach to determining which blocks of data are “cold” or “warm”, it is designed to keep all blocks in
memory that are associated with root B-tree levels.
With statements that have the potential to change it is existing data, the key cache first writes the changes to the
internally buffered index block and marks it as “dirty”. If this “dirty” block is accessed by the key cache for the
purpose of purging it, the key cache first writes the changes to the index file before removing it from memory.

key_buffer_size
The key_buffer_size system variable controls the size of the key cache. If this variable is set equal to zero, no
key cache is used. The key cache also is not used if the key_buffer_size value is too small to allocate the
minimal number of block buffers (8). When the key cache is not operational, index files are accessed using only the
native file system buffering provided by the operating system (in other words, table index blocks are accessed using
the same strategy as that employed for table data blocks).

9 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

9.7.1 Multiple Key Caches

The MyISAM storage engine provides shared access to the key cache thus improving performance; however, this does
not eliminate contention among threads entirely. They still compete for control structures that manage access to the key
cache buffers. To reduce key cache access contention further, MySQL also provides multiple key caches. This feature
enables the assignment of different key caches to different table indexes. With high CPU use, particularly high system
CPU use, suspect contention for access to the key cache and try using multiple key caches to improve performance
194 CACHE INDEX
Where there are multiple key caches, the server must know which cache to use when processing queries for a given
MyISAM table. By default, all MyISAM table indexes are cached in the default key cache. To assign table indexes to a

Oracle University and BOS-it GmbH & Co.KG use only

specific key cache, use the CACHE INDEX statement. For example, the following statement assigns indexes from the
tables t1, t2, and t3 to the key cache named hot_cache:

CACHE INDEX t1, t2, t3 IN hot_cache;

+---------+--------------------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+--------------------+----------+----------+
| test.t1 | assign_to_keycache | status | OK |
| test.t2 | assign_to_keycache | status | OK |
| test.t3 | assign_to_keycache | status | OK |
+---------+--------------------+----------+----------+

The key cache referred to in a CACHE INDEX statement can be created by setting its size with a SET GLOBAL
parameter setting statement or by using server startup options. For example:

SET GLOBAL hot_cache.key_buffer_size=128*1024;

Destroying a key cache

To destroy a key cache, set its size to zero:

SET GLOBAL hot_cache.key_buffer_size=0;

Note that the default key cache can not be destroyed. Any attempt to do this will be ignored.

9 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

195 Using the CACHE INDEX

For servers that are exceptionally active, MySQL recommends a strategy that uses three key caches:
● HOT - A “hot” key cache that takes up 20% of the space allocated for all key caches. This cache is for tables
that are heavily used for searches but that are not updated:
○ The hot cache is used only for retrieval queries, so its contents are never modified. Consequently,
whenever an index block needs to be pulled in from disk, the contents of the cache block chosen for
replacement need not be flushed first.
○ For an index assigned to the hot cache, if there are currently no queries requiring an index scan, there is
a high probability that the index blocks corresponding to non-leaf nodes of the index B-tree remain in

Oracle University and BOS-it GmbH & Co.KG use only

the cache.
● COLD - A “cold” key cache that takes up 20% of the space allocated for all key caches. This cache is for
medium-sized, intensively modified tables, such as temporary tables.
○ An update operation frequently executed for temporary tables is performed much faster when the
updated node is in the cache and need not be read in from disk first. If the sizes of the indexes of the
temporary tables are comparable with the size of cold key cache, the probability is very high that the
updated node is in the cache.
● WARM - A “warm” key cache that takes up 60% of the key cache space. This cache should be set up as the
default key cache, to be used by default for all other tables.
CACHE INDEX sets up an association between a table and a key cache, but the association is lost each time the
server restarts. For the association to take effect each time the server starts it is best to use an option file that includes
variable settings that configure the key caches, and an init-file option that names a file containing CACHE
INDEX statements to be executed. For example, with a total key buffer of 3G, the following settings would be set:

hot_cache.key_buffer_size = 500M
cold_cache.key_buffer_size = 500M
key_buffer_size = 2G
init_file=/path/to/data-directory/mysqld_init.sql

The statements in mysqld_init.sql are executed each time the server starts. The file should contain one SQL statement
per line. The following example assigns several tables each to hot_cache and cold_cache:

CACHE INDEX db1.t1, db1.t2, db2.t3 IN hot_cache

CACHE INDEX db1.t4, db2.t5, db2.t6 IN cold_cache

9 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

196 9.7.2 Preloading Indexes

The LOAD INDEX INTO CACHE statement pre-loads a table index into the key cache to which it has been assigned by
an explicit CACHE INDEX statement, or into the default key cache. Adding the IGNORE LEAVES modifier causes only
blocks for the non-leaf nodes of the index to be preloaded. The following statement pre-loads nodes (index blocks) of
indexes for the tables t1 and t2:

LOAD INDEX INTO CACHE t1, t2 [IGNORE LEAVES];

+---------+--------------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+--------------+----------+----------+

Oracle University and BOS-it GmbH & Co.KG use only

| test.t1 | preload_keys | status | OK |
| test.t2 | preload_keys | status | OK |
+---------+--------------+----------+----------+

This statement pre-loads all index blocks from t1. It pre-loads only blocks for the non-leaf nodes from t2.
Currently, pre-loading of indexes for a table is only allowed if all are using the same page size. Any tables using the 4k
size or any other size will prevent key cache pre-loading. This usually happens when FULLTEXT indexes or indexes with
long character columns are present.
197
9.7.3 Midpoint Insertion Strategy
By default, the key cache management system uses the LRU strategy for choosing key cache blocks to be evicted, but it
also supports a more sophisticated method called the midpoint insertion strategy. When using the midpoint insertion
strategy, the LRU chain is divided into two parts: a hot sub-chain and a warm sub-chain. The division point between two
parts is not fixed, but the key cache management system takes care that the warm part is not “too short,” always
containing at least key_cache_division_limit percent of the key cache blocks. The
key_cache_division_limit is a component of the structured key cache variables, so its value is a parameter that
can be set per cache.
● Initial Entry into Key Cache - When an index block is read from a table into the key cache, it is placed at the
end of the warm sub-chain. After a certain number of hits (accesses of the block), it is promoted to the hot sub-
chain. At present, the number of hits required to promote a block (3) is the same for all index blocks.

9 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Initial Entry into Key Cache Promotion to Hot sub -chain First Candidate for Eviction

Key Cache Memory Key Cache Memory Key Cache Memory

#### #### #### #### #### ####
#### #### #### #### #### ####
#### #### #### #### #### ####
#### #### #### #### #### #### Hot
Hot Hot
#### #### #### #### #### ####
sub- sub- sub-
#### #### #### #### #### ####
chain chain chain
#### #### #### #### #### ####
#### #### #### #### #### ####
#### #### #### #### #### ####
#### #### #### 71470 #### ####

Oracle University and BOS-it GmbH & Co.KG use only

71470 #### #### #### #### ####
#### #### Warm #### #### Warm #### #### Warm
#### #### sub- #### #### sub- #### #### sub-
#### #### #### #### #### ####
chain chain chain
#### #### #### #### #### ####
#### #### #### #### #### 71470

● Promotion to Hot sub-chain - A block promoted into the hot sub-chain is placed at the end of the chain.
The block then circulates within this sub-chain. If the block stays at the beginning of the sub-chain for a long
enough time, it is demoted to the warm chain. This time is determined by the value of the
key_cache_age_threshold component of the key cache.
● First Candidate for Eviction - The threshold value prescribes that, for a key cache containing N blocks, the
block at the beginning of the hot sub-chain not accessed within the last N ×
key_cache_age_threshold / 100 hits is to be moved to the beginning of the warm sub-chain. It
then becomes the first candidate for eviction, because blocks for replacement always are taken from the
beginning of the warm sub-chain.
The midpoint insertion strategy allows more-valued blocks to be kept in the cache. If the plain LRU strategy is
preferred, leave the key_cache_division_limit value set to its default of 100.
198
9.7.4 Other Key Cache Status Variables
There are a number of status variables that assist in maintaining, monitoring and optimizing the key cache for
MyISAM tables. The following lists the other key cache status variables not discussed up to this point:
● key_blocks_not_flushed – This variable displays the number of key blocks in the key cache that
have changed but have not yet been flushed to disk. These can be considered “dirty” key blocks in the key
cache. The index will be corrupted during a crash and will need to be repaired before it can be used again.
● key_blocks_used – This variable displays the number of used blocks in the key cache. This value is a
high-water mark that indicates the maximum number of blocks that have ever been in use at one time. This
value should be high. The more blocks in the key cache, the less the server is using disk-based I/O to
examine the index data.
● key_blocks_unused – This variable displays the number of unused blocks in the key cache. This value
can be used to determine how much of the key cache is in use.

9 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

● key_read_requests - This variable represents the number of requests that have been executed to read a key
block from the cache.
199 ● key_reads – This variable displays the number of physical reads of a key block from disk. The cache miss
ratio can be calculated as:

key_reads/key_read_requests

If the ratio of key_reads to key_read_requests is high, more than about one in one hundred, consider
increasing key_buffer_size, remembering to balance all of the RAM allocations on the server, so it may
need to be left as it is or even reduced to obtain the best balance. In addition to the cache miss ratio, it is also

Oracle University and BOS-it GmbH & Co.KG use only

possible to watch for key_reads/sec, and match against the I/O system.
● key_write_requests – This variable displays the number of requests to write a key block to the cache.
● key_writes - This variable stores the number of actual writes - logical and physical key block writes - to
disk. It is normal for the write miss ratio to be much larger than the read miss ratio. If necessary, consider using
DELAY_KEY_WRITE for most frequently updated tables, at the risk of an increased chance of index corruption
and the need to repair the table if there is a crash. Use myisam-recover=BACKUP,FORCE if this is going to
be utilized.

9 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

InLine Lab 9-A

In this exercise you will perform tests using the key cache system variables by monitoring the key_% variables.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered
the contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.
Action: After logging into the MySQL server as root, type the following in the mysql client:

Oracle University and BOS-it GmbH & Co.KG use only

SET GLOBAL key_buffer_size = 16 * 1024;

Effect: This effectively resets the key cache.

Step 2. Review the key cache state

Action: Issue the following SQL command to view the status of the key cache:

SHOW GLOBAL STATUS LIKE 'key%';

Effect: A response similar to the listing below will be displayed in the mysql client:

+------------------------+-------+
| Variable_name | Value |
+------------------------+-------+
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 14 |
| Key_blocks_used | 0 |
| Key_read_requests | 0 |
| Key_reads | 0 |
| Key_write_requests | 0 |
| Key_writes | 0 |
+------------------------+-------+
7 rows in set (#.## sec)

Step 3. Execute some queries and monitor key cache state changes

Action: In the world database, issue the following SQL command to query the Country table:

SELECT * FROM Country WHERE Code = 'SWE'\G

Effect: The Country table is queried and the result ("Sweden") is displayed in the mysql client.

9 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Action: Issue the following SQL command again to view the status of the key cache now:

SHOW GLOBAL STATUS LIKE 'key%';

Are there changes to any key cache variables? If so, which variables have been changed? What are their values?
Action: In the world database, issue the following SQL command to query the Country table:

SELECT * FROM Country WHERE Code = 'SWZ'\G

Effect: The Country table is queried and the result ("Swaziland") is displayed in the mysql client.

Oracle University and BOS-it GmbH & Co.KG use only

Action: Issue the following SQL command again to view the status of the key cache now:

SHOW GLOBAL STATUS LIKE 'key%';

Are there changes to any key cache variables? If so, which variables have been changed? What are their values?
Action: In the world database, issue the following SQL command to query the Country table:

SELECT * FROM Country WHERE Code = 'FIN'\G

Effect: The Country table is queried and the result ("Finland") is displayed in the mysql client.

Action: Issue the following SQL command again to view the status of the key cache now:

SHOW GLOBAL STATUS LIKE 'key%';

Are there changes to any key cache variables? If so, which variables have been changed? What are their values?

9 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

200 9.8 MyISAM Tuning

Up to this point, the MyISAM storage engine has been thoroughly examined and many ideas have been presented to
improve the performance of MyISAM tables. The following settings can also be managed for the purpose of
improving performance and better tuning MyISAM tables for the needs of the end users and/or database applications:
● bulk_insert_buffer_size - When inserting many rows from the same client at the same time, use
INSERT statements with multiple VALUES lists to insert several rows at a time. This is considerably faster
(many times faster in some cases) than using separate single-row INSERT statements. If data is being added
to a non-empty table, tuning the bulk_insert_buffer_size variable can make data insertion faster.
● myisam_sort_buffer_size – The size of the buffer that is allocated when sorting MyISAM indexes

Oracle University and BOS-it GmbH & Co.KG use only

during a REPAIR TABLE or when creating indexes with CREATE INDEX or ALTER TABLE. This
variable should be set high for index builds - as large as the largest index in the table if there is sufficient
RAM.
● myisam_max_sort_file_size - The maximum size of the temporary file that MySQL is allowed to
use while re-creating a MyISAM index (during REPAIR TABLE, ALTER TABLE, or LOAD DATA
INFILE). If the file size would be larger than this value, the index is created using the key cache instead,
which is slower. The value is given in bytes. Set this value a bit larger than the largest index file if there is
enough disk space, so REPAIR TABLE is sure to be done by sort, which can be tens to hundreds of times as
fast as repair by key cache.
201 ● myisam_recover=<option> - This variable controls the MyISAM storage engine recovery mode. The
option value is any combination of the following values:
○ DEFAULT – This option is the same as not giving any option to –myisam-recover
○ BACKUP - If the data file is changed during recovery, this option tells the server to save a backup of the
tbl_name.MYD file as tbl_name-datetime.BAK.
○ FORCE – This option runs recovery even if more than one row would be lost from the .MYD file.
○ QUICK – This option tells the server to put off checking the rows in the table to see if there are any
delete blocks.
If multiple values are used, separate them by commas. The value of "" can also be used to disable this option.
If this option is used, each time mysqld opens a MyISAM table, it checks whether the table is marked as
crashed or wasn't closed properly. If this is the case, mysqld runs a check on the table. If the table was
corrupted, mysqld attempts to repair it.

9 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

202 9.9 Other Optimization Tools for MyISAM

The MyISAM storage engine is an extremely flexible and powerful part of the MySQL server. This section will conclude
with additional thoughts on how to optimize MyISAM to provide the utmost performance. The following is a list of ideas
that can be implemented rather easily and can provide great returns in performance:
● Not null – By declaring columns as NOT NULL, the tables can save some substantial disk space, especially for
larger tables.
● Optimize table – Defragmenting, reclaiming free space and making concurrent inserts work again are just some
of the benefits of running OPTIMIZE TABLE on a regular basis. For MyISAM tables, OPTIMIZE TABLE
works as follows:

Oracle University and BOS-it GmbH & Co.KG use only

1. If the table has deleted or split rows, repair the table.
2. If the index pages are not sorted, sort them.
3. If the table's statistics are not up to date, update them.
203 ● Disable indexes – When deleting, updating and/or adding a high percentage of the data it is a best practice to
disable indexes; finding the ideal percentage changed for the specific application. It's always best when doing a
bulk load into an empty table. This can be accomplished in one of the following two ways:
○ ALTER TABLE t DISABLE KEYS (keeps UNIQUE keys active). After the action is accomplished that
initiated the disabling of the keys, using ALTER TABLE t ENABLE KEYS will reactivate the indexes.
... or …
○ Drop indexes entirely, and add them back in later.
● Data warehouse build/load optimizations – When dealing with large historical data, MERGE tables are an
effective approach to not only improving performance but also for the purpose of managing the large amount of
data associated with data warehouses.
● Index tree – The index tree should fit completely in the key cache.
● Column types - Use fixed width column types for the entire table definition for extra speed, if possible - don’t
add too much to row size though!

9 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Further Practice
In this further practice you will test the features of the key cache to include performing a midpoint insertion strategy.
1. Populate the city_huge table with approximately 800,000 records by passing an argument of 200 to the
create_city_huge stored procedure. This might take some time to run, so now might be a good time
204 for a break.
2. Reset the key cache to a 16K key cache using the following SQL statement:

SET GLOBAL key_buffer_size = 16 * 1024;

Oracle University and BOS-it GmbH & Co.KG use only

3. Execute a large query against the city_huge table that performs a full index scan using the following SQL
statement:

SELECT SUM(Population) FROM city_huge;

Record the time it took to execute the query: _____________ (Query A-1)
4. Execute a FLUSH TABLES command to flush the query cache. This will ensure that the next command
gives an accurate result.
5. Execute a large query against the city_huge table that performs a full table scan (eliminating the file
system cache effect) using the following SQL statement:

SELECT MAX(District) FROM city_huge;

Record the time it took to execute the query: _____________ (Query A-2)
6. Reset the key cache to 75% of system memory.
7. Execute a FLUSH TABLES command again to flush the query cache.
8. Execute the original full index scan query (step 3) against the city_huge table again.
Record the time it took to execute the query: _____________ (Query B-1)
9. Execute a FUSH TABLES command again to flush the query cache.
10. Execute the original full table scan query (step 5) against the city_huge table again.
Record the time it took to execute the query: _____________ (Query B-2)
11. Execute a FLUSH TABLES command again to flush the query cache.
12. Execute the original full index scan query (step 3) against the city_huge table again.
Record the time it took to execute the query: _____________ (Query C-1)
13. Compare the three different queries (A, B and C) performance against each other.
14. Reset the key cache to 50M.

9 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

15. Execute the mid-point.sql file using the following O/S command to populate the cache with one iteration:

mysqlslap -uroot -p -i 1 -q mid-point.sql –create_schema=world

16. In the mysql client, execute the following SQL statement to reset the key cache statistics:

FLUSH STATUS;

17. Execute the mid-point.sql file again using mysqlslap.

18. In the mysql client, determine the hit rate by reviewing the key cache statistics and using the following

Oracle University and BOS-it GmbH & Co.KG use only

formula:

100 - (Key_reads/Key_read_requests) * 100

What is the hit rate percentage: ______________ (hit rate A)

19. Execute a FLUSH TABLES command again to flush the query cache.
20. Execute the original full index scan query (step 3) against the city_huge table again.
21. In the mysql client, determine the hit rate of key cache again.
What is the hit rate percentage: ______________ (hit rate B)
22. Reset the key cache statistics.
23. Execute the mid-point.sql file again using the following O/S command to populate the cache with ten
iterations:

mysqlslap -uroot -p -i 10 -q mid-point.sql –create_schema=world

24. In the mysql client, determine the hit rate of key cache again.
What is the hit rate percentage: ______________ (hit rate C)
25. Compare hit rate A and hit rate C.
26. Set the key_cache_division_limit system variable to 50 by issuing the following SQL statement:

SET GLOBAL key_cache_division_limit = 50;

27. Reset the key cache statistics.

28. Execute the mid-point.sql file again to populate the cache with ten iterations.
29. In the mysql client, determine the hit rate of key cache again.
What is the hit rate percentage: ______________ (hit rate D)
30. Reset the key cache statistics.

9 - 25
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

31. Execute a FLUSH TABLES command again to flush the query cache.
32. Execute the original full index scan query (step 3) against the city_huge table again.
33. In the mysql client, determine the hit rate of key cache again.
What is the hit rate percentage: ______________ (hit rate E)
34. Compare all the hit rates to determine the best use of the key cache.

Oracle University and BOS-it GmbH & Co.KG use only

9 - 26
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

205 9.10 MERGE Storage Engine

The MERGE engine type allows for a combination of a number of identical tables into a single table. This allows queries
executed against this “table” to return the results from multiple tables as if they were just one table. Each table merged
must have the same table definition.
The MERGE table is particularly effective for logging data directly or indirectly into a MySQL database. Other uses
include creating an individual table per day, week or month and producing aggregate queries from multiple tables.

Limitations:
206 ● MyISAM tables - Only MyISAM tables can be merged into a MERGE table. If other table types were allowed,
the reality is that they would probably not be needed (for example, InnoDB).

Oracle University and BOS-it GmbH & Co.KG use only

● Identical table definition – This restriction is strictly enforced.
As stated, the MERGE storage engine uses a collection of identically structured MyISAM tables. Each MERGE table
has:
● A format file (.frm)
● A file that lists the constituent MyISAM tables (.MRG)

2006_QTR1_Stats
2006_QTR1_Stats
207
2006_QTR1_Stats.MRG
2006_QTR1_Stats.MRG

2006_03_Stats

2006_02_Stats

2006_01_Stats
2006_03_Stats.MYI
2006_QTR1_Stats
2006_03_Stats.MYD
.frm
(Metadata)

2006_02_Stats.MYD 2006_02_Stats.MYI

2006_01_Stats.MYD 2006_01_Stats.MYI

MyISAM tables can be added or removed to an existing MERGE table with the ALTER TABLE command.

9 - 27
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

208 MERGE Locks

All locks on a MERGE table escalate and acquire a lock for each underlying MyISAM table; the locks are table-level
locks. The locking strategies are exactly the same as for MyISAM tables:
● Reads use shared locks (READ LOCK).
● Writes use exclusive locks (WRITE LOCK).
● Concurrent inserts supported as with MyISAM.

Advantages
● MERGE tables can assist in exceeding possible file size limitations of the O/S.

Oracle University and BOS-it GmbH & Co.KG use only

● MySQL manages contention between queries for MERGE table access using table-level locking.
● A MERGE table is very portable.
● MERGE tables support SELECT, DELETE, UPDATE, and INSERT statements.

Disadvantages
● Increase in the number of file descriptors required based on the increased number of files.
● Index reads are slower as MySQL has to search the indexes of multiple tables.
● No global indexes are available.

9 - 28
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Quiz
In this exercise you will answer the following questions pertaining to the MyISAM Storage Engine.
1. Describe the different concurrent insert modes in MyISAM
__________________________________________________________________________________
__________________________________________________________________________________
2. How many priority queues are there for MyISAM tables? How can you change the priority of a query?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

3. What is stored in the MyISAM key caches?
__________________________________________________________________________________
__________________________________________________________________________________
4. How can MyISAM secondary key caches be monitored?
__________________________________________________________________________________
__________________________________________________________________________________
5. What strategy does MyISAM use for evicting blocks from the key caches by default? How can this behavior be
changed?
__________________________________________________________________________________
__________________________________________________________________________________
6. How can MyISAM tables be optimized for bulk inserts?
__________________________________________________________________________________
__________________________________________________________________________________
7. How can you optimize MyISAM table reparation and ALTER TABLE operations?
__________________________________________________________________________________
__________________________________________________________________________________
8. What can you do with the MERGE storage engine?
__________________________________________________________________________________
__________________________________________________________________________________
9. What advantages does the MERGE storage engine provide?
__________________________________________________________________________________
__________________________________________________________________________________

9 - 29
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

Further Practice
In this further practice you will test the MyISAM index building.
1. In the mysql client, export the city_huge table to a flat file using the following SQL statement:

209 SELECT * INTO OUTFILE '/tmp/city_huge.txt' FROM city_huge;

2. In an O/S terminal, setup up monitoring for the mysqladmin processlist by entering the flowing
statement:

watch mysqladmin -uroot pro

Oracle University and BOS-it GmbH & Co.KG use only

3. In another O/S terminal, setup up monitoring of disk usage by entering the following statement:

watch -d ls -l /usr/local/mysql/data/world/city_huge*

You may have to be logged in as root to access the data directory.

4. In another O/S terminal, setup up monitoring of CPU/disk usage by entering the following statement:

vmstat 1

5. In the mysql client, set the key cache to a small number (16K).
6. Set the MyISAM sort buffer to 4 by entering the following SQL statement:

SET SESSION myisam_sort_buffer_size = 4;

7. Truncate the city_huge table, and reload the records exported in step 1.
Watch the monitors; how long did the import take: _____________ (Import A)
8. In the mysql client, set the key cache to a big number (75% of system memory).
9. Truncate the city_huge table, and reload the records exported in step 1.
Watch the monitors; how long did the import take: _____________ (Import B)
10. In the mysql client, set the key cache to a small number (16K).
11. Set the MyISAM sort buffer to 80% of system memory.
12. Truncate the city_huge table, and reload the records exported in step 1.
Watch the monitors; how long did the import take: _____________ (Import C)
13. Compare the different times for each of the imports.
14. Shut down the other terminal windows that were opened during this lab exercise.

9 - 30
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 9 - MyISAM Storage Engine

9.11 Chapter Summary

210 In this lesson, you have learned to:
● List the characteristics of the MyISAM storage engine
MyISAM allows fast retrieval of data via a non-clustered index and data organization. MyISAM is the default
storage engine for all versions of MySQL.
● Explain where MyISAM tables are most effective
The most typical usage of the MyISAM storage engines includes the following applications: logging
applications, read only/read mostly applications, full table scan reporting, bulk data loads/data crunching and
read/write with no transactions low concurrency.

Oracle University and BOS-it GmbH & Co.KG use only

● Describe the different MyISAM row storage formats
There are two possible choices that MyISAM has to choose when determining the record format to use: fixed
(static) and dynamic (row to row variations). Compressed record format is another row format option available
in MyISAM that allows for an entire table to be assigned a read-only status and compressed up to 70% of its
original size.
● Explain the MyISAM indexing features
The MyISAM storage engines main responsibility is to manage the index file and ensure that that file is
organized in such a way to provide the best performance.
● Utilize the MyISAM locking characteristics
The MyISAM storage engine utilizes one of three types of locks depending on the request issued by the
connecting thread: READ LOCAL, READ or WRITE.
● Describe the MyISAM key cache
The key cache is a repository for frequently used B-tree index data blocks for all MyISAM tables.
● List additional MyISAM tuning aspects
bulk_insert_buffer_size, myisam_sort_buffer_size, myisam_max_sort_file_size
and myisam_recover
● Describe other MySQL optimization tools
The following is a list of ideas that can be implemented rather easily and can provide great returns in
performance: not null, optimize table, disable indexes, etc.
● List the characteristics of the MERGE storage engine
The MERGE engine type allows for a combination of a number of identical tables into a single table.

9 - 31
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

211
10 INNODB STORAGE ENGINE

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

212 10.1 Learning Objectives

This chapter explains the details of the InnoDB storage engine and how it relates to performance tuning. In this
lesson, you will learn to:
● List the characteristics of the InnoDB storage engine
● Describe the InnoDB transactions
● Explain the internal InnoDB buffers
● Describe how InnoDB handles crash recovery
● List InnoDB best practices

Oracle University and BOS-it GmbH & Co.KG use only

● Describe other InnoDB optimization tools
● Compare InnoDB and MyISAM storage engines

10 - 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

213 10.2 InnoDB

InnoDB is a storage engine for MySQL, included as standard in all current binaries distributed by MySQL AB. The
original company that developed InnoDB was Innobase, which was acquired by Oracle® in October of 2005. InnoDB is
not a standalone database product: it is distributed as a part of the MySQL database and is distributed under an open
source license.

Storage Medium: Shared Tablespace

InnoDB, like all MySQL storage engines, uses one or more files to handle operations within the data sets that are located
under the data_dir directory. The data_dir directory contains one subdirectory for each schema (database) residing
on the server. The InnoDB storage engine, by default, stores all InnoDB tables in a single shared tablespace located in the

Oracle University and BOS-it GmbH & Co.KG use only

root directory (unless otherwise identified in the my.cnf file). The three files created to support InnoDB tables are:
● ibdata – These files which are labeled sequentially (ibdata1, ibdata2, etc.) and located directly under the
data directory by default, contain the actual data for all tables using the InnoDB storage engine. In addition the
ibdata files contain an internal data dictionary and indexes for all the tables using the InnoDB storage engine
within the server. These files can grow to the size limitation of the operating system itself.
NOTE: Setting configuration options

If the server is started with no InnoDB configuration options, MySQL creates an auto-extending 10MB data file
named ibdata1 and two 5MB log files named ib_logfile0 and ib_logfile1 in the MySQL data directory.
To ensure that InnoDB’s performance is good, configuration options should be set to meet the needs of the
application that will be utilizing the tables. Configuration options and parameters will be discussed throughout the
remainder of this chapter.
● ib_logfile – These files are used with the InnoDB transaction processing system and are usually followed
by a sequential number to identify the log segment. The files hold information related to transaction history and
are located by default in the data directory.
● table_name.frm - This file (located in the schema directory under the MySQL datadir) contains all the
meta information about the InnoDB table definition. This is the only file, in an InnoDB shared tablespace setup,
that is located in the actual schema directory.

10 - 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

MySQL
MySQL

Data
Data

mysql
mysql

test
test
City.frm
(Metadata)

Oracle University and BOS-it GmbH & Co.KG use only

world
world

Internal data
Internal data
dictionary Transaction Country.frm
dictionary Transaction
(Metadata)
---- ---- --- -- -- -----

Insert undo
--- ---- --- ----- -- -- -
records,
Insert undo records,
logs redo
redolog
log
logs records, etc.
Update records, etc.
Update
undo
undologs
logs Countrylanguage.frm
ibdata files Ib_logfile files
(Metadata)

214 Storage Medium: Multiple Tablespaces

InnoDB tables can be stored in individual table spaces and the associated indexes in their own file. This feature is
called “multiple tablespaces” because in effect each table has its own tablespace. Using multiple tablespaces can be
beneficial to users who want to move specific tables to separate physical disks or who wish to restore backups of
single tables quickly without interrupting the use of the remaining InnoDB tables. In addition, using multiple
tablespaces provides the ability to reclaim file system space.
When “multiple tablespaces” is activated, the system still requires the ibdata, ib_logfile's and
table_name.frm files; however, new data files are created for each table in the schema directory located under
the MySQL datadir. To support the individual table, the following file is included in the schema directory for each
individual InnoDB table:
● table_name.ibd – If the MySQL server is started with the innodb_file_per_table setting,
InnoDB will store each individual table in its own file *.ibd. This is called “multiple tablespaces” and acts
similar to MyISAM with the exception that all the data and the indexes are stored in this individual file
(MyISAM tables create two files: one for the data and one for the indexes).
NOTE: Enabling Multiple Tablespaces

To enable multiple tablespaces in InnoDB, the innodb_file_per_table setting should be placed in the
[mysqld] section of the server configuration file. After this setting is added and MySQL restarted, all new
InnoDB tables will use multiple tablespaces, and any InnoDB tables created previously will continue to use the
shared tablespace. If the innodb_file_per_table line is removed from the configuration file and
MySQL restarted, new tables are created in the shared tablespace, but access is still available for any tables
that were created using multiple tablespaces. InnoDB always needs the shared tablespace because it puts its
internal data dictionary and undo logs there. The .ibd files are not sufficient for InnoDB to operate.

10 - 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

MySQL
MySQL

Data
Data

mysql
mysql
InnoDB multiple tablespaces
test
test City.ibd
City.ibd

Oracle University and BOS-it GmbH & Co.KG use only

Country.ibd
Country.ibd
world
world
Countrylanguage.ibd
Countrylanguage.ibd
Internal Multiple
Multipletablespaces
Internaldata
data tablespaces Country.frm
dictionary Transaction requires
requiresthe
the
dictionary Transaction (Metadata)
innodb_file_per_table
--------------------

Insert records, innodb_file_per_table

Insertundo
--------------------

undo records,
redo clause Country.frm
logs
logs redolog
log clause
(Metadata)
Update records,
records,etc.
etc.
Update
undo
undologs
logs Countrylanguage.frm
ibdata files Ib_logfile files
(Metadata)

215 Transaction Capabilities

InnoDB provides atomicity, consistency, isolation and durability of multiple statements executed within a transaction. A
transaction is a logical unit of work consisting of a set of updating SQL statements (UPDATE, INSERT, DELETE) that
are grouped together so that they either all succeed in updating the database, or all are discarded. The InnoDB storage
engine is ”ACID” compliant, that is the transactions are atomic, consistent, isolated, durable.

Locking
The InnoDB storage engine supports row-level locking, providing the finest level of lock granularity, where only the row
that is read or updated is locked; and table-level locking, which by default is only used when there are changes to the
table structure itself (as is the case with ALTER TABLE).
216 ● Row-level locking - With row-level locking, InnoDB allows other concurrent transactions to access other rows
on the same page. This is in contrast with page level locking, where an entire page containing the row is locked,
thus forcing concurrent transactions to wait to access the same page until the lock is released.
NOTE: Next-Key Locking

In row level locking InnoDB uses next-key locking. This means that besides index records, InnoDB can also lock
the 'gap' before an index record to block insertions by other users immediately before the index record. A next-key
lock means a lock which locks an index record and the gap before it. A gap lock means a lock which only locks a
gap before some index record.
● Table-level locking - With table-level locking, InnoDB ensures the utmost integrity of the data it contains when
doing direct changes to the table structure itself. This feature is controlled by the --innodb-table-locks
server setting, which is ON by default.

10 - 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

217 Backup and Recovery

Outside of purchasing and using the “InnoDB Hot Backup®” tool, which allows for backing up the InnoDB database
while it is running, shutting down and performing a complete datadir backup is the safest and most complete
backup scenario. InnoDB supports this through the files mentioned earlier. By copying each of the files associated
with the storage engine (ibdata, ib_logfile's, *.ibd and *.frm), the user can make a complete binary backup of all
tables and indexes. However, this is only acceptable for making a complete recovery of all InnoDB data associated
with the server, not a partial recovery of specific data. Other issues to consider in relation to backup and recovery:
● Replication - Replication works with InnoDB tables, so MySQL can use replication capabilities to keep a
copy of the database at database sites requiring high availability.
● Logical backups - Logical backups are stored into text files that are human-readable, so spotting table

Oracle University and BOS-it GmbH & Co.KG use only

corruption becomes easier. Also, because the format is simpler, the chance for serious data corruption is
smaller. mysqldump also has a --single-transaction option that can be used to make a consistent
snapshot without locking out other clients.
To be able to recover an InnoDB database to the present from the binary backup just described, run the
MySQL server with binary logging turned on. Then apply the binary log to the backup database to achieve
point-in-time recovery.
Another option for making logical backups for InnoDB tables is using the SELECT * INTO
OUTFILE ... command.
218 Optimization
The key aspect of the InnoDB engine is a data caching and indexing structure where both indexes and data are cached
in memory as well as being stored on disk. This enables very fast operations, and works even on very large data sets.
InnoDB is used in production at numerous large database sites requiring such high performance and the remainder of
this chapter will be focused on passing on the knowledge that these companies have.

Special Features
As stated earlier, InnoDB provides MySQL with a transactional storage engine that has commit, rollback, and crash
recovery capabilities. In addition, InnoDB does locking on the row level and also provides a consistent non-locking
read in SELECT statements. These features increase multi-user concurrency and performance. InnoDB also supports
FOREIGN KEY constraints. InnoDB tables can be freely mixed with tables from other MySQL storage engines, even
within the same statement.

10 - 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

10.3 InnoDB Transactions

In the InnoDB transaction model, the goal of the developers has been to combine the best properties of a row-level multi-
versioning database with traditional two-phase locking. InnoDB does locking on row level and runs queries by default as
non-locking consistent reads, in the style of other database engines. The lock table in InnoDB is stored so space-
efficiently that lock escalation is not needed. Typically several users are allowed to lock every row in the database, or any
random subset of the rows, without InnoDB running out of memory.
219 AUTOCOMMIT (--autocommit)
In InnoDB all user activity happens inside transactions. If the autocommit mode is switched on with SET AUTOCOMMIT
= 1, then each SQL statement forms a single transaction. The user can still perform a multi-statement transaction by

Oracle University and BOS-it GmbH & Co.KG use only

starting it with START TRANSACTION or BEGIN and ending it with COMMIT or ROLLBACK.
If the autocommit mode is switched off with SET AUTOCOMMIT = 0, then MySQL sees a transaction as always open.
If an SQL COMMIT or ROLLBACK statement is issued, it ends the current transaction, and a new one starts. Both
statements will release all InnoDB locks that were set during the current transaction. A COMMIT means that the changes
made in the current transaction are made permanent and become visible to other users. A ROLLBACK statement, on the
other hand, cancels all modifications made by the current transaction.
220 TRANSACTION ISOLATION LEVEL (--transaction-isolation)
Transaction isolation levels specify what data is visible to statements within a transaction. These levels directly impact
the level of concurrent access by defining what interaction is possible between transactions against the same target data
source. InnoDB offers all four transaction isolation levels described by the SQL standard:
● READ UNCOMMITTED (weak protection) - SELECT statements are performed in a non-locking fashion, but a
possible earlier version of a record might be used. Thus, using this isolation level, such reads are not consistent.
The bottom line problem is that open transactions see changes made by other open transactions, and if the other
open transactions rollback, the transactions end up using data that was never committed and thus in essence
never really "existed". This is also called a “dirty read.”
● READ COMMITTED (better protection) - All SELECT ... FOR UPDATE and SELECT ... LOCK IN
SHARE MODE statements lock only the index records, not the gaps before them, and thus allow the free
insertion of new records next to locked records. UPDATE and DELETE statements using a unique index with a
unique search condition lock only the index record found, not the gap before it. With this level selected, the
changes made by all COMMITTED transactions are made as soon as they are committed.
● REPEATABLE READ (good protection) - This is the default isolation level of InnoDB. SELECT ... FOR
UPDATE, SELECT ... LOCK IN SHARE MODE, UPDATE, and DELETE statements that use a unique
index with a unique search condition lock only the index record found, not the gap before it. With other search
conditions, these operations employ next-key locking, locking the index range scanned with next-key or gap
locks, and block new insertions by other users. With this level selected, data reads always see tables in the same
consistent state even if there are transactions committed between the reads.
● SERIALIZABLE (zero anomalies) - This level is like REPEATABLE READ, but InnoDB implicitly converts
all plain SELECT statements to SELECT ... LOCK IN SHARE MODE. This level is rarely used (and
shouldn't be used unless absolutely necessary as performance is usually reduced). The point with the name is
that in this mode two interlacing transactions could be replayed serially and it would give the same result.

10 - 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

221 10.4 Internal InnoDB Buffers

To speed up SQL processing and take loads off the database server, InnoDB caches information in two major internal
buffers:
● Buffer pool - This buffer contains cached data and index pages (both leaf and no-leaf). The
innodb_buffer_pool_size server configuration variable controls the size of the memory buffer
InnoDB uses to cache data and indexes of its tables. The larger this value is set, the less disk I/O is needed to
access data in tables. On a dedicated database server, this could be set to utilize 80% of the machine physical
memory size. However, do not set it too large because competition for the physical memory might cause
paging in the operating system.

Oracle University and BOS-it GmbH & Co.KG use only

The In-Memory
buffer pool writes
and reads data
pages from the DB
server

Read
Read
Buffer Database
Disk Storage Pool Server
Write
Write

The In-Memory
buffer pool writes
and reads data
pages from O/S

222 ● Log buffer - This buffer contains cached log records. The innodb_log_buffer_size server
configuration variable controls the size of the buffer that InnoDB uses to write to the log files on disk.
Sensible values range from 1MB to 8MB. The default is 1MB. A large log buffer allows large transactions to
run without a need to write the log to disk before the transactions commit. Thus, with big transactions,
making the log buffer larger will save disk I/O.
Doublewrite Buffer (From the InnoDB online manual – section 11.4.3)
InnoDB uses a novel file flush technique involving a structure called the doublewrite buffer. It adds safety to
recovery following an operating system crash or a power outage, and improves performance on most varieties
of Unix by reducing the need for fsync() operations. Before writing pages to a data file, InnoDB first writes
them to a contiguous tablespace area called the doublewrite buffer. Only after the write and the flush to the
doublewrite buffer has completed does InnoDB write the pages to their proper positions in the data file. If the
operating system crashes in the middle of a page write, InnoDB can later find a good copy of the page from the
doublewrite buffer during recovery.

10 - 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Log
Buffer
Buffer
Writes to disk Pool
at COMMIT and
checkpoints

Oracle University and BOS-it GmbH & Co.KG use only

Writes to disk
at checkpoints
Internal data
Internal data
Transaction dictionary
Transaction dictionary
--------------------

records, Insert
Insertundo
--------------------

records, undo
redo
redolog
log logs
logs
records,
records,etc.
etc. Update
Update
undo
undologs
logs
Ib_logfile files ibdata files

As the figure demonstrates, InnoDB inserts transactional statements as log records into the log buffer, while
simultaneously executing the modifications those statements make against the in-memory copy of the record data
available in the buffer pool. This dual-buffer write explains the doublewrite buffer terminology.
223 The innodb_flush_log_at_trx_commit server variable tells the InnoDB storage when to execute a transaction
commit to flush log buffer to disk, thus making the modifications made by the transaction permanent and able to survive
a database crash. With many short transactions changing this variable can really improve performance. The values that
this variable can take are:
● 0 - The logs are flushed to disk approximately once per second
● 1 - The COMMIT statement initiates the flush (some disks have internal caches, etc., that could affect this).
● 2 - The log buffer is written to the file, but the file is not necessary flushed to disk, as the file system may have a
cache. An fsync() occurs every second.

10 - 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

224 InnoDB checkpoint

In memory On disk
The InnoDB checkpoint is the
point up to which all
modifications in the buffer pool

Log files
has been flushed to disk (log Log Transaction
Transaction
buffer also flushed). Checkpoints COMMIT records,
Buffer (+ checkpoint)
records,
redo
influence crash recovery time, not redolog
log
records,
records,etc.
etc.
the ACID properties of the storage
engine. Checkpoints are done
Ib_logfile files
when the database is idle, or the
Redo Log is filling up (and new

Oracle University and BOS-it GmbH & Co.KG use only

Table space
space for log records is needed). Undo Log
Checkpoint timing is not
controlled by the programmer, but Buffer Internal data
Internal data
dictionary
dictionary
can be read from the InnoDB Pool checkpoint --------------------

Insert
Insertundo
--------------------

undo
Monitor. Checkpoints are written logs
logs
onto the Redo Log header and are Update
Update
undo
undologs
used by InnoDB to know at logs
recovery time what to recover.
Additional ibdata files
Memory Pool
225 Recovery Management
Subsystem (RMS)
The recovery management subsystem inside InnoDB flushes database pages and transaction operations to the log files
for the purpose of backup and recovery. The RMS implements a fuzzy checkpoint operation by continually flushing
modified database pages from the buffer pool in small batches. InnoDB writes to the log files in a circular fashion, so
if the log file has reached the configured limit set by the innodb_log_file_size system variable, a checkpoint
operation is executed at once to flush the modified database pages. This is done in order to make sure the committed
pages are available in case of recovery.
226 Log File Size
The size of each log file should be chosen to avoid executing checkpoint operations too often. The bigger log file size
reduces disk I/O in checkpointing. However, the larger size of the log file increases the “redo” recovery time in case
of a server crash.
The innodb_log_file_size server variable defines the size of each log file in a log group in megabytes.
Sensible values range from 1M to 1/n-th of the size of the InnoDB_log_buffer_size, where n is the number of log files
in the group. The larger the value, the less checkpoint flush activity is needed in the buffer pool, saving disk I/O. But
larger log files also mean that recovery will be slower in case of a crash. The combined size of log files must be less
than 4 GB on 32-bit computers. The default is 5M.

10 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

InLine Lab 10-A

In this exercise you will test the InnoDB storage engine to determine how often a commit occurs affects performance.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered the
contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.
Step 2. Test InnoDB INSERT bulk performance with varying COMMIT intervals

Oracle University and BOS-it GmbH & Co.KG use only

Action: Execute the following stored procedure from the mysql client:

CALL innodb_inserts(100000,1);

Effect: This stored procedure will create a table called innodb_test and load it with 100,000 random records,
committing changes after each record. (Note that slow disk speed may cause this procedure to be very slow!)
How long did it take to perform the insert of the 10,000 records with a commit after each record?

__________________________________________________________________________________________
Action: Run the innodb_inserts stored procedure again, but this time commit every 5 records:

CALL innodb_inserts(100000,5);

How long did it take to perform the insert of the 100,000 records with a commit performed every 5 records?

__________________________________________________________________________________________
Action: Run the innodb_inserts stored procedure again, but this time commit every 50 records:

CALL innodb_inserts(100000,50);

How long did it take to perform the insert of the 100,000 records with a commit performed every 50 records?

__________________________________________________________________________________________
Action: Run the innodb_inserts stored procedure again, but this time commit every 500 records:

CALL innodb_inserts(100000,500);

How long did it take to perform the insert of the 100,000 records with a commit performed every 500 records?

__________________________________________________________________________________________

10 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Action: Run the innodb_inserts stored procedure again, but this time commit every 5000 records:

CALL innodb_inserts(100000,5000);

How long did it take to perform the insert of the 100,000 records with a commit performed every 5000 records?

__________________________________________________________________________________________
Action: Run the innodb_inserts stored procedure again, but this time commit every 10000 records:

CALL innodb_inserts(100000,10000);

Oracle University and BOS-it GmbH & Co.KG use only

How long did it take to perform the insert of the 100,000 records with a commit performed every 10000
records?

__________________________________________________________________________________________
Action: Run the innodb_inserts stored procedure again, but this time commit every 100000 records:

CALL innodb_inserts(100000,100000);

How long did it take to perform the insert of the 100,000 records with a single commit performed afterward?

__________________________________________________________________________________________
Step 3. Review the results

Of all the different commits, which performed the fastest and why

__________________________________________________________________________________________

10 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

227 10.5 InnoDB Indexes

Within the tablespace two segments (internal files) will be created and maintained for each InnoDB table. One
segment is the clustered index data pages, where rows are sorted according to the primary key. The leaves of the index
are the actual rows.
The second segment is used for any secondary indexes built on the clustered key. This assists performance when
adding data sequentially in large blocks to both data and secondary index pages. Each secondary index has its own
(pair of) segment(s). The secondary indexes do not point to physical addresses but contain the primary keys instead.
This requires another index retrieval to find the pages.
Clustered index
Primary Key Index Secondary index
index pages

Oracle University and BOS-it GmbH & Co.KG use only

Non-leaf
index pages

PKV

PKV

PKV

PKV

PKV
Leaf

Row

Row

Row

Row
Row

Primary Key Values

NOTE: Physical Structure of an InnoDB Index

All InnoDB indexes are B+trees where the index records are stored in the leaf pages of the tree. The default size of
an index page is 16KB. When new records are inserted, InnoDB tries to leave 1/16 of the page free for future
insertions and updates of the index records. If index records are inserted in a sequential order (ascending or
descending), the resulting index pages are about 15/16 full. If records are inserted in a random order, the pages are
from 1/2 to 15/16 full. If the fill factor of an index page drops below 1/2, InnoDB tries to contract the index tree to
free the page.

10 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

228 10.6 Crash Recovery

In order to satisfy the durability requirement of the "ACID" test, InnoDB logs a combination of physical (disk
address) and logical (field content) data. The logging is used at crash recovery. One transaction can have any number
of redo log entries for the statements that make up that transaction. At crash time the redo log is designed to reissue
changes not yet flushed to the table space. One redo log is kept for the entire server. At the same time an undo log is
kept for removing uncommitted changes already flushed to the table space at crash time. There are two undo logs for
each transaction (update/delete + insert).

The History List: Committed Undo Logs

Undo Log Undo Log Undo Log Undo Log

Oracle University and BOS-it GmbH & Co.KG use only

PURGE
The header of the transaction system

Free unnecessary
F log entries
an lus C
d h O
m lo M Rollback segment
ov g M
IT used for:
e bu
1000 slots per page

to ffe Crash recovery

hi r t Read isolation
st o
or d
y isk
lis
t

A transaction
Undo Log: Updates, Deletes
Two undo logs per trx

Undo Log: Inserts COMMIT Discarded

When the system recovers from a crash, the InnoDB logs will be applied to any rows that were being updated at the
time of the crash, but had not been flushed to disk. First, the system applies the redo, basically re-applying the
changes it has recorded in a serial manner to the tables and indexes in question. And, since the commits are also
logged, the system will know what undo's to apply to bring the database back to a consistent state. That is the state
where all committed data is consistent in the database.

10 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

229 Redo Log

Every change that is made affecting InnoDB tables is put into the redo log. Each redo log line consists of the following
structure:

PageNo Offset Record Type Changes on that page

4 bytes 2 bytes 1 byte

● PageNo - The page number makes up 4 bytes of the overall line. It represents the page within the tablespace
where the change was made.

Oracle University and BOS-it GmbH & Co.KG use only

● Offset – This structure component contains the location of the change within the page, and takes up 2 bytes.
● Record Type - The log record type is 1 byte and determines what took place on the page - Insert, Update, Delete
or other types (like ”fill space with blanks”).
● Changes on that page – The changes (only redo values, no old values) are recorded so that the particular record
type can be reapplied, except for deletions, which need no change data at all. (The delete itself is recorded as the
Record Type).
230 Undo Log
If a row in an InnoDB table was updated, then the undo log record contains the information necessary to rebuild the
content of the row before it was updated. Each undo log consists of the following structure:

Primary Key Value Old trx id Old values on that row

● Primary Key Value - The undo log records begin with the primary key value.
NOTE: InnoDB and Primary Keys

If an InnoDB table is created without a PRIMARY KEY, MySQL picks the first UNIQUE index that has only NOT
NULL columns as the primary key and uses it as the clustered index (primary key value). If there is no such index
in the table, InnoDB internally generates a clustered index where the rows are ordered by the row ID that InnoDB
assigns to the rows in such a table. The row ID is a 6-byte field that increases monotonically as new rows are
inserted. Thus the rows ordered by the row ID will be physically in the insertion order.
● Old trx id - The next section of the record is the old transaction ID, the ID of the transaction that changed that
row.
● Old values on that row – This contains the old values that were replaced during the transaction.

10 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

231 Multiversioning
Multiversion Concurrency Control (MVCC) is an advanced technique for improving database performance in a
multiuser environment. The InnoDB tablespace contains many versions of the same rows in order to maintain
isolation between transactions. Different transactions see different versions of the same rows. To manage this, the
InnoDB storage engine maintains the following ID’s:
● Create Version ID – The data records are tied to a version, this identification is used to tell InnoDB when
the data record was created.
● Delete Version ID – When a record was deleted (or expired), this identification is maintained to track the
deletion of the data record.

Oracle University and BOS-it GmbH & Co.KG use only

● System Version ID – This identifies a value that is associated with a serialized execution time, which the
server increments as transactions are executed against it.
232 Only one ID will be returned or used by InnoDB during a transaction execution. The ID type used depends on the
transaction type and the details of that transaction:
● SELECT statements – When MySQL receives a SELECT statement it first determines if there are criteria in
the SELECT statement that match any records that have a version ID. The return values are based on the
following facts:
○ The create version ID must be less than or equal to the system version ID. This criterion ensures that the
data record was created before the currently executing transaction began.
○ If there is a delete version ID, then it must be greater than the system version ID. This makes it
impossible to return records that were deleted before the currently executing transaction began.
○ The create version ID cannot be in the list of actively running transactions. This ensures that the row
data being returned has not been created or modified by any uncommitted transactions.
● INSERT statements – When MySQL receives an INSERT operation, a new record is added to the table and
the system version ID is used as the create version ID.
● DELETE statements – When MySQL receives a DELETE operation, instead of removing the record, the
delete version ID is set to the system version ID.
● UPDATE statements – On an UPDATE, InnoDB adds a new row record to the table rather than modifying
the existing data row. The create version ID for this new record is the system version ID. For the old row
data record, InnoDB places a delete version ID (based on the system version ID) on the old record, thus
maintaining concurrency control.
NOTE: InnoDB Holes

Due to MVCC ‘s handling of UPDATE statements, the physical table spaces can be riddled with holes. These
holes can cause the physical side of the ibdata files to become quite large with tables that have lots of
updates. This should be considered when discussing multiple tablespaces in the application architecture.

10 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

233 10.7 InnoDB Best Practices

Now with an understanding of the inner workings of the InnoDB storage engine, the discussion on best practices can be
connected to what has been taught. The following best practices can be used to improve the performance and resources
associated with the InnoDB storage engine:
● Use short, integer primary keys - Long Primary keys waste space in InnoDB tables. The Primary Key is also
included in any secondary indexes. Consider producing a synthetic short, integer primary key by adding a
PRIMARY KEY AUTO_INCREMENT column and changing the current PRIMARY KEY to UNIQUE.
● Load/Insert data in primary key order – If necessary, perform an external sort on the data being inserted to
ensure that the records will be entered in sequential order.

Oracle University and BOS-it GmbH & Co.KG use only

● Increase log file size - When InnoDB has written the log files full, it has to write the modified contents of the
buffer pool to disk in a checkpoint. Small log files cause many unnecessary disk writes. The drawback of big log
files is that the recovery time is longer.
● Avoid large rollbacks - InnoDB uses the log buffer to save disk I/O in inserts, but no such mechanism is used in
a corresponding rollback. A disk-bound rollback can take 30 times as long to perform as the corresponding
insert. Killing the database process does not help because the rollback starts again on server startup. The only
way to get rid of a runaway rollback is to increase the buffer pool so that the rollback becomes CPU-bound and
runs fast.
● Avoid mass inserts – Perform large loads in chunks, as rollback of failed LOAD DATA INFILE can take long
periods of time, even days. In addition, to speed up data loads consider setting UNIQUE_CHECKS=0 and
FOREIGN_KEY_CHECKS=0 for data loads. The risk may be outweighed by the alternative resource drain on
the server.
● Utilize prefix keys - This can be extremely effective as there is no key compression in InnoDB tables.

10 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

234 10.8 InnoDB Tuning

To review how the InnoDB storage engine is running in a specific environment, utilize the SHOW ENGINE INNODB
STATUS command. This command provides a number of InnoDB-specific statistics. The output of SHOW ENGINE
INNODB STATUS is different from that of SHOW STATUS in that it reads more as a textual report, with section
headings and such. There are different sections of the report that provide information on transaction statistics, buffer
information, transaction logs, and so forth.
The following settings can also be managed for the purpose of improving performance and better tuning InnoDB
tables for the needs of the end users and/or database applications:
● innodb_log_files_in_group – This server variable defines the number of log files that InnoDB

Oracle University and BOS-it GmbH & Co.KG use only

should use. The default is 2 and should not be changed.
● innodb_additional_mem_pool_size – This server variable defines the size of the dictionary cache.
Set to 8-16M; increase if SHOW ENGINE INNODB STATUS reports spills.
● innodb_autoextend_increment – This server variable defines the increment size (in MB) for
extending the size of an auto-extending tablespace when it becomes full. The default value is 8.
● innodb_thread_concurrency - InnoDB tries to keep the number of operating system threads
concurrently inside InnoDB less than or equal to the limit given by this variable. If performance issues arise,
and SHOW ENGINE INNODB STATUS reveals many threads waiting for semaphores, thread “thrashing”
is likely, and setting this variable lower or higher should be tried. If the server is running on a computer with
many processors and disks, try setting the value higher to make better use of the computer's resources. A
recommended value is the sum of the number of processors and disks the system has. The default value is 8,
and concurrency checking will be disabled if the setting is greater than or equal to 20 (considered infinite
concurrency).

10 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

InLine Lab 10-B

In this exercise you will alter InnoDB system variables based on feedback from the SHOW ENGINE INNODB STATUS
report. Note: All tables have to be InnoDB for this lab to run properly.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered the
contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.

Oracle University and BOS-it GmbH & Co.KG use only

Action: In the mysql client, create the city_huge table with approximately 400,000 records by executing the
following stored procedure:

CALL create_city_huge(100);

Effect: This stored procedure will create a table called city_huge that is based on the City table. Once the
city_huge table is created, the stored procedure will load in the records from the City table 100 times, creating
approximately 400,000 records in the new table.
Action: Change the storage engine being used for the city_huge table to InnoDB by executing the following
SQL statement:

ALTER TABLE city_huge ENGINE=InnoDB;

This process will take some time, thus it may be a good time to take a break.
Step 2. Put InnoDB under some traffic load

Action: Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the city_huge table:

mysqlslap -uroot -q /labs/scripts/innodb_query.sql

--create-schema=world -i 1 -c 4

Effect: This will execute a pre-defined SQL file that will run multiple queries against the city_huge table to give
us a good number of InnoDB statistics to look at. (Note: The innodb_query.sql file is identified here as being
in the home directory of the current user, please alter that location to where this file is located on your system.)

10 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Step 3. Collect InnoDB status output for review

Action: In the mysql client, enter the following command to display the InnoDB performance counters,
statistics and information about transaction processing along with a large number of other statistics to help in the
tuning process:

SHOW ENGINE INNODB STATUS\G

Effect: The InnoDB status report will display a large amount of information and takes some time to decipher.
(Note: With the way that MySQL displays information, you may have to scroll through the window to get to the
information that is required.)

Oracle University and BOS-it GmbH & Co.KG use only

Step 4. Review InnoDB status sample size

Action: The first thing to check for in the InnoDB status report is to make sure that you have a large enough
sampling to get proper statistics. This on the first line under the title INNODB MONITOR OUTPUT. The
following is an example of this line:

Per second averages calculated from the last 56 seconds

Due to the fact that many of the values are based on per second, it is important to have a sampling of data of at
least 20-30 seconds. Anything less is unusable and should be discarded.
Step 5. Review SEMAPHORES

Action: Under the SEMAPHORES section of the InnoDB status report, there are two sub-sections. The first is the
current waits (which will only be displayed if running in a high concurrency environment and InnoDB has to
rely on O/S waits) and the second is event counters. For our purposes, the event counters will provide the
information we are looking for. The following is an example of what this section looks like:

----------
SEMAPHORES
----------
OS WAIT ARRAY INTO: reservation count 2685, signal count 2087
Mutex spin waits 0, rounds 171700, OS waits 1636
RW-shared spins 960, OS waits 319;
RW-excl spins 1832, OS waits 730

Reservation count and signal count displays the how often InnoDB uses the internal sync array, and thus the
ratios between them represent how frequently InnoDB is falling back on the OS wait functions.
The OS waits located under exclusive locks and shared locks are not related to the wait array info, as those refer
to the number of times InnoDB relayed on the OS to handle the name thread hoping that the object was freed
already. These OS waits can be very slow; thousands of them per second is a problem and identifies an area
needing attention.
Spin waits and spin rounds identify a problem if both are high and show that significant CPU resources are
being wasted. However, even then there is only a concern when it is hundreds of thousands spin rounds per
second. The system variable innodb_sync_spin_loops can be used to help correct this problem.

10 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Step 6. Review TRANSACTIONS

Action: Under the TRANSACTIONS section of the InnoDB status report, review the content associated with the
transactions that occurred during the sampling period. This section can be very helpful in determining if your
application has lock contention and the reasons for transaction deadlocks. The following is an example of what this
section looks like:

------------
TRANSACTIONS
------------
Trx id counter 0 264408

Oracle University and BOS-it GmbH & Co.KG use only

Purge done for trx's n:0 < 0 264408 undo n:0 < 0 0
History list length 6
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0 252406, not started, process
no 4330, OS thread id 303062926 MySQL thread
id 19330, query id 3328299 localhost root
SHOW INNODB STATUS
...

The Trx id counter, which is incremented for each transaction, is the current transaction identifier.
Purge done for trx's n:o is the number of transactions to which a purge is completed. Uncommitted
transactions (that may be old and stale) may eat up resources by blocking the purge process. Reviewing the
transaction counter difference between the current and last purged transaction will help identify if this is occurring.
The system variable innodb_max_purge_lag can be used to help correct this problem.
History list length 6 is the number of unpurged transactions in undo space. This value is increased as
transactions which have done updates are committed and decreased as purge runs.
In the LIST OF TRANSACTIONS ... sub-section, the transactions that have occurred during the sampling will be
displayed. If the number of connections made during the sampling was small, all the connections will be printed in
the transaction list. However, if there were a large number of connections only a few will be printed. This ensures
that the report output will not grow too large.

10 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Step 7. Review FILE/IO

Under the FILE I/O section of the InnoDB status report, the state of the file input/output helper threads are
displayed. These helper threads are responsible for insert buffer merges (insert buffer thread), asynchronous log
flushes (log thread), read-ahead (read thread) and flushing of dirty buffers (write thread). This section can be
very helpful in determining if your application has lock contention and the reasons for transaction deadlocks.
The following is an example of what this section looks like:

--------
FILE I/O
--------

Oracle University and BOS-it GmbH & Co.KG use only

I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (write thread)
Pending normal aio reads: 0, aio writes: 0, ibuf aio reads: 0, log i/o's:
0,
sync i/o's: 0
Pending flushes (fsync) log: 0, buffer pool: 0
127065 OS file reads, 254400 OS file writes, 133225 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s

For each helper thread you can see thread state (if thread is ready: waiting for i/o request) or if it is
executing certain operation.
The number of pending operations is shown for each of the helper threads (these are the amount of operations
queued for execution or being executed at the same time). This includes the number of pending fsync
operations. Calling fsync() on modified files is used by InnoDB to ensure data makes it to the disk (simply
passing it to the OS cache is not enough). If any of these values are constantly high, an I/O bound workload is
possible. However, not all I/O threads are accounted for here (eg. I/O requests submitted by threads executing
requests), so there may still be an I/O bound workload while the numbers are displaying zeroes.
The number of file I/O operations are shown as computed averages. These numbers can be very helpful in
monitoring and graphing. The bytes/read shows the average size of read requests. It is best to consider these
values as read-ahead efficiency due to what they are tracking. For random I/O these will most likely display 16K
(a page size), for full table scan or index scan read-ahead may be performed which can increase average read
size significantly.

10 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Step 8. Review INSERT BUFFER AND ADAPTIVE HASH INDEX

Under the INSERT BUFFER AND ADAPTIVE HASH INDEX section of the InnoDB status report, the status of
the insert buffer and adaptive hash are displayed. An adaptive hash is a hash index Innodb builds for some pages to
speed up row lookup by replacing btree searches with hash searches. The following is an example of what this
section looks like:

-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
ibuf: size 1, free list len 405, seg size

Oracle University and BOS-it GmbH & Co.KG use only

407, 1736598 inserts, 1736598 merged
recs, 64254 merges
Hash table size 34679, used cells 625, node heap has 2 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s

For the most part, there is very little that can be done to improve the performance of this section, so this section can
be considered informational only.
The first line shows status of the insert buffer (ibuf) to include the segment size and the free list along with the any
records that are located in the insert buffer. This is followed by how many inserts were done in the insert buffer, how
many records were merged and how many merges took place. Calculating the ratio of merges to the number of
inserts provides the insert buffer efficiency.
The last section displays the hash table size, number of used cells and number of buffers used by adaptive hash
index. The number of hash index lookups and number of non-hash index lookups indicate the hash index efficiency.
Step 9. Review LOG

Under the LOG section of the InnoDB status report, information associated with the log subsystem is displayed. The
following is an example of what this section looks like:

---
LOG
---
Log sequence number 2 2284053902
Log flushed up to 2 2284053902
Last checkpoint at 2 2284053902
0 pending log writes, 0 pending chkp writes
129214 log i/o's done, 0.00 log i/o's/second

This section displays the current log sequence number (which is the amount of bytes InnoDB has written in log files
since system tablespace creation). This section also identifies up to which point logs have been flushed along with
how much data is not flushed in the log buffer. In addition, when the last checkpoint was performed is displayed.
Monitoring this information provides the information need to adjust the innodb_log_buffer_size. If there is
a more than 30% of the log buffer size being flushed, it can be an indication that this system variable needs to be
increased.

10 - 23
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

The last two lines display the number of pending normal log writes and number of checkpoint log writes and the
number of log i/o operations, which allows for the separation of the tablespace related I/O from log related I/O
so that you can see how much I/O your log file requires.
Log writes may be more or less expensive based on the value of the
innodb_flush_log_at_trx_commit system variable. When this variable is set to 2, log writes are sent
to the O/S cache. With these writes being sequential the log writes are considered very fast.
Step 10. Review BUFFER POOL AND MEMORY

Under the BUFFER POOL AND MEMORY section of the InnoDB status report, buffer pool activity and memory
usage is displayed. This report provides the information needed to determine if the buffer pool is sized properly.
The following is an example of what this section looks like:

Oracle University and BOS-it GmbH & Co.KG use only

----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 18431950; in additional pool allocated 908416
Dictionary memory allocated 27456
Buffer pool size 512
Free buffers 0
Database pages 510
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages read 138501, created 11706, written 188858
2.50 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 961 / 1000

In this section, the display shows total memory allocated by InnoDB, amount of memory allocated in additional
memory pool, total number of pages in buffer pool, number of pages free, pages allocated by database pages and
dirty pages.
In relation to sizing the buffer pool, if there is constantly a lot of pages free, it probably means the active
database size is smaller than allocated buffer pool size and can be tuned down.
The pending reads and writes lines identify pending requests on buffer pool level. InnoDB may merge multiple
requests to one on file level so these can be different. In addition, there are different types of I/O submissions by
InnoDB displayed: least recently used (LRU) which identifies dirty pages that have not been accessed in an
excessive amount of time, old pages that need to be flushed by the check-pointing process and independent
single page writes.
The third to the last line displays the number of pages being read and written. Any created pages is empty pages
created in the buffer pool for new data (showing previous page content was not read to the buffer pool).
On the last line, the buffer pool hit ratio is displayed which measures buffer pool efficiency. 1000/1000
corresponds to 100% hit rate. Due to the fact that buffer pool hit rate is workload dependent it is hard to
determine what is poor performance. Sometimes 950/1000 will be enough, sometimes you can see I/O bound
workload with hit rate of 995/1000.

10 - 24
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Step 11. Review ROW OPERATIONS

Under the ROW OPERATIONS section of the InnoDB status report, activity on the row basics along with some
addition system information is displayed. The following is an example of what this section looks like:

--------------
ROW OPERATIONS
--------------
2 queries inside InnoDB, 0 queries in queue
2 read views open inside InnoDB
Main thread process no. 4330, id 2996149136, state: sleeping

Oracle University and BOS-it GmbH & Co.KG use only

Number of rows inserted 1517817, updated 6077, deleted 0, read 20254
0.60 inserts/s, 1.86 updates/s, 0.00 deletes/s, 6.26 reads/s

This section displays the InnoDB thread queue status by answering the questions: "how many threads are waiting
and how many are active?"
The second line answers the question: "how many read views are open inside InnoDB?", which defines transactions
that were started but no statement is currently active.
The third line displays the state of the InnoDB main thread which is responsible for scheduling the number of
system operations to include flushing dirty pages, check-pointing, purging, flushing logs and insert buffer merge.
The fourth line displays the number of row operations since the system has started along with average values
providing valuable data for monitoring and graphing the InnoDB load.
Step 12. Tune InnoDB performance

Action: Now with the InnoDB status report being explained, it is time to make some changes to some of the
InnoDB system variables and view how it affects the performance. The first system variable that will be changed is
innodb_buffer_pool_size. The value assigned to this system variable should be approximately 80% of
system memory. This system variable can not be set dynamically, so the server will need to be stopped and restarted
with the following command line option:

--innodb_buffer_pool_size=430000000

Alternatively, place the option in the my.cnf file under the [mysqld] section:

[mysqld]
innodb_buffer_pool_size=430000000

The value of 430000000 should be altered to around 80% of the system memory available to mysqld. This value,
80%, is appropriate for a dedicated database server; however, setting it too high can create competition for physical
memory and cause paging in the O/S.

10 - 25
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Action: Issue the following command twice (the first is a buffer warm up) in an O/S terminal window to run the
mysqlslap benchmarking tool against the city_huge table again:

mysqlslap -uroot -q /labs/scripts/innodb_query.sql

--create-schema=world -i 1 -c 4

This will execute a pre-defined SQL file that will run multiple queries against the city_huge table again. Did the
performance of the operation improve or worsen after adjusting the InnoDB buffer pool size?

______________________
Action: In the mysql client, change the setting of the innodb_flush_log_at_trx_commit system

Oracle University and BOS-it GmbH & Co.KG use only

variable to 0 (should currently be set to 1) using the following SQL statement:

SET GLOBAL innodb_flush_log_at_trx_commit=0;

When this system variable is changed to 0, it is possible to lose one second worth of transactions during a crash.
However, the trade of is that performance generally improves with a setting other than 1.
Action: Issue the following command twice (the first is a buffer warm up) in an O/S terminal window to run the
mysqlslap benchmarking tool against the city_huge table again:

mysqlslap -uroot -q /labs/scripts/innodb_query.sql

--create-schema=world -i 1 -c 4

This will execute a pre-defined SQL file that will run multiple queries against the city_huge table again. Did the
performance of the query improve or worsen after adjusting the InnoDB log flush behavior?

______________________
Step 13. Clean up

Action: Return the system variables changed during this exercise to their original values. This will require a
restart of the MySQL server with the innodb_buffer_pool_size or
innodb_flush_log_at_trx_commit system variables removed from the command line and my.cnf.
Effect: The MySQL server is now back to its original configuration with the altered settings reverted to their
default values.

10 - 26
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

235 10.9 Comparing InnoDB and MyISAM

MyISAM and InnoDB are the storage engines that are utilized in the majority of MySQL architectures. The following are
the differences between them:
● Crash recovery - InnoDB recovers from a crash or other unexpected shutdown by replaying its logs. MyISAM
must fully scan and repair or rebuild any indexes or possibly tables which had been updated but not fully flushed
to disk. Since the InnoDB approach is approximately fixed time while the MyISAM time grows with the size of
the data files, InnoDB offers greater perceived availability and reliability as database sizes grow. InnoDB’s
crash-recovery guarantees that the tables are in a consistent state after a crash but does not guarantee that all the
data will be in a consistent state.
Caching - MyISAM relies on the operating system for caching reads and writes to the data rows and indexes

Oracle University and BOS-it GmbH & Co.KG use only

●
while InnoDB does this within the engine itself, combining the row caches with the index caches. Dirty
(changed) database pages are not immediately sent to the operating system to be written by InnoDB, which can
make it substantially faster than MyISAM in some situations.
236 ● Physical storage - InnoDB stores data rows physically in primary key order while MyISAM typically stores
them mostly in the order in which they are added. When the primary key is selected to match the needs of
common queries this can give a substantial performance benefit.
● Compression - MyISAM provides compression and terse row formats. InnoDB tables utilize more space
because of transactional and multi-version concurrency control support. A lower overhead format is available,
reducing InnoDB overhead by about 20%.
● Transactions overhead - With all the benefits of InnoDB (Referential Integrity, Executing Transactional
Statements, Row Level Locks) there comes added overhead that affects performance. MyISAM, without these
specific benefits, performs generally faster.

10 - 27
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Quiz
In this exercise you will answer the following questions pertaining to the InnoDB storage engine.
10. What are the possible disk footprints for InnoDB tables?
__________________________________________________________________________________
__________________________________________________________________________________
11. What types of locking does InnoDB support?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

12. In which isolation levels can InnoDB use next-key locking?
__________________________________________________________________________________
__________________________________________________________________________________
13. What is cached in the InnoDB buffer pool?
__________________________________________________________________________________
__________________________________________________________________________________
14. Why should log_buffer be large?
__________________________________________________________________________________
__________________________________________________________________________________
15. What are the different modes for innodb_flush_log_at_trx_commit?
__________________________________________________________________________________
__________________________________________________________________________________
16. How does InnoDB handle the isolation level REPEATABLE-READ?
__________________________________________________________________________________
__________________________________________________________________________________
17. Why doesn’t InnoDB have the same kind of concurrent_inserts option as MyISAM?
__________________________________________________________________________________
__________________________________________________________________________________
18. How are UPDATE statements handled internally in InnoDB?
__________________________________________________________________________________
__________________________________________________________________________________

10 - 28
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

19. Why is the size of primary keys particularly important in InnoDB?

__________________________________________________________________________________
__________________________________________________________________________________
20. What is the purpose of the InnoDB double-write buffer?
__________________________________________________________________________________
__________________________________________________________________________________

Oracle University and BOS-it GmbH & Co.KG use only

10 - 29
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

Further Practice
In this further practice you will tune InnoDB settings for improved performance during an import operation.
1. In the mysql client, create a smaller city_huge database to complete the steps to follow by executing the
following SQL statement:

CALL create_city_huge(25)

ALTER TABLE city_huge ENGINE=INNODB;

2. In the mysql client, export the data from the city_huge table randomly to a flat file using the following

Oracle University and BOS-it GmbH & Co.KG use only

SQL statement:

SELECT * INTO OUTFILE '/tmp/innodb_rand.txt' FROM city_huge ORDER BY

237 RAND();

3. Export the data from the city_huge table ordered by the primary key to a flat file using the following
SQL statement:

SELECT * INTO OUTFILE '/tmp/innodb_pk_order.txt' FROM city_huge ORDER

BY id;

4. Truncate the city_huge table and import the random text file (created in step 2) back into the table.
Record how long it took approximately to load the records back in: _________________
5. Truncate the city_huge table and import the primary key text file (created in step 3) back into the table.
Record how long it took approximately to load the records back in: _________________
6. Shut down the mysql client.
7. Edit the /etc/my.cnf file and add the following line:

innodb_buffer_pool_size = 1652704000;

This will increase the size of the innodb_buffer_pool_size system variable to approximately 80%
of the system memory (Using 2GB as an example). Close and save the file.
8. Restart the MySQL server.
9. Restart the mysql client.
10. Disable (turn off) unique check in mysql by entering the following SQL statement:

SET UNIQUE_CHECKS=0;

10 - 30
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

11. Truncate the city_huge table and import the random text file (created in step 2) back into the table.
Record how long it took approximately to load the records back in this time: _________________
12. Truncate the city_huge table and import the primary key text file (created in step 3) back into the table.
Record how long it took approximately to load the records back in this time: _______________

Oracle University and BOS-it GmbH & Co.KG use only

10 - 31
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 10 - InnoDB Storage Engine

10.10 Chapter Summary

238 In this lesson, you have learned to:
● List the characteristics of the InnoDB storage engine
The InnoDB storage engine provides support for all of the database functionality associated with the
MyISAM engine along with full transaction ACID compliant capabilities.
● Describe InnoDB transactions
In the InnoDB transaction model, the goal of the developers has been to combine the best properties of a
row-level multi-versioning database with traditional two-phase locking.

Oracle University and BOS-it GmbH & Co.KG use only

● Explain the internal InnoDB buffers
To speed up SQL processing and take loads off the database server, InnoDB caches information in two major
internal buffers: buffer pool and log buffer.
● Describe how InnoDB handles crash recovery
In order to satisfy the durability requirement of "ACID" test, InnoDB logs a combination of physical (disk
address) and logical (field content) data.
● List InnoDB best practices
The following best practices can be used to improve the performance and resources associated with the
InnoDB storage engine: Use short integer primary keys; load/insert data in primary key order; increase log
file size; avoid large rollbacks; avoid mass inserts; and utilize prefix keys.
● Describe other InnoDB optimization tools
To review how the InnoDB storage engine is running in a specific environment, utilize the SHOW ENGINE
INNODB STATUS command.
● Compare InnoDB and MyISAM storage engines
MyISAM and InnoDB are the storage engines that are utilized in the majority of MySQL architectures.

10 - 32
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

239
11 ALTERNATE STORAGE ENGINES

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

240 11.1 Learning Objectives

This chapter explains the details of the alternate (non mainstream) storage engines and how they relate to performance
tuning. In this lesson, you will learn to:
● List the characteristics of the MEMORY storage engine
● List the characteristics of the FEDERATED storage engine
● List the characteristics of the ARCHIVE storage engine
● List the characteristics of the CSV storage engine
● List the characteristics of the BLACKHOLE storage engine

Oracle University and BOS-it GmbH & Co.KG use only

● Explain how using multiple storage engines can be used to improve performance
● Explain the concept of custom storage engines
● Describe the fundamentals of MySQL cluster

11 - 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

241 11.2 MEMORY Storage Engine

The MEMORY storage engine (previously known as the HEAP storage engine) stores all data in memory; once the
MySQL server has been shut down any information stored in a MEMORY table will have been lost. However, the format
of individual tables is kept and this enables the creation of temporary tables that can be used to store information for
quick access without having to recreate the tables each time the database server is started.

Advantages
● Fast – Reading or writing data from MEMORY tables is very fast, due to everything being in memory.
○ Non-transactional and stores all data in memory.

Oracle University and BOS-it GmbH & Co.KG use only

○ Creates no data or index files on disk.
● Index options – The MEMORY storage engine supports both hash and balanced binary tree indexes. When
using indexing, the MEMORY storage engine allows the use of non-unique keys (which is an unusual feature for
implementations of hash indexes).
○ Hash indexes (HASH)
■ MEMORY tables use hash indexes by default.
■ Very fast lookups for all operations that use a unique index.
■ Usable only for equality comparisons (= or <=>).
Caution is required when there is a high degree of key duplication (many index entries containing the same
value) with hash indexing due to the proportional degree of slowdown in updates to the tables. BTREE
indexes should be used in such situations to eliminate this issue.
○ Red-black binary trees (BTREE)
■ Takes much more space in memory.
■ Supports range searches (<, >, <= or >=) as well as equality searches.
■ Faster for indexes with lots of duplicates.
242 Disadvantages
● Memory drain – MEMORY tables are designed to use available memory which can affect other processes that
utilize memory. It is best not to create too many MEMORY tables or tables that can become very large in size.
Memory can be freed up from MEMORY tables by dropping them or truncating them.
● Data in memory only - All data is lost if the server crashes or is shutdown due to the data existing in memory
only, however the table definitions will remain.
● Limitations on data types – MEMORY tables can not store BLOB or TEXT type fields.
● Fixed length rows - All MEMORY rows are fixed length. For example, a VARCHAR(200) will act like
CHAR(200) fields. This may require more memory for the data than expected.

11 - 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

243 11.2.1 MEMORY Best Practices

The MEMORY storage engine is a valid and useful storage engine and should be considered with almost any
application design to improve performance and to meet the needs of specific business rules. The following list of
suggestions are seen as best practices with the MEMORY storage engine:
● Minimize the size that a MEMORY table can grow to - Many applications use the MEMORY storage
engine to load content dynamically, and without proper guidelines can grow to a size that results in negative
performance issues. The following solutions are in place to meet this issue:
○ max_heap_table_size - This variable sets the maximum size to which MEMORY tables are
allowed to grow. The value of the variable is used to calculate MEMORY table MAX_ROWS values.

Oracle University and BOS-it GmbH & Co.KG use only

Setting this variable has no effect on any existing MEMORY table, unless the table is re-created with a
statement such as CREATE TABLE or altered with ALTER TABLE or TRUNCATE TABLE.
○ MAX_ROWS – For individual MEMORY tables, the maximum number of rows a table can store can be
set in the CREATE TABLE (or ALTER TABLE) syntax using the MAX_ROWS clause.
● Populate data to MEMORY tables on start up – When storing MEMORY tables with consistent data,
there is definitely an advantage to having that data present when the server starts up. This can be
accomplished with the --init-file start up option. In the file that is referred to in this startup option, a
command such as LOAD DATA INFILE or INSERT INTO ... SELECT commands can be executed.
This can be useful in server crashes and in replication.

InLine Lab 11-A

In this exercise you will compare the MEMORY and the MyISAM storage engines for reading and writing data.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered
the contents of the database.
Effect: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.
Action: Create (or recreate) the city_huge table (which defaults to using the MyISAM storage engine) with
approximately 160,000 records by executing the following SQL statement:

CALL create_city_huge(40);

Effect: This stored procedure will create (or recreate) a table called city_huge that is based on the City
table. Once the city_huge table is created, the stored procedure will load in the records from the City table
40 times, creating approximately 160,000 records in the new table.

11 - 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Step 2. Test the MyISAM table with SELECT queries

Action: Set the key_buffer_size system variable to approximately 50% of system memory by typing the
following in the mysql client:

SET GLOBAL key_buffer_size=1032940000;

Effect: The key_buffer_size system variable controls the size of the key cache. If your system does not have
512M of system memory, change the 256M key_buffer_size system variable to approximately 50% of the
memory on your system.
Action: Load the index into memory by typing the following SQL statement:

Oracle University and BOS-it GmbH & Co.KG use only

LOAD INDEX INTO CACHE city_huge;

Effect: This LOAD INDEX INTO CACHE statement pre-loads a table index into the default key cache. (Note:
LOAD INDEX INTO CACHE is used only for MyISAM tables.)
Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -q -i 5 /labs/scripts/MEMORY_query.sql

--create-schema=world

Effect: The MEMORY_query.sql file (which runs multiple SELECT statements) is executed against the
MyISAM city_huge table using the mysqlslap benchmarking tool 5 times.
Action: Record the number of seconds it takes to run all the SELECT statements:
Average: _______________
Minimum: _______________
Maximum: _______________
Step 3. Test the MyISAM table with UPDATE queries

Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -q -i 1 /labs/scripts/MEMORY_update.sql

--create-schema=world

Effect: The MEMORY_update.sql file (which runs multiple UPDATE statements) is executed against the
MyISAM city_huge table using the mysqlslap benchmarking tool.
Action: Record the number of seconds it takes to run all the UPDATE statements:
Average: _______________
Minimum: _______________
Maximum: _______________

11 - 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Step 4. Convert the table from MyISAM to MEMORY

Action: Reduce the size of the key_buffer_size system variable to approximately 10% of system memory
by typing the following in the mysql client:

SET GLOBAL key_buffer_size=206588000;

If your system does not have 512M of system memory, change the 50 M key_buffer_size system variable
to approximately 10% of the memory on your system.
Action: Increase the size of the max_heap_table_size system variable to approximately 50% of system
memory by typing the following in the mysql client:

Oracle University and BOS-it GmbH & Co.KG use only

SET SYSTEM max_heap_table_size=1032940000;

This variable sets the maximum size to which MEMORY tables are allowed to grow. If your system does not
have 512M of system memory, change the 256M max_heap_table_size system variable to approximately
50% of the memory on your system.
Action: Change the city_huge table to utilize the MEMORY storage engine by entering the following SQL
statement:

ALTER TABLE city_huge ENGINE=MEMORY;

Effect: The MEMORY storage engine moves all the city_huge data into memory (RAM).
Step 5. Test the MEMORY table with SELECT queries

Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -q -i 5 /labs/scripts/MEMORY_query.sql

--create-schema=world

Action: Record the number of seconds it takes to run all the SELECT statements now that city_huge table is
using the MEMORY storage engine:
Average: _______________
Minimum: _______________
Maximum: _______________

11 - 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Step 6. Test the MEMORY table with UPDATE queries

Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -p -q -i 1 /labs/scripts/MEMORY_update.sql

--create-schema=world

Action: Record the number of seconds it takes to run all the UPDATE statements now that city_huge table is
using the MEMORY storage engine:
Average: _______________

Oracle University and BOS-it GmbH & Co.KG use only

Minimum: _______________
Maximum: _______________
Step 7. Review the results

Action: Compare the MyISAM results with the MEMORY results, what do the numbers tell you?
Effect: You may find that the SELECT statements have similar run times; however, the UPDATE statements will
most likely run faster using the MEMORY storage engine.

11 - 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

244 11.3 FEDERATED Storage Engine

The FEDERATED storage engine accesses data in tables
of remote databases rather than in local tables. When a
FEDERATED table is created, the server creates a table
format file in the database directory. The file begins with
the table name and has an .frm extension. No other files Connection
MySQL API Client
are created, because the actual data is in a remote table. FEDERATED Remote
This differs from the way that storage engines for local Table Database
tables work. Table
With the MySQL FEDERATED storage engine, there

Oracle University and BOS-it GmbH & Co.KG use only

are no local data files for a table (for example, there is
no .MYD file). Instead, a remote database stores the data that normally would be in the table. The local server
connects to a remote server, and uses the MySQL client API to read, delete, update, and insert data in the remote
table. Data retrieval is initiated via a SELECT * FROM tbl_name SQL statement. To read the result, rows are
fetched one at a time by using the mysql_fetch_row() C API function, and then converting the columns in the
SELECT result set to the format that the FEDERATED handler expects. The FEDERATED storage engine is only
available in the MySQL MAX distribution.
245 The flow of information is as follows:
1. SQL calls issued locally
2. MySQL handler API (data in handler format)
3. MySQL client API (data converted to SQL calls)
4. Remote database -> MySQL client API
5. Convert result sets (if any) to handler format
6. Handler API -> Result rows or rows-affected count to local
246 Limitations
The following items indicate features that the FEDERATED storage engine does and does not support:
● Remote servers must be MySQL servers.
● A remote table that a FEDERATED table points to must exist before access to the table through the
FEDERATED table is allowed.
● It is possible for one FEDERATED table to point to another, but care must be taken to not create a loop.
● There is no support for transactions.
● There is no way for the FEDERATED engine to know if the remote table has changed. The reason for this is
that this table must work like a data file that would never be written to by anything other than the database.
The integrity of the data in the local table could be breached if there was any change to the remote database.
● The FEDERATED storage engine supports SELECT, INSERT, UPDATE, DELETE, and indexes. It does not
support ALTER TABLE, DROP TABLE, or any other Data Definition Language statements. The current
implementation does not support prepared statements.

11 - 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

● The implementation uses SELECT, INSERT, UPDATE, and DELETE, but not HANDLER.
● FEDERATED tables do not work with the query cache.
NOTE: FEDERATED Tables and Performance

Due to the numerous performance limitations, the FEDERATED storage engine is not a choice for applications
that require speed and/or the use of extensive business rules. The FEDERATED storage engine has features that
are useful in a less demanding capacity but should be carefully evaluated prior to putting into any system that
requires demanding performance.

Oracle University and BOS-it GmbH & Co.KG use only

11 - 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

247 11.4 ARCHIVE Storage Engine

The ARCHIVE storage engine is used for storing large amounts of data without indexes in a very small footprint.
When an ARCHIVE table is created, the server creates a table format file in the database directory. The file begins
with the table name and has an .frm extension. The storage engine creates other files, all having names beginning
with the table name. The data and metadata files have extensions of .ARZ and .ARM, respectively.
The ARCHIVE engine supports INSERT and SELECT, but not DELETE, REPLACE, or UPDATE. It does support
ORDER BY operations, BLOB columns, and basically all but spatial data types. The ARCHIVE engine uses row-level
locking.
248 Storage

Oracle University and BOS-it GmbH & Co.KG use only

Rows are compressed as they are inserted. The OPTIMIZE TABLE can be used to analyze the table and pack it into
a smaller format. There are several types of insertions that are used:
● INSERT Statement - An INSERT statement just pushes rows into a compression buffer, and that buffer
flushes as necessary. The insertion into the buffer is protected by a lock. A SELECT forces a flush to occur,
unless the only insertions that have come in were INSERT DELAYED (those flush as necessary).
● Bulk Insert - A bulk insert is visible only after it completes, unless other inserts occur at the same time, in
which case it can be seen partially. A SELECT never causes a flush of a bulk insert unless a normal insert
occurs while it is loading.
249 Retrieval
On retrieval, rows are uncompressed on demand; there is no row cache. A SELECT operation performs a complete
table scan: when a SELECT occurs, it finds out how many rows are currently available and reads that number of
rows. SELECT is performed as a consistent read.
NOTE: SELECT statements

A large number of SELECT statements during insertion can deteriorate the compression, unless only bulk or
delayed inserts are used.

Performance Issues
The ARCHIVE storage engine reduces the size of most other storage engine files by up to 70%. This can be
extremely effective for reducing the size of data that is no longer being modified and is strictly historical in nature.
Furthermore, due to the nature of business data, having historical data can be extremely important for a multiple
number of business and sometimes government regulation reasons.

11 - 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

250 11.5 CSV Storage Engines

The CSV storage engine is a specialized storage engine that stores all the data associated with it in *.csv file format.
The CSV storage engine is not available on the default installations. It must be built with the --with-csv-storage-
engine option. When a CSV table is created, the server creates a table format file in the database directory. The file
begins with the table name and has an .frm extension. The storage engine also creates a data file. Its name begins with
the table name and has a .csv extension. The data file is a plain text file. When data is stored into the table, the storage
engine saves it into the data file in comma-separated values format. The table data can be copied from the database
directly and transferred to a client such as a spreadsheet application, and the table may be opened in a spreadsheet format.
As no indexes are allowed for the storage engine, there is limited use for this storage engine table type.
251

Oracle University and BOS-it GmbH & Co.KG use only

Duplicating the CSV function with other storage engines
A SELECT statement on other storage engine tables with the INTO OUTFILE clause can also create the table data in
*.csv file format with a command similar to the one below:

SELECT * INTO OUTFILE 'c:\\mysql\\city.csv' FIELDS TERMINATED BY ', '

ENCLOSED BY '"' FROM city;

The following is a sample of the data represented in either a CSV table or using the above command:

"1", "Kabul", "AFG", "Kabol", "1780000"

"2", "Qandahar", "AFG", "Qandahar", "237500"
"3", "Herat", "AFG", "Herat", "186800"
"4", "Mazar-e-Sharif", "AFG", "Balkh", "127800"
"5", "Amsterdam", "NLD", "Noord-Holland", "731200"
"6", "Rotterdam", "NLD", "Zuid-Holland", "593321"
"7", "Haag", "NLD", "Zuid-Holland", "440900"
"8", "Utrecht", "NLD", "Utrecht", "234323"
"9", "Eindhoven", "NLD", "Noord-Brabant", "201843"
"10", "Tilburg", "NLD", "Noord-Brabant", "193238"
"11", "Groningen", "NLD", "Groningen", "172701"
"12", "Breda", "NLD", "Noord-Brabant", "160398"

InLine Lab 11-B

In this exercise you will compare the ARCHIVE and the MyISAM (using compression) storage engines for read speeds
and size on disk.

Step 1. Prepare the database

Action: Install a fresh copy of the world database to remove any indexes or other actions that may have altered the
contents of the database.
Action: Source the pt_stored_procedures.sql file to load the stored procedures that will be used in the
labs for this chapter.

11 - 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Step 2. Create a MyISAM table

Action: Recreate the city_huge table (which defaults as the MyISAM storage engine) with approximately
800,000 records by executing the following SQL statement:

CALL create_city_huge(200);

Effect: This stored procedure will create (or recreate) a table called city_huge that is based on the City
table. Once the city_huge table is created, the stored procedure will load in the records from the City table
200 times, creating approximately 800,000 records in the new table.

Oracle University and BOS-it GmbH & Co.KG use only

Step 3. Compress the MyISAM table

Action: Execute the myisampack command in the world directory underneath the mysql data directory
(most likely /usr/local/mysql/data/world) to compress the MyISAM table:

myisampack city_huge.MYI

Effect: The myisampack utility compresses MyISAM tables. myisampack works by compressing each
column in the table separately. Usually, myisampack packs the data file 40%-70%. (Note: You must have O/S
access rights to the data directory to perform actions against the data in the database outside of the mysql
client.)
Step 4. Check the compressed MyISAM table

Action: Execute the following myisamchk command in the O/S terminal window (command line client):

myisamchk -rq city_huge.MYI

Effect: The myisamchk utility gets information about your MyISAM database tables or checks, repairs, or
optimizes them. Using the -rq options tells myisamchk to repair (-r) using a faster repair (-q) by not
modifying the data file.
Step 5. Test the compressed MyISAM table with SELECT queries

Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -q -i 1 /labs/scripts/MEMORY_full_query.sql

--create-schema=world

Effect: The MEMORY_full_query.sql file (which runs two SELECT statements that produce a full search
against the data) is executed against the city_huge table using the mysqlslap benchmarking tool.

11 - 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Action: Record the number of seconds it takes to run all the SELECT statements:
Average: _______________
Minimum: _______________
Maximum: _______________
Action: Execute the following mysqlslap command in the O/S terminal window (command line client):

mysqlslap -uroot -q -i 1 /labs/scripts/MEMORY_query.sql

--create-schema=world

Effect: The MEMORY_query.sql file (which runs multiple SELECT statements) is executed against the

Oracle University and BOS-it GmbH & Co.KG use only

city_huge table using the mysqlslap benchmarking tool.
Action: Record the number of seconds it takes to run all the SELECT statements:
Average: _______________
Minimum: _______________
Maximum: _______________
Step 6. Create an ARCHIVE table

Action: Create an identical table to city_huge called city_huge_archive by executing the following the
SQL statements in the mysql client:

CREATE TABLE city_huge_archive ENGINE=archive SELECT * FROM city_huge;

RENAME TABLE city_huge TO city_huge_myisam, city_huge_archive TO city_huge;

Effect: There are now two tables: city_huge_myisam that contains the data compressed using MyISAM
compression, and the city_huge table which is now utilizing the ARCHIVE storage engine.
Step 7. Review the files utilized by the two tables

Action: In the world directory under the MySQL data directory (most likely
/usr/local/mysql/data/world), compare the files city_huge_myisam.MYD and city_huge.ARZ
using the following command in an O/S terminal window:

ls -l city_huge*

Action: Record the sizes on disk below:

city_huge_myisam.MYD: ___________
city_huge.ARZ: ___________
How do the sizes compare?

11 - 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Step 8. Test the ARCHIVE table with SELECT queries

Action: With the city_huge table now using the ARCHIVE storage engine, execute the following
mysqlslap command in the O/S terminal window (command line client) again:

mysqlslap -uroot -q -i 1 /labs/scripts/MEMORY_full_query.sql

--create-schema=world

Action: Record the number of seconds it takes to run all the SELECT statements:
Average: _______________

Oracle University and BOS-it GmbH & Co.KG use only

Minimum: _______________
Maximum: _______________
Action: With the city_huge table now using the ARCHIVE storage engine, execute the following
mysqlslap command in the O/S terminal window (command line client) again:

mysqlslap -uroot -q -i 1 /labs/scripts/MEMORY_query.sql

--create-schema=world

(Note: This step could take a considerably long time to complete, please feel free to kill the process if the time is
excessive.)
Action: Record the number of seconds it takes to run all the SELECT statements:
Average: _______________
Minimum: _______________
Maximum: _______________
Step 9. Review the results

Action: Compare the MyISAM compression results with the ARCHIVE results, what do the numbers tell you?

11 - 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

252 11.6 BLACKHOLE Storage Engine

The BLACKHOLE storage engine acts as a “black hole” that accepts data but essentially throws it away and does not
store it. When a BLACKHOLE table is created, the server creates a table format file in the database directory. The file
begins with the table name and has an .frm extension. There are no other files associated with the table.
Inserts into a BLACKHOLE table do not store any data, but if the binary log is enabled, the SQL statements are logged
(and replicated to slave servers).

Relay Slave
A BLACKHOLE can be useful as a slave server whose purpose is to relay the master server’s data to a slave or multiple
slaves through a relay slave. This relay slave (or middle server) would contain BLACKHOLE tables that are designed to

Oracle University and BOS-it GmbH & Co.KG use only

pass the data on but not hold any of the data itself. This would allow the master to have to send the data only once and
then using the middle server to replicate to the other slaves, depicted as follows:

Slave
Server

Master Relay
Host Slave

Slave
Server

253 Filter Mechanism

A BLACKHOLE can also be useful as a repeater or filter mechanism. For example, suppose an application requires
slave-side filtering rules, but transferring all binary log data to the slave first results in too much traffic. In such a case, it
is possible to set up on the master host a “dummy” slave process whose default storage engine is BLACKHOLE, depicted
as follows:

11 - 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Master Slave
Host Server

Oracle University and BOS-it GmbH & Co.KG use only

Master Dummy Slave
mysqld mysqld mysqld
process process process

The master writes to its binary log. The “dummy” mysqld process acts as a slave, applying the desired combination
of replicate-do-* and replicate-ignore-* rules, and writes a new, filtered binary log of its own. The dummy process
does not actually store any data, so there is little processing overhead incurred by running the additional mysqld
process on the replication master host. This type of setup can be repeated with additional replication slaves.
NOTE: Other possible uses for the BLACKHOLE storage engine

• Verification of dump file syntax.

• Measurement of the overhead from binary logging, by comparing performance using BLACKHOLE
with and without binary logging enabled.
• BLACKHOLE is essentially a “no-op” storage engine, so it could be used for finding performance
bottlenecks not related to the storage engine itself.

11 - 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

254 11.7 How Using Multiple Storage Engines Can Affect Performance
Storage engines can be mixed on the same server or even in a single query. Constant data can be stored in MyISAM,
dynamic critical data in InnoDB, and MEMORY for temporary tables. Conversion back and forth is simple:

ALTER TABLE <table> ENGINE=<engine>;

On the downside mixed database configurations are more complicated, including for backup, maintenance, tuning
purposes. There is also the potential of bugs while using multiple storage engines. The optimizer may have an especially
hard time. In addition, all storage engines have their own memory buffers. Thus using many storage engines results in
less buffer space for each engine.

Oracle University and BOS-it GmbH & Co.KG use only

NOTE: Performance Issues with Using Multiple Storage Engines

Mixing of storage engines can improve overall performance, for example, using MEMORY for small tables that
change a lot or caches, MyISAM for read-heavy data, or when FULLTEXT indexes are needed, InnoDB for data
with lots of UPDATEs and DELETEs, ARCHIVE for logs, etc.
A disadvantage with mixing of multiple storage engines is that they all have their own buffers, thus it becomes
much more difficult to configure MySQL for all used engines as the memory has to be efficiently shared. Another
disadvantage comes from mixing transactional and non-transactional tables in the same transaction. This should
not be done as operations on non-transactional tables cannot be rolled back.

11 - 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

255 11.8 Creating a Custom Storage Engine

Custom storage engines for MySQL can be difficult to develop but can provide a custom solution to meet the needs of
a specific problem not solved by the other available storage engines. The EXAMPLE Storage Engine is supplied as a
stub for persons desiring to build their own storage engines. The following list displays examples of a number of
possible custom storage engines:
● “Archive” storage engine to deal with huge log files.
● Special distributed storage engines.
● Storage engines for fuzzy matches.
Storage engine for network lookup.

Oracle University and BOS-it GmbH & Co.KG use only

●
● Storage engine to read apache log files.
MySQL development and support can help with design and implementation. Custom storage engines can be built in a
progressive manner: Developers can start with a read-only storage engine and later add support for INSERT,
UPDATE, and DELETE operations, and even later add support for indexing, transactions, and other advanced
operations. When developing a custom storage engine, the developer needs to address the handler interface necessary
to access the storage engine via the optimizer.

11 - 18
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

256 11.9 MySQL Cluster

With the rising demand for high availability and scalability, organizations are looking for approaches to meet the needs
required to grow their businesses, improve efficiencies and cut costs. MySQL Cluster meets these requirements by
implementing a distributed, highly fault tolerant architecture with no single point of failure. In essence, MySQL Cluster
delivers mainframe reliability on commodity hardware making it the most cost-effective solution for high availability.

Advantages of MySQL Cluster

MySQL Cluster’s unique parallel fault tolerant architecture provides the following benefits:
● Cost Effective – requiring less hardware, lower maintenance costs, and affordable licensing compared to
proprietary databases.

Oracle University and BOS-it GmbH & Co.KG use only

● High Performance – that only an in-memory database can provide.
● Linear Scalability – to incrementally scale the system without a high initial hardware investment.
● No Single Point of Failure – using a distributed node-based architecture with fast failover.
● Automatic Fast Failover - enabling systems to automatically fail over in less than a second.

Architecture of MySQL Cluster

In order to achieve fault tolerance a number of distributed components are necessary to run the application, access the
database, and manage the distributed environment. These components are typically distributed across machines and
continuously share the latest changes to the database with one another.
If a node or machine fails, there are other nodes that contain the same information so an application can continue to run
without interruption. The system must also ensure that transactions are successfully executed and database information
remains consistent throughout the recovery process.
MySQL Cluster offers a flexible distributed database management system that consists of three kinds of nodes which
replicate data, access the database, and manage the system:
● Data Nodes – These nodes are the main nodes of the system. All data is replicated and stored on the database
nodes and the database handles all database transactions. If a database node fails, then there is always another
database node storing the same information.
● SQL Nodes – These nodes are the applications connecting to the database. If a database node fails then there is
always another database node storing the same information allowing the application node to continue to run.
● Management Server - These nodes handle the system configuration. The management server nodes are used
only at startup and for system reconfiguration. Management server nodes can be stopped and restarted without
affecting the ongoing execution of the database and application nodes.

11 - 19
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

257 SQL
Nodes MySQL MySQL MySQL MySQL
Server Server Server Server

Data
NDBCluster
Nodes

Oracle University and BOS-it GmbH & Co.KG use only

Management Server Manager
Client

This architecture ensures there is no single point of failure. Applications continue to run and data remains consistent,
even if any one of the data, SQL, or management server nodes fail. In addition, nodes and computers can be
distributed across geographies. This is important to recover a database or an entire cluster in case of a physical
disaster in a particular location.
258 NDBCluster Storage Engine
The NDBCluster storage engine runs in server memory and supports transactions and synchronous replication.
“Synchronous replication" is between servers only and is automatic. The data is thus spread across many servers.
The engine spreads data redundantly across many servers in the cluster. The storage engine allows one of the servers
to go offline and does not interrupt the data availability. The storage engine uses row-level locking. All reads are non-
locking by default. READ-COMMITTED is the only supported isolation level. As with clusters in general, the NDB
Cluster storage engine ensures individual node crashes do not stop the cluster. This storage engine provides for
automatic synchronization of data nodes at restart as well as recovery from checkpoints and logs at cluster crash.
The NDB Cluster storage engine can perform many maintenance tasks online, including on-line backup and on-line
software upgrades. In addition, the storage engine supports unique hash and T-tree ordered indexes.
NOTE: MySQL Cluster for High Availability Training Course

This section was designed to be an introduction to MySQL Clustering; however, due to the enormity of this
database application, time does not permit for this training course to give the application the time it deserves.
The MySQL Cluster for High Availability training course is the recommended course. This is a 3 day course
taught by an Authorized MySQL instructor who will explain and demonstrate important details of clustering:
how to get started with MySQL Cluster, how to properly configure and manage the cluster nodes to ensure
high availability, how to install the different nodes, and understanding of the internals of the cluster.

11 - 20
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

Quiz
In this exercise connect the storage engine type with the features(s) associated. Note: Each storage engine may have more
than one feature it connects to.
Storage Engine Type Feature

________ ARCHIVE A. Does not support ALTER TABLE, DROP

TABLE, or any other Data Definition Language statements

________ BLACKHOLE B. Can not store BLOB or TEXT type fields

________ CLUSTER C. Implements a distributed, highly fault tolerant

Oracle University and BOS-it GmbH & Co.KG use only

architecture with no single point of failure.

________ CSV D. Stores all data in memory; once the MySQL

server has been shut down any information stored in it will
have been lost.

________ FEDERATED E. Used for storing large amounts of data without

indexes in a very small footprint.

________ MEMORY F. Accesses data in tables of remote databases rather

than in local tables.
G. max_heap_table_size sets the maximum
size to which the tables are allowed to grow.
H. On retrieval, rows are uncompressed on demand;
there is no row cache.
I. When data is stored into the table, the storage
engine saves it into the data file in comma-separated values
format.
J. Only an .frm file is created on the local server
when using this storage engine type.
K. Inserts do not store any data, but if the binary log
is enabled, the SQL statements are logged (and replicated
to slave servers).
L. Offers a flexible distributed database management
system that consists of three kinds of nodes which replicate
data, access the database, and manage the system.
M. Can also be useful as a repeater or filter
mechanism.

11 - 21
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 11 - Alternate Storage Engines

11.10 Chapter Summary

259 In this lesson, you have learned to:
● List the characteristics of the MEMORY storage engine
The MEMORY storage engine (previously known as the HEAP storage engine) stores all data in memory.
● List the characteristics of the FEDERATED storage engine
The FEDERATED storage engine accesses data in tables of remote databases rather than in local tables.
● List the characteristics of the ARCHIVE storage engine
The ARCHIVE storage engine is used for storing large amounts of data without indexes in a very small

Oracle University and BOS-it GmbH & Co.KG use only

footprint.
● List the characteristics of the CSV storage engine
The CSV storage engine is a specialized storage engine that stores all the data associated with it in *.csv file
format.
● List the characteristics of the BLACKHOLE storage engine
The BLACKHOLE storage engine acts as a “black hole” that accepts data but essentially throws it away and
does not store it.
● Explain how using multiple storage engines can be used to improve performance
Mixing of the storage engines can improve overall performance: for example, using MEMORY for small
tables that changes a lot or caches; MyISAM for read-heavy data, or when FULLTEXT indexes are needed;
InnoDB for data with lots of UPDATES and DELETES; ARCHIVE for logs, etc.
● Explain the concept of custom storage engines
Custom storage engines for MySQL can be difficult to develop but can provide a custom solution to meet the
needs of a specific problem not solved by the other available storage engines. The EXAMPLE storage
engine is supplied as a stub for persons desiring to build their own storage engines.
● Describe the fundamentals of MySQL Cluster
MySQL Cluster delivers mainframe reliability on commodity hardware, making it the most cost-effective
solution for high availability.

11 - 22
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

260
MySQL Performance Tuning

12 CONCLUSION

12 - 1
Chapter 12 - Conclusion

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

12.1 Learning Objectives

This chapter concludes the course and allows the instructor to answer any remaining questions you may have at this
time. It also contain additional information regarding MySQL training and certification. You should now be able to:
261 ● Describe specific contents of the MySQL training and certification web pages
● Complete a course evaluation, to aid in continuing improvements to MySQL courses
● Use the various contact information for additional training information and support
● Find additional training information

Oracle University and BOS-it GmbH & Co.KG use only

12 - 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

262 12.2 Training and Certification Website

View the course catalog, schedule, certification program information, venue information (and more) on the Training and
Certification website: http://www.mysql.com/training/

Oracle University and BOS-it GmbH & Co.KG use only

12 - 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

On the site, you will find links to:

● Certification – Description of current certification program. Links to exam registration and more
information.
● Curriculum Paths – The categories and order in which training courses should be taken.
● Catalog – Comprehensive course catalog. Links to more information on courses.
● Schedule – Schedule of courses, listed by title. Links to course registration and more information.
● Delivery Options – The available options for training delivery.
● Savings and Promotions – Information about training bundles and credits.

Oracle University and BOS-it GmbH & Co.KG use only

● Training Bundles – Available bundles and pricing.
● Training Partners – List and links of MySQL training partners.
● Testimonials – Training endorsements.
● FAQ – Frequently asked questions and links to more other training information.
Additional training information, including Articles, White Papers and Web Seminars, can be found on the community
Developer Zone web page: http://dev.mysql.com/

12 - 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

263 12.3 Course Evaluation

We need your evaluation!

Please take the time to complete a course evaluation to let us know about your experience. We are continuously making
improvements to our courses and our overall training program. The feedback of our students is invaluable in assisting us
to make the right choices.
Please follow the instructions given by your instructor regarding completion of the evaluation form.

Thank you in advance for taking the time to give us your opinions!

Oracle University and BOS-it GmbH & Co.KG use only

12 - 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

264 12.4 Thank you!

We appreciate your attendance at this course. Congratulations on your completion. We hope that your training needs
regarding this course topic have been fully met. Do not hesitate to contact us, if you have any training questions:
Website: http://www.mysql.com/training/
Email: [email protected]
Phone: USA Toll Free: 1-866-697-7522
Worldwide: 1-208-514-4780

Oracle University and BOS-it GmbH & Co.KG use only

12 - 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Chapter 12 - Conclusion

265 12.5 Q&A Session

If you have more questions after the class, get answers from the MySQL Reference Manual FAQ page:
http://dev.mysql.com/doc/refman/5.1/en/faqs.html .
And if you would like to work on the exercises after completing this course, you may download the world database
(which was used throughout the course) from the Documentation page of our website: http://dev.mysql.com/doc .
You may record any questions and answers that are shared at this point in the class, in the space provided below:

Oracle University and BOS-it GmbH & Co.KG use only

12 - 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX A MYSQL LAB PREPARATION

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix A - MySQL Lab Preparation

A.1 MySQL Server installation on Microsoft Windows

NOTE: For the purposes of this course, you will download the 5.1 Generally Available (GA) release,
Essentials Version:

1) Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2) Under column entitled MySQL Community Server, click on DOWNLOAD link
3) Scroll down to the MySQL Community Server 5.1.x GA release section.

Oracle University and BOS-it GmbH & Co.KG use only

4) Find the Windows Essentials (x86, 32-bit) MSI Installer version and click on the Download button.

5) The registration at the top of the page is optional, so skip it and select the link at the bottom of the page that
reads No thanks, just take me to the downloads!. You will not perform registration during the course.
6) Scroll down the page to the section showing the “Mirrors in: ...”. Find the location closest to you and click
on the link next to it called HTTP:

Appendix A-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix A - MySQL Lab Preparation

7) A window will pop up with the option to Save File or Cancel. Select Save File to download the software
for installation. (If a window regarding security appears, click on Run to allow the download.)
8) When the file (mysql-essential-<release_#>-win32.msi) is finished downloading, open the
file to start the installation.
9) The Windows Setup Wizard will appear. Click on the Next> button to perform the set-up.
10) Select a Typical installation.

Oracle University and BOS-it GmbH & Co.KG use only

The wizard is ready to install and indicates that the install directory will be as follows:
“<local_disk>\Program Files\MySQL\MySQL Server 5.1”
11) Start the install by clicking on the Install button. The window will indicate that it is Installing MySQL
Server 5.1.
12) A MySQL Enterprise window will pop-up. Click the Next> button to move through the information and
continue with the installation.
13) In the Wizard Completed window, make sure that check box labeled Configure the MySQL Server now
is selected.
Deselect the Register the MySQL Server now check box. (You will not need to register for the purposes
of this class.)
14) The MySQL Server Instance Configuration wizard will appear. Click on the Next> button to perform
the configuration.
15) If your system already contains an older version of the MySQL server, you will select the Reconfigure
Instance choice.
NOTE: Make sure to save your previous my.ini file, just in case you need to get any old settings back.

16) Select Detailed Configuration, then click the Next> button.

17) Select Developer Machine, then click the Next> button.
18) Select Multifunctional Database, then click the Next> button.
19) Do not change any InnoDB Tablespace Settings at this time, and click the Next> button.
20) Select Decision Support, then click the Next> button.
21) Select Enable TCP/IP Networking and Enable Strict Mode, then click the Next> button.
22) Select Standard Character Set, then click the Next> button.
23) Select Include Bin Directory in Windows PATH, then click the Next> button.
24) Select Modify Security Settings, then enter “root” as the new password. Then click the Next> button.
NOTE: Use of an “anonymous” account is not recommended, due to the lack of security.

25) Click on the Execute button.

The wizard now shows the execution of the steps to configuration, and that the configuration file was
created successfully.
26) Click on the Finish button to complete.

Appendix A-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix A - MySQL Lab Preparation

A.2 MySQL Server installation on Linux

NOTE: For the purposes of this course, you will download the 5.1 Generally Available (GA) release,
Essentials Version:

1) Go to the MySQL web site downloads page: http://dev.mysql.com/downloads/

2) Under column entitled MySQL Community Server, click on DOWNLOAD link
3) Scroll down to the MySQL Community Server 5.1.x GA release section.

Oracle University and BOS-it GmbH & Co.KG use only

4) Under Select Platform: use the pull-down menu to choose Linux-Generic
5) Scroll down and find the Generic Linux (glibc 2.3) (x86, 32-bit), Compressed TAR Archive version
(mysql-5.1.44-linux-i686-glibc23.tar.gz).
6) Click on the DOWNLOAD button corresponding to the above version.
7) The registration at the top of the page is optional, so skip it and select the link at the bottom of the page that
reads No thanks, just take me to the downloads!. You will not perform registration during the course.
8) Scroll down the page to the section showing the “Mirrors in: ...”. Find the location closest to you and click
on the link next to it called HTTP:

9) A window will pop up with the option to Save File or Cancel. Select Save File to download the software
for installation.
10) Unpack the tarball to /usr/local with (change /tmp/ to the proper path):

cd /usr/local
tar –zxvf /tmp/mysql-5.1.44-linux-i686-glibc23.tar.gz

11) Add a symbolic link to the directory:

ln –s mysql-5.1.44-linux-i686-glibc23 mysql

12) Install the system tables:

Appendix A-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix A - MySQL Lab Preparation

cd mysql
scripts/mysql_install_db

13) Add a mysql system user and group:

groupadd mysql

useradd –g mysql mysql

Oracle University and BOS-it GmbH & Co.KG use only

14) Change the owner of the data directory:

chown –R mysql:mysql data

15) Copy the startup script to /etc/init.d:

cp support-files/mysql.server /etc/init.d/mysql

16) Start the script:

/etc/init.d/mysql start

17) Edit the /etc/bashrc file and add this path:

PATH=${PATH}:/usr/local/mysql/bin

A.3 MySQL 'world' database file installation

1) Go to the MySQL community web site “Documentation” page: http://dev.mysql.com/doc/
2) Select the Other Docs link at the top of the page

3) Under the Example Databases section, select the Zip file for download of the world database:
4) Unzip the file into the <local_disk> folder. (i.e. directly under the C:/ or D:/ drive.)
Note: For the purposes of this course, you will not need to download the setup guide.

Appendix A-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX B INTRODUCTION TO LINUX

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

This appendix is intended as a quick introduction to the Linux operating system. The purpose is to allow individuals
not familiar with Linux or the Linux command line the ability to navigate and create files, view directories. This
document was created as part of the classes offered by MySQL so it is biased towards commands that we use in our
classes.

Windows in Linux?
This document assumes that the Linux version being used is the most recent and that a graphical environment has
been installed. The basic system for a graphical environment in Linux is X-Windows. X-windows or X-Free-86 is
the Linux graphical user interface (GUI) and provides the basic framework for utilizing the Linux operating system.

Oracle University and BOS-it GmbH & Co.KG use only

Running within “X” is a window manager or a more complete environment known as a “desktop”. The most
common “desktops” are Gnome or Kde. There are other lightweight “window manager's” that run within X. Even
though Linux offers a windows like environment, Linux is centered around the command line and will be the topic
of major discussion in this document.
B.1 The Basics

Getting to the command line

When Linux starts up it may launch a GUI interface but it will also launch a command line interface. If you start
with the GUI then a command line can be obtained in one of the following ways:
● CTRL-ALT-F1
● CTRL-ALT-F2
● CTRL-ALT-F3
● CTRL-ALT-F4
● CTRL-ALT-F5
● CTRL-ALT-F6
● CTRL-ALT-F7 (BACK TO THE GUI)

Starting X-Windows from a command line

If the operating system starts up without loading the X-Windows environment (the display only showing the
command line), or there is a need to have multiple windows environments, the X-Windows GUI can be loaded by
entering one of the following ways:
● startx
○ When this command is entered on the command line it will start up the X-Windows server and display
the default windows manager.
● startx --:#
○ When startx is executed without any parameters, it is actually starting the server with startx --:0.
This identifies that it is loaded in display 0. Additional X-Windows servers can be loaded by executing
the startx command with other display identifiers being passed through the parameter (eg. startx
--:1 would create a second X-Windows server and supporting GUI). Several GUI terminals can be
running concurrently with the following commands allowing switching between the environments:
■ CTRL-ALT-F7
■ CTRL-ALT-F8

Appendix B-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

Getting to the command line from the GUI

In most Linux distributions a command line or “terminal” is available in the windows manager as an application. In
most operating system it is under APPLICATIONS => ACCESSORIES => TERMINAL. The Terminal application
can be added to the "Start bar" by simply dragging the icon to the "Start panel" in most operating systems.

General Notes
● Linux is CaSe-SeNsItIvE which is different from a Windows environment. Dog.txt, DOG.txt and
doG.txt would be considered three different file names in Linux, where in Windows they would all be

Oracle University and BOS-it GmbH & Co.KG use only

seen as one file. Three are other specific file name differences that must be taken into account when using
Linux:
● Filenames can be up to 256 characters long and can contain letters, numbers, "." (dot), "_" (underscore), "-"
(dash), for obvious reasons (spaces) should be avoided.
● If a file starts with a “.” it will be hidden. It will not usually show up in the ls or dir command (discussed
in detail later in this appendix). Using ls -a will show hidden files.
● “/” is more or less equivalent to a Microsoft DOS “\”. The difference is that there is only one directory tree
in Linux and it spans all disks. In Microsoft Windows there is one tree per disk. The Linux filesystem
hierarchy is independent of the disks.
● Filenames can be up to 256 characters long and can contain letters, numbers, "." (dot), "_" (underscore), "-"
(dash), for obvious reasons (spaces) should be avoided.
● In most files and programs (eg. the MySQL configuration file /etc/my.cnf), a line starting with a # is a
comment.

Assorted non-command features

In Linux there are many time-saving features that are not available in most window like environments. The
following is a listing of some of the most useful features:
● ALL commands available
Press <Tab> to see the listing of all available commands in the path selected.
● TAB completion
<TAB> in a text terminal will autocomplete the command if there is only one option. For example ls
/usr/local/my<TAB> will return ls /usr/local/mysql thus completing the only possible command that fit
what was typed in. Simply hitting enter allows the command, in this case loading the mysql client, to
execute.
● Command history
<ARROWUP> will scroll through approximately the last 1000 commands that were executed (from the
most recent on). The command displayed can be reexecuted by simply pressing the <ENTER> or
<RETURN> key; however, it can also be altered and then executed with the new changes.
● Scroll through multiple page output
When the contents of the terminal window exceed the display size, using the scroll bar can move the
display up and down through the contents. However, there is also keyboard commands that can accomplish
this task also: <SHIFT><PgUp> and <SHIFT><PgDN>.
● Screen resolution in the GUI

Appendix B-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

The resolution of the screen can be changed using the following keyboard commands:
○ <CTRL><ALT><+> - Increases the resolution (eg. 800x600 => 1024x768).
○ <CTRL><ALT><-> - Decreases the resolution (eg. 1024x768 => 800x600).
Note: In X-Windows it may be necessary to add the resolution setting sizes to the X-Windows
configuration file (eg. /etc/X11/XF86Config). The first resolution size identified will be the default setting
when X-Windows starts and the other settings will be used when the <CTRL><ALT><+> and
<CTRL><ALT><-> keys are used. If the entries are not in normal resolution order (eg. "1024x767",
"800x600", "640x480", etc.), then these keys may not necessarily increase or decrease as expected (it

Oracle University and BOS-it GmbH & Co.KG use only

moves up and down through the list of resolutions not evaluating if it is smaller or larger than the resolution
being used).
● Terminating the GUI environment
There are times when it is not possible to terminate an X-Windows environment using proper procedures.
In the event that this happens, the X-Windows environment can be "killed" by using either of the following
key combinations:
○ <CTRL><ALT><BACKSPACE> - This should terminate the X-Windows server and provide a
command prompt.
○ <CTRL><ALT><DELETE> - The majority of Linux environments will catch this and reboot the
system.
● Terminating a running application
In addition to times when an X-Windows environment will not terminate properly, applications running
within the X-Windows environment can also fail to terminate properly. To terminate the unresponsive
application use the following key combinations: <CTRL><c>. This will send a KILL signal to the current
process.
● Close a Terminal window
To terminate the current Terminal window use the following key combinations: <CTRL><d>. This sends
an [End-of-File] to current open application. Caution: If this key combination is pressed twice, the system
will log out the current user.
● Stopping and resuming transfers
If a terminal mysteriously stops responding, it may be in the process of receiving or sending actions. In this
case, it is possible to stop a transfer being received by the terminal by pressing the following key
combination: <CTRL><s>. The transfer can also be resumed by pressing the following key combinations:
<CTRL><q>.
● Placing current process in the background
There are times when the current running application does not need to be monitored by the end user or no
additional interaction is required by the end user, but the application still needs to be executing. Rather than
starting another terminal window, the following key combination places the application execution in the
background: <CTRL><z>.
B.2 Using the command line
In Linux there are distinct differences from other operating systems (such as Microsoft Windows) that must be
understood to reduce frustration when attempting to execute applications. The first such difference is the fact that
Linux is a file system with a single tree like structure that starts at "/". This type of file system supports what is
known as absolute or relative file or directory references.

Appendix B-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● Absolute path - This type of path is a full path that identifies the actual location of the file or directory
being called without giving any thought to the current directory being used. This can be useful when it is
necessary to start an application in a directory that the executables are not located in. By calling the
executable full path (eg. /usr/local/mysql), from within another directory (eg. /tmp), any external
dealings from within the application will take place in the directory started (unless otherwise identified by
the application itself).
● Relative path - This type of path identification allows for application that are located within the current
directory to be executed without having to identify the exact path (eg. executing mysql within the
/usr/local directory). The operating system actually appends the current directory to the file name

Oracle University and BOS-it GmbH & Co.KG use only

being executed (invisble to the end user). In addition, Linux also allows for user identified paths (or
commands) to be relative no matter where they are executed from (such as many of the traditional
applications like help which is located in a common operating system executables directory).

Time-saving features
When using the command line there are many time-saving features that can be used to minimize the amount of
typing that is needed. The following is a recap (or full description) of the most useful:
● TAB completion - <TAB> in a text terminal will autocomplete the command if there is only one option.
● Wildcards - Wildcards can subsituted for file names. The most common wildcard is *. By issuing the
following command (with a wild card):, the entire contents of the directory are displayed (including all
subdirectories and there contents):

ls /usr/local/*

● Shortcuts - In Linux there are multiple characters that assist in minimizing the amount of typing necessary.
The following are the most common:
○ ~ (tilde) - This character is short for the current users home directory (most likely
/home/username). If the following was entered: cd ~/my_dir, the operating system would go to
the /home/username/my_dir directory. The command cd alone is equivalent to typing cd ~.
○ . (dot) - This character refers to the current directory. Sometimes this is necessary to execute
applications located with the current directory. The following: ./my_program will attempt to
execute the application my_program in the current working directory.
○ .. (two dots) - These characters (when used consecutively) will reference the parent directory to the
current working directory. The following: cd .. when executed in the /usr/local directory will
make the /usr directory the current working directory.
○ & (ampersand) - This character when placed following the application name (including a space
between the application and ampersand) will run the application in the background.

Common commands
There are some commands that are absolutely necessary to know (or at least understand) when using the Linux
command line. The following is a large number of these commands to include those that are most useful in this
training along with some general need-to-know commands:
● Getting around - The first steps of learning Linux is learning how to get around. The following is a recap
(or full description) of these commands:

Appendix B-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

○ ls - Under Linux, the command "dir" is an alias to ls and lists the contents of either the current
directory (when executed without any parameters) or the contents identified by the parameters entered.
The following is a feature of the ls command that is useful to know:
■ ls -al|more - This can be used when the output flows off of the screen. The -al parameter
identifies to list all the files (including hidden files). The |more command pauses the display
after each screenful.
○ cd - This command is an acronym for "change directory". When followed by a directory name, that
directory will become the active working directory. cd when used by itself is equivalent to cd ~

Oracle University and BOS-it GmbH & Co.KG use only

which changes the working directory to the users home directory.
○ pwd - This command is an acronym for "print working directory" and displays the current working
directory.
● File management - The process of managing directories and files (copying, moving, renaming, etc.) is a
crucial skill to understand in any operating system. The following is a listing of the most common Linux
commands associated with managing files:
○ cp [source] [destination] - This command when used with parameters for the source file and the
destination location will copy the source file to the destination location. For example cp
/usr/local/mysql/support-files/my-large.cnf /etc/my.cnf would not only
copy the file my-large.cnf to the /etc directory but also rename it to my.cnf. As stated earlier, to
. (dot) character can be used as the destination to tell the operating system to copy the file to the
current location.
■ Recursive copy (-r) - When the cp command is executed with the -r parameter, the contents of
the whole directory will be copied to the destination. For example, cp -r ~
/backup/username/. will copy all files (and directories) from the current users home
directory to the /backup/username directory.
○ mv [source] [destination] - This command is similar to the cp command, except it will remove the file
from the source location and place it in the new destination. This command can also be used to rename
a file in the same directory by identifying the original name in the source and the new name in the
destination.
○ ln [source] [destination] - This command when used with parameters for the source file and the
destination location will create a link (identified by the destination parameter) to the source parameter.
There are two types of links that can created:
■ Hard link - This type of link appears as a copy of the original files, but in reality only one copy of
the file is kept, just two (or more) directory entries point to it. Any changes made to the file are
automatically visible throughout. When one directory entry is removed, the other(s) stay(s) intact.
Files that are hard linked have to be on the same filesystem. In addition, hard links to directories
or special files are impossible.
■ Soft links - Adding a -s prior to the source file name creates a symbolic (soft) link. The symbolic
link just specifies a path where to look for the file. In contrast to hard links, the source and
destination don't not have to be on the same filesystem. If the original file is removed, the link is
"broken". In addition, symbolic links can also create circular references (like circular references
in spreadsheets or databases, e.g., "a" points to "b" and "b" points back to "a").

Appendix B-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

○ rm filename - This command will delete the file that is identified by the filename parameter. The user
attempting to delete the file(s) must also be the owners of those files (with the exception of
superusers). Most versions of the Linux operating systems will prompt the user verifying that they
wish to delete the files. However, the user can force the deletion without this prompt by adding the -f
option.
■ Deleting large number of files - Using the rm command with wildcards can delete large number
of files in a single command (eg. rm -f * will remove all files in the current working directory,
no questions asked).

Oracle University and BOS-it GmbH & Co.KG use only

■ Removing directories (rmdir) - Using the rmdir command, followed by the directory name,
will delete empty directories.
■ Removing directories and files - Using the -r option with the rm command will perform a
recursive delete which will remove files, directories and their subdirectories. This of course can
be a very useful command but can also be a very dangerous command (especially a user who has
super privileges). The following is an example of how this can be used:

rm -rf /usr/local/apache

In this example, the /usr/local/apache directory (along with all the files and subdirectories it
contains) will be removed from the system. The -f option is almost a necessity, otherwise, the end
user would have to verify each deletion that takes place. There is no undo for this action.
○ mkdir directoryname - This command will create a subdirectory (identified by the directoryname
parameter) in the current directory (unless an absolute path is used). This command will produce an
error if the user creating the subdirectory does not have the required privilege or the directory name is
already being used in the directory (either the working directory or the absolute path directory, if used).
● Viewing and editing files - The following is a listing of the most common Linux commands associated
with viewing and editing files:
○ cat filename |more- This command will display the contents of the filename to the screen. If the file is
a text-based file, then that which is displayed will most likely be readable. However, if the file is an
application or other binary type file, the display will be unreadable and most likely cause havoc with
the operating systems attempting to display it. The contents will be displayed with no pause unless the
parameters |more are used. With the |more combination, the display is paused after every screenful is
displayed. Note: Use the command reset after displaying a binary file to ensure that the character
set displays the proper default character set. Other commands that are similar to the cat command:
■ more filename - This command, which has been used as a parameter with other commands, can
be used to scroll through the content of a file.
■ less filename - This command is roughly equivalent to more but can be more convenient to use.
Press the <q> key to terminate the command.
○ vi filename - This command provides simple and standard text editing capabilities. There is a separate
appendix that describes how to use the vi text editor. There are other text editors to choose from. The
following is a list of the most common:
■ emacs filename - This command utilizes the emacs text editor to display and manipulate text files.
■ kwrite filename - This command utilizes the kwrite text editor to display and manipulate text
files.
■ nedit filename - This command utilizes the nedit text editor to display and manipulate text files.

Appendix B-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

■ vim filename - This command utilizes the vim text editor to display and manipulate text files.
○ <Middle mouse button> - This hardware interface tells Linux to paste the text which is currently
highlighted somewhere else. This is the normal "copy-paste" operation in Linux.
○ touch filename - This command will alter the date/time stamp of the file filename to the current time
for files that exist. If a file does not exist with the identified filename, the operating system will create
an empty file with that name.
● Finding files - The following commands provides the means in which the operating system searches for
files versus the end user having to search each directory individually for a file or files:

Oracle University and BOS-it GmbH & Co.KG use only

○ find / -name "filename" - This command is used to find the file called "filename" on the filesystem
starting the search from the root directory "/". The "filename" may contain wildcards (*,?).
○ locate filename - This command is used to find the file name of which contains the string "filename".
This command is easier and faster than the find command but depends on a database that normally
rebuilds at night.

Getting help
With the large number of commands offered by the Linux operating system, it is difficult to remember the syntax
and/or parameters to perform the command action. This is where having a built-in manual for the majority of
commands is helpful. The following are the most common ways to access the help features of Linux:
● commandname --help |more - These parameters used with most commands will display a brief help on a
command. "--help" works similar to DOS "/h" switch. The "more" pipe is needed if the output is longer
than one screen. help commandname is the equivalent to using the --help parameter.
● man commandname - The man command, followed by the commandname, displays the contents of the
system manual pages (help) on the topic. Press "q" to quit the viewer. The command info
commandname works similar to the man command but may contain more up-to-date information. For the
majority of commands, the manual pages are hard to read and understand, it is advisable to use the
commandname --help parameter first. If that does not answer the question being asked, then look in
the /usr/doc directory for the specific command verbage file. To display a manual page from a specific
section, add a section number between the man command and the commandname queried (eg. man 3
exit displays an info on the command exit from section 3 of the manual pages).
● apropos commandname - This command will provide a list of commands associated with the
commandname entered.

Shutting Down
Terminating the operating system is similar to Microsoft Windows when using the Linux GUI; however, to terminate
the application from the command line the following commands can be used:
● exit - This command logs the user out of the current shell that is running. If there is only one Linux shell
running, the operating system will be shutdown.
● shutdown -h now - This command, when used by the root user, will shut down the system to a halt.
● halt - This command, when used by the root user, will halt the machine. This command can be very useful
on remote shutdown.
● reboot - This command, when used by the root user, will reboot the machine. This command can be very
useful when managing a remote server.

Appendix B-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● reset - This command can be used to rescue a window that has been frozen or messed up (usually trying to
display a binary file with a command such as cat). The end user may not be able to see the typing of the
command; however, continuing to type the command followed by the enter key should produce the desired
result of reseting the operating system.
B.3 Obtaining system information
Collecting and reviewing system information can be an important aspect for system administrators and end users
alike. System information can include everything from determining the name of the computer that is being used to
determining if the computer can connect to the internet or to another machine on the network. The following are the

Oracle University and BOS-it GmbH & Co.KG use only

most common commands that can be used to obtain system information:
● hostname - This command displays the name of the local host (the machine that the end user is connecting
to). To alter the name of the local host, a user with root privileges can use the netconf command.
● ifconfig - This command (for root users) displays information about the network connection that the local
host is using. The ethernet address for the local host will show up under eth0. Non-root users can use
/sbin/ifconfig to display this information.
● ping ipaddress - This command sends an echo request (via packets) to the ipaddress (or computer name)
identified looking for a response. Ping will provide not only information determining if the computer can
be seen by the local host but alos how long the response took.
● time commandname - This command, when used with another command, will display how long the other
command took to complete. This command should not be confused with the date command (which
provides the current date and time according to the local host being accessed).
● ps - This command displays the current processes that the end user is running. The command ps axu |
more displays additional information to includes the name of the user that owns each process. Many users
use this command to find the process id (PID) of an application they wish to terminate. Once the end user
has obtained the PID of an application to terminate, the following commands can be used:
○ kill PID - This command shuts down the application associated with the PID. If an application does
not terminate with this command, the end user can append the kill command with a -9 argument. This
argument tells the system to kill the process (no matter what) without determining if it will affect any
other applications running. This argument should be used only as a last resort.
○ killall program_name - This command is similar to the kill PID command, except it uses the
program name as an argument. This of course could cause problems if the end user is running two
applications with the same name. The kill PID command is the better choice of the two.
● top - This command displays the current running processes, sorted by cpu usage (top users first). This
command continues to run until the end user terminates the command.
● history | more - This command will display the last 1000 commands (or approximately 1000 commands)
that have been issued by the end user.
● uptime - This command displays how long the local host has been running since the last reboot.
● uname -a - This command provides basic information currently available from the system.
● free - This command provides information about unused and used memory and swap space on the local
host.
● df -h - This command will display the amount of disk space (including free disk space) for the file systems
on the local host. The -h argument tell the df command to display the information in "human readable"
form.

Appendix B-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● du / -bh | more - This command displays detailed disk usage (thus the command name being du) for each
subdirectory starting at the file system root (/). The -bh works together to display the information in
"human readable" form.
● env - This command displays information about the current user environment.
● echo $PATH - This command displays a specific component of the current user environment called PATH
(which displays the directories whose command can be executed without having to be in that specific
directory).
B.4 Accessing a system remotely

Oracle University and BOS-it GmbH & Co.KG use only

There are times when it is necessary to access another physical system remotely from the system that the end user is
currently using. The following commands are the most common used to accomplish remote access:
● ssh -1 username IPaddress - This command provides a secure shell login to a remote machine. The -l
parameter specifies the user to log in as on the remote machine. The following is an example of Joe Smith
(username: jsmith) logging into a remote machine securely located at the IP address of 192.168.0.1:

ssh -l jsmith 192.168.0.1

● scp source_filename username@IPaddress:destination_filename - This command (which is short for secure

copy protocol) provides the means to copy files from the localhost to a remote machine. In addition, the
end user can copy files from one remote machine to another remote machine by providing the required
information in the source_filename parameter. The following is an example of Joe Smith (username:
jsmith) copying the file My_file.txt from his local host to a remote machine securely located at the IP
address of 192.168.0.1:

scp My_file.txt [email protected]:My_file.txt

B.5 File compression/decompression

Many files can be reduced in file size by compressing their contents. In addition, many file compression activities
include combining multiple files (especially their directory structure) into one file for easily passing along to others
or simply moving contents from one machine to another. The following commands are the most common
commands used in Linux to handle the compression and decompression of files:
● tar -cvf file_name.tar filename(s)/directory - This command will bundle a bunch of files together and
creates an archive file (tar = tape archive) in the location it was run (or identified by) in. This command
does not provide compression, but simply puts multiple files together into one file name. The -cvf
parameters tell the program to create (c) a new archive, display a verbose (v) output to the end user with a
user defined file name (f) provided. The original files remain intact and the new archive file is created. The
following is an example of creating a compressed tar file (or affectionately called a "tar ball" by most Linux
users) that creates a single archive file called My_Documents.tar of all files and subdirectories of the
My_Documents files in the end users home directory:

tar -cvf My_Documents.tar ~/My_Documents/

Appendix B-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● gzip file_name.tar - This command is used to compress files (in most cases files that have been tar'd) using
the Lempel-Ziv coding (LZ77) method. The original file is replaced with a new file with the same file
name but an extended identifier that includes the characters (.gz) creating a file that is
file_name.tar.gz. Sometimes these files will be shortened even further by other users to have an
extension of only .tgz.
● tar -zxvf file_name.tar.gz - This command will separate and decompress files that have been tar'd and
zipped using the tar and gzip commands. The -zxvf parameters tell the program to compress or
decompress files automatically (z), extract (x) the contents of the archive, display a verbose (v) output to

Oracle University and BOS-it GmbH & Co.KG use only

the end user with a user defined file name (f) provided. The following is an example of decompressing a
compressed tar file that extracts the My_Documents directory to the root directory (~/)of the end user:

~/tar -zxvf My_Documents.tar.gz

● gunzip file_name.tar.gz - This command decompresses files that have been compressed with gzip. This
command is useful with files that are tar'd and zipped if the end user only wants the archive file without
having it untar'd in the process.
● bzip2 file_name - (bzip = big zip) This command compresses a file (identified by the file_name parameter)
using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. This type of
compression is generally considered better than that achieved by the gunzip command which uses the
Lempel-Ziv coding (LZ77) method. The original file is deleted and a new compressed file is created with
the original name of the file but a new extension to identify it is zipped with bzip (*.bz2). The following
are two examples of compressing files with bzip2. The first is a standard bzip action, while the second
show hows to compress mutliple files into one bzipped file:

bzip2 My_file.txt
bzip2 My_file.txt His_file.txt Her_file.txt > Our_files.bz2

● bunzip2 file_name.bz2 - This command decompresses files that have been compressed with bzip2. The
Bzip compression tools are very useful for very large files.
● unzip file_name.zip - This command decompresses files that have been compressed with the PKZIP
command for the Microsoft disk operating system (DOS).
B.6 User management
Adding, deleting and otherwise managing access for new and existing users is an important aspect for system
administrators and for most users these commands are not going to be required for their day-to-day tasks. However,
if these tasks are required for use in the classroom or it will be part of the tasks required of the end user, it is
important to understand them. The following commands are the most common commands used in Linux to handle
user management:
● adduser username - This command creates a user (identified by the username parameter) on the local host
(or the remote host if being utilized). The useradd username command is equivalent to the adduser
command.
● groupadd groupname - This command creates a new group (identified by the groupname parameter). A
group can exist without having any members; however, it is not good user management to have groups
without any members.

Appendix B-10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● passwd username password- This command changes the password (or adds a password for a new user)
for the user identified in the username parameter. If the username parameter is ommited (basically
there is only one parameter), the passwd program will change the current users password to the parameter
entry.
● chmod permissions file_name - This command changes the file access permission for the files that the end
user has ownership rights to (or all files if end user is root). File permissions are set using three modes:
read (r), write (w) and execute (e) which are assigned by the class of users (three classes in all: owner,
group and all). The following is an example of a file that has been given access to read, write and execute
files for all three user groups: rwxrwxrwx. The permissions can be set using the format shown or using a

Oracle University and BOS-it GmbH & Co.KG use only

binary equivalent. For example, to give the owner read, write and execute access while giving the group
and all others only read and execute access the end user could enter rwxr-xr-x or 755 (which is
equivalent to 111-101-101 in binary notation).
● chown owner_name file_name - This command changes (or adds) an owner (identified by the
owner_name parameter) to a file (identified by the file_name parameter). This command should be
run after a file has been copied from another system.
● chgrp group_name file_name - This command changes (or adds) a group (identified by the
group_name parameter) to a file (identified by the file_name parameter). This command should be
run after a file has been copied from another system.
● su username - (su = substitute user) This command allows the end user to log in as another user (such as
root). The end user will be required to enter the password for the username that they wish to log into
(except for root or another superuser account). Type exit to return the previous login. It is not
advisable to habitually work on a machine as root. The root account is for administration purposes only
and the su command is to ease access to the administration account when required.
B.7 Using the RPM package management utility
RPM (Red Hat's Package Manager) is a powerful software manager that can install, remove, query, and verify the
software on a system. It is necessary to be the root user to successfully execute the RPM application. The following
are some of the common commands associated with using this utility:
● rpm -ivh filename.rpm - This command installs a RPM package(s) and then provide a detailed output on
what was accomplished. The -ivh parameters identify that the package should be installed (-i), provide a
verbose (v) output and display hashes (h).
● rpm -qpi filename.rpm - This command reads the information on the content of the RPM file (identified
by filename.rpm) without installing the actual RPM and provides information about the package. The
-qpi parameters identify that a query (-q) will be performed against a yet to be installed package (p) along
with a display of the information (i) that the package contains. The difference between the i parameter
(identifying the display of information) used here and the i parameter used in the previous command
(identifying that the RPM should be installed) is where the i is located in the list. If the i comes after the
dash (-i), it tell the RPM application to install the package. If the i is not the first character after the dash, it
tells the RPM package to display the information.
● rpm -qpl filename.rpm - This command reads the information on the content of the RPM file (identified
by filename.rpm) without installing the actual RPM and lists the files that the package contains. The -qpl
parameters identify that a query (-q) will be performed against a yet to be installed package (p) along with
a listing (1) of the files the package contains.

Appendix B-11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix B - Introduction to Linux

● rpm -qf filename - This command displays the name of the *.rpm package to which the file filename (on
belongs. This is a useful command if the original RPM package is unknown and there is a need to reinstall
the application.
● rpm -e packagename - This command uninstalls the RPM package (identified by the packagename
parameter). The packagename is the same name as the original RPM package installed but without the
dash and version number (or the .rpm extension).

Oracle University and BOS-it GmbH & Co.KG use only

Appendix B-12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX C BASIC VI COMMANDS

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix C - Basic vi Commands

C.1 What is vi?

The default editor that comes with the UNIX operating system is called vi (visual editor). [Alternate editors for
UNIX environments include pico and emacs, a product of GNU.]
The UNIX vi editor is a full screen editor and has two modes of operation:
● Command mode commands which cause action to be taken on the file.
● Insert mode in which entered text is inserted into the file.

Oracle University and BOS-it GmbH & Co.KG use only

In the command mode, every character typed is a command that does something to the text file being edited; a
character typed in the command mode may even cause the vi editor to enter the insert mode. In the insert mode,
every character typed is added to the text in the file; pressing the <Esc> (Escape) key turns off the Insert mode.
While there are a number of vi commands, just a handful of these is usually sufficient for beginning vi users. To
assist such users, this Web page contains a sampling of basic vi commands. The most basic and useful commands
are marked with an asterisk (* or star) in the tables below. With practice, these commands should become automatic.
NOTE: Both UNIX and vi are case-sensitive. Be sure not to use a capital letter in place of a lowercase letter;
the results will not be what you expect.

C.1.1 Start vi
To use vi on a file, type in vi filename. If the file named filename exists, then the first page (or screen) of
the file will be displayed; if the file does not exist, then an empty file and screen are created into which you may
enter text.

vi filename edit filename starting at line 1

vi -r filename recover filename that was being edited when system crashed

C.1.2 Stop vi
Usually the new or modified file is saved when you leave vi. However, it is also possible to quit vi without saving
the file.
NOTE: The cursor moves to bottom of screen whenever a colon (:) is typed. This type of command is
completed by hitting the <Return> (or <Enter>) key.

:x<Return> quit vi, writing out modified file to file named in original invocation

:wq<Return> quit vi, writing out modified file to file named in original invocation

:q<Return> quit (or exit) vi

:q!<Return> quit vi even though latest changes have not been saved for this vi call

Appendix C-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix C - Basic vi Commands

C.2 Moving the Cursor

Unlike many of the PC and MacIntosh editors, the mouse does not move the cursor within the vi editor screen (or
window). You must use the key commands listed below. On some UNIX platforms, the arrow keys may be used as
well; however, since vi was designed with the Qwerty keyboard (containing no arrow keys) in mind, the arrow keys
sometimes produce strange effects in vi and should be avoided.
If you go back and forth between a PC environment and a UNIX environment, you may find that this dissimilarity in
methods for cursor movement is the most frustrating difference between the two.

Oracle University and BOS-it GmbH & Co.KG use only

In the table below, the symbol ^ before a letter means that the <Ctrl> key should be held down while the letter key
is pressed.

j or <Return> [or down-arrow] move cursor down one line

k [or up-arrow] move cursor up one line

h or <Backspace> [or left-arrow] move cursor left one character

l or <Space> [or right-arrow] move cursor right one character

0 (zero) move cursor to start of current line (the one with the cursor)

$ move cursor to end of current line

w move cursor to beginning of next word

b move cursor back to beginning of preceding word

:0<Return> or 1G move cursor to first line in file

:n<Return> or nG move cursor to line n

:$<Return> or G move cursor to last line in file

Appendix C-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix C - Basic vi Commands

C.3 Screen Manipulation

The following commands allow the vi editor screen (or window) to move up or down several lines and to be
refreshed.

^f move forward one screen

^b move backward one screen

Oracle University and BOS-it GmbH & Co.KG use only

^d move down (forward) one half screen

^u move up (back) one half screen

^l redraws the screen

^r redraws the screen, removing deleted lines

C.4 Adding, Changing, and Deleting Text

Unlike PC editors, you cannot replace or delete text by highlighting it with the mouse. Instead use the commands in
the following tables.
Perhaps the most important command is the one that allows you to back up and undo your last action. Unfortunately,
this command acts like a toggle, undoing and redoing your most recent action. You cannot go back more than one
step.

u UNDO WHATEVER YOU JUST DID; a simple toggle

The main purpose of an editor is to create, add, or modify text for a file.

C.4.1 Inserting or Adding Text

The following commands allow you to insert and add text. Each of these commands puts the vi editor into insert
mode; thus, the <Esc> key must be pressed to terminate the entry of text and to put the vi editor back into command
mode.

i insert text before cursor, until <Esc> hit

I insert text at beginning of current line, until <Esc> hit

a append text after cursor, until <Esc> hit

A append text to end of current line, until <Esc> hit

o open and put text in a new line below current line, until <Esc> hit

O open and put text in a new line above current line, until <Esc> hit

C.4.2 Changing Text

The following commands allow you to modify text.

Appendix C-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix C - Basic vi Commands

r replace single character under cursor (no <Esc> needed)

R replace characters, starting with current cursor position, until <Esc> hit

cw change the current word with new text, starting with the character under cursor, until <Esc> hit

cNw change N words starting from the cursor, until <Esc> hit; e.g., c5w changes 5 words

C change (replace) the characters in the current line, until <Esc> hit

Oracle University and BOS-it GmbH & Co.KG use only

cc change (replace) the entire current line, stopping when <Esc> is hit

Ncc or cNc change (replace) the next N lines, starting with the current line, stopping when <Esc> is hit

C.4.3 Deleting Text

The following commands allow you to delete text.
x delete single character under cursor.

Nx delete N characters, starting with character under cursor

dw delete the single word beginning with character under cursor

dNw delete N words beginning with character under cursor; e.g., d5w deletes 5 words

D delete the remainder of the line, starting with current cursor position

dd delete entire current line

Ndd or dNd delete N lines, beginning with the current line; e.g., 5dd deletes 5 lines

C.4.4 Cutting and Pasting Text

The following commands allow you to copy and paste text.

yy copy (yank, cut) the current line into the buffer

Nyy or yNy copy (yank, cut) the next N lines, including the current line, into the buffer

p put (paste) the line(s) in the buffer into the text after the current line

C.5 Other Commands

There are many other commands available. The following are some of the most commonly useful.

Appendix C-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix C - Basic vi Commands

C.5.1 Searching Text

A common occurrence in text editing is to replace one word or phase by another. To locate instances of particular
sets of characters (or strings), use the following commands.

/string search forward for occurrence of string in text

?string search backward for occurrence of string in text

Oracle University and BOS-it GmbH & Co.KG use only

n move to next occurrence of search string

N move to next occurrence of search string in opposite direction

C.5.2 Determining Line Numbers

Being able to determine the line number of the current line or the total number of lines in the file being edited is
sometimes useful. The following display their results at the bottom of the screen:

:.= returns line number of current line

:= returns the total number of lines

^g returns both current line number and total lines

C.5.3 Saving and Reading Files

These commands permit you to input and output files other than the named file with which you are currently
working.

:r filename<Return> read file named filename and insert after current line

:w<Return> write (save) the current open file

:w newfile<Return> write entire content to a new file named newfile

:12,35w smallfile<Return> write lines 12 through 35 to a new file named smallfile

:w! prevfile<Return> write entire content over a pre-existing file named prevfile

Appendix C-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX D QUIZ ANSWERS

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.1 Chapter 5 - Indexing

D.1.1 Chapter 5.2 - 5.4 (Located at end of Chapter 5.4)

21. What are the differences between B-Trees and B+Trees?
A B-tree uses the key values as nodes in the tree; each node has a pointer to the data. A B+tree uses a
tree structure, however the leaves of the tree store the data (not pointers to the data).
22. What are the differences between tree indexes and hash indexes?

Oracle University and BOS-it GmbH & Co.KG use only

Tree indexes can find values by range. Hash indexes can only find exact matches.
23. What are prefix indexes? What is their purpose?
Prefix indexes can be used to index part of a column. This takes less space and often does not greatly hurt
the selectivity of the index.
24. How do FULLTEXT indexes differ from other indexes?
FULLTEXT indexes use a tree structure, however, they index individual words instead of entire columns.
Also, they must be explicitly used in queries via the MATCH()/AGAINST() functions.
25. What are covering indexes? For which Storage Engines can they be useful?
If all parts of a query are in a single index, then MySQL can execute the entire query by using the index
instead of reading the data. All storage engines can use covering indexes

Appendix D-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.2 Chapter 6 - MySQL Statement Tuning

D.2.1 Chapter 6.2 - 6.3 (Located at end of Chapter 6.3.4)

Term Definition

_C__ const A. Refers to a non-unique key that may not have

NULL values

Oracle University and BOS-it GmbH & Co.KG use only

_B__ eq_ref B. Refers to an unique key

_A__ ref C. The “fastest” explain type

_E__ ref_or_null D. Identifies the outer query uses a non-unique key

to match against values returned from the nested query

_F__ unique_subquery E. A non-unique key that may have NULL values

_D__ index_subquery F. Identifies the outer query uses a unique index to

match against values returned from the nested query

Appendix D-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.3 Chapter 8 - MySQL Server Configuration

D.3.1 Chapter 8.2 - 8.6 (Located at end of Chapter 8.6)

1. How can you monitor if there are joins not using indexes being executed on the server?
SHOW GLOBAL STATUS LIKE ‘select_full_join’;
2. How can you monitor if you might be having problems with lock contention on the server?

Oracle University and BOS-it GmbH & Co.KG use only

SHOW GLOBAL STATUS LIKE ‘table_locks%’;
3. 3.Which handler_* status variables show efficient use of indexes and which show inefficient use (or
no use)?
handler_read_key is good; handler_read_first could be bad
handler_read_rnd is usually bad; handler_read_rnd_next is bad
4. Do all the handler_* status variable counters correspond to the number of executed queries? If not,
which of them do?
Some represent individual key values or rows that are read. These handler variables represent a number
of queries: handler_delete, handler_update, handler_write

Appendix D-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.4 Chapter 9 - MyISAM Storage Engine

D.4.1 Chapter 9.2 - 9.5 (Located at end of Chapter 9.5)

1. What is the disk footprint of MyISAM tables? What are the purposes of the different files?
MyISAM tables each have a:
• MYD - this file contains the actual table row data.

Oracle University and BOS-it GmbH & Co.KG use only

• MYI - this file contains the indexing data.
• frm - this file contains all the meta information about the MyISAM table definition
2. What uses are best suited to MyISAM?
Logging, read-only data, reporting, bulk loading, data warehouses, or in applications where data
integrity is not critical.
3. What types of indexes are supported by MyISAM?
B-tree, fulltext indexes, R-tree
4. What is the difference between the different MyISAM row formats? What are their respective
advantages? Do they have implications to the index file?
• Fixed – speed, takes more space
• Dynamic – less space required, slower
• Compressed – less space, read-only; indexes must be re-enabled with myisampack -rq
5. Why are MyISAM indexes organized into blocks?
This helps reduce disk I/O because keys are often read and written in sequence.
6. How many key values are in each index block?
It depends on the size of the keys

D.4.2 Chapter 9.6 - 9.10 (Located at end of Chapter 9.10.1)

1. Describe the different concurrent insert modes in MyISAM
Off, Default, Always use concurrent insert even if there are holes in the table
2. How many priority queues are there for MyISAM tables? How can you change the priority of a query?
SELECT HIGH_PRIORITY, INSERT/UPDATE/DELETE, SELECT, INSERT/UPDATE/DELETE
LOW_PRIORITY
3. What is stored in the MyISAM key caches?
Index blocks
4. How can MyISAM secondary key caches be monitored?
SELECT @@cache_name.key_buffer_size;
5. What strategy does MyISAM use for evicting blocks from the key caches by default? How can this
behavior be changed?

Appendix D-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

Least-recently-used keys are removed first, however, you can choose the size for “cold”, “hot” and
“warm” caches
6. How can MyISAM tables be optimized for bulk inserts?
Increase the bulk_insert_buffer_size variable, DISABLE KEYS/ENABLE KEYS, OPTIMIZE TABLE
before the insert.
7. How can you optimize MyISAM table reparation and ALTER TABLE operations?
Increase myisam_sort_buffer_size and myisam_max_sort_file_size

Oracle University and BOS-it GmbH & Co.KG use only

8. What can you do with the MERGE storage engine?
Split data into different MyISAM tables.
9. What advantages does the MERGE storage engine provide?
This allows you to bypass file size limits of a large MyISAM table. MERGE tables can be easier for
deleting or purging old data (DROP TABLE). Read-only parts of the MERGE can be compressed with
myisampack. Table maintenance commands would be faster on the smaller MyISAM tables.

Appendix D-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.5 Chapter 11 - InnoDB Storage Engine

D.5.1 Chapter 10.2 - 10.10 (Located at end of Chapter 10.10)

1. What are the possible disk footprints for InnoDB tables?
ibdata, iblogfile, .frm files for each table (if you use –innodb-file-per-table, then each table will have its
own tablespace file)
2. What types of locking does InnoDB support?

Oracle University and BOS-it GmbH & Co.KG use only

Innodb supports both row-level locking and table-level locking granularity; You can use implicit and
explicit locks
3. In which isolation levels can InnoDB use next-key locking?
repeatable read and serializable
4. What is cached in the InnoDB buffer pool?
Innodb data and indexes
5. Why should log_buffer be large?
The log_buffer does not need to be very large. It keeps track of uncommitted changes
6. What are the different modes for innodb_flush_log_at_trx_commit?
0 – InnoDB will only write committed transactions to the logs once per second
1 – This is safest; InnoDB will write and fsync after every commit
2 – InnoDB will write to the log file after every transaction, however only fsyncs once per second
7. How does InnoDB handle the isolation level REPEATABLE-READ?
InnoDB uses MVCC which keeps different versions of data
8. Why doesn’t InnoDB have the same kind of concurrent_inserts option as MyISAM?
InnoDB always allows INSERTs because of the row-level locking. MyISAM uses table-level locks, so the
concurrent_insert option is very useful for concurrency.
9. How are UPDATE statements handled internally in InnoDB?
On an update, InnoDB adds a new row record to the table rather than modifying the existing data row.
The create version ID for this new record is the system version ID. For the old row data record, InnoDB
places a delete version ID (based on the system version ID) on the old record, thus maintaining
concurrency control.
10. Why is the size of primary keys particularly important in InnoDB?
Long Primary keys waste space in InnoDB tables. The Primary Key is also included in any secondary
indexes. Use short, integer primary key by adding PRIMARY KEY AUTO_INCREMENT column and
change the current PRIMARY KEY to UNIQUE.
11. What is the purpose of the InnoDB double-write buffer?
InnoDB inserts transactional statements as log records into the log buffer, while simultaneously executing
the modifications those statements make against the in-memory copy of the record data available in the
buffer pool. This dual-buffer write explains the doublewrite buffer terminology.

Appendix D-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix D - Quiz Answers

D.6 Chapter 11 - Alternate Storage Engines

D.6.1 Chapter 11.2 - 11.9 (Located at end of Chapter 11.9)

Storage Engine Type Feature

_E_H____ ARCHIVE A. Does not support ALTER TABLE, DROP

TABLE, or any other Data Definition Language
statements

Oracle University and BOS-it GmbH & Co.KG use only

_K_M____ BLACKHOLE B. Can not store BLOB or TEXT type fields

_C_L____ CLUSTER C. Implements a distributed, highly fault tolerant

architecture with no single point of failure.

_I______ CSV D. Stores all data in memory; once the MySQL

server has been shut down any information stored in it
will have been lost.

_A_F_J__ FEDERATED E. Used for storing large amounts of data without

indexes in a very small footprint.

_B_D_G__ MEMORY F. Accesses data in tables of remote databases

rather than in local tables.
G. max_heap_table_size sets the maximum
size to which the tables are allowed to grow.
H. On retrieval, rows are uncompressed on
demand; there is no row cache.
I. When data is stored into the table, the storage
engine saves it into the data file in comma-separated
values format.
J. Only an .frm file is created on the local server
when using this storage engine type.
K. Inserts do not store any data, but if the binary
log is enabled, the SQL statements are logged (and
replicated to slave servers).
L. Offers a flexible distributed database
management system that consists of three kinds of nodes
which replicate data, access the database, and manage
the system.
M. Can also be useful as a repeater or filter
mechanism.

Appendix D-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX E FURTHER PRACTICE SOLUTIONS

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.1 Chapter 2 – Architecture

1. In the world schema create a new table called city_huge that contains the structure and all the data
from City table.

CREATE TABLE city_huge LIKE City;

INSERT INTO city_huge SELECT * FROM City;

2. Increase the number of records in the city_huge table by inserting all the records from the city_huge

Oracle University and BOS-it GmbH & Co.KG use only

table between four (4) and eight (8) times. The data in the city_huge table will grow exponentially as
you run the INSERT command against and from the city_huge table. The last two executions should
start to run slower. Depending on the speed of your hardware, adjust the number of repetitions to challenge
it.

INSERT INTO city_huge (Name, CountryCode, District, Population)

SELECT Name, CountryCode, District, Population FROM city_huge;

3. Create a copy of the city_huge table using other storage engines: MEMORY, ARCHIVE, MYISAM (if not
the default) , INNODB (if not the default). For MEMORY, you will likely need to increase the
max_heap_table_size variable to fit the data in memory.

SET max_heap_table_size = 134217728;

CREATE TABLE city_huge_memory LIKE city_huge;
ALTER TABLE city_huge_memory ENGINE=MEMORY;
INSERT INTO city_huge_memory SELECT * FROM city_huge;
CREATE TABLE city_huge_archive LIKE city_huge;
ALTER TABLE city_huge_archive ENGINE=ARCHIVE;
INSERT INTO city_huge_archive SELECT * FROM city_huge;

CREATE TABLE city_huge_myisam LIKE city_huge;

ALTER TABLE city_huge_myisam ENGINE=MYISAM;
INSERT INTO city_huge_myisam SELECT * FROM city_huge;
CREATE TABLE city_huge_innodb LIKE city_huge;
ALTER TABLE city_huge_innodb ENGINE=INNODB;
INSERT INTO city_huge_innodb SELECT * FROM city_huge;

4. Search each table created for all records where the city name is “Paris” and compare the average response
times. If the response times differ, why?

SELECT * FROM city_huge_memory WHERE Name='Paris';

SELECT * FROM city_huge_archive WHERE Name='Paris';
SELECT * FROM city_huge_myisam WHERE Name='Paris';
SELECT * FROM city_huge_innodb WHERE Name='Paris';

Appendix E-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

5. Search each table created for all records where the city identification number (ID) is equal to 123456 and
compare the average response times. If the response times differ, why?

SELECT * FROM city_huge_memory WHERE ID=123456;

SELECT * FROM city_huge_archive WHERE ID=123456;
SELECT * FROM city_huge_myisam WHERE ID=123456;
SELECT * FROM city_huge_innodb WHERE ID=123456;

6. Using the SHOW TABLE STATUS, review the Data_length and the other information associated with

Oracle University and BOS-it GmbH & Co.KG use only

the tables created in step 3.

SHOW TABLE STATUS LIKE 'city_huge_memory' \G

SHOW TABLE STATUS LIKE 'city_huge_archive'\G
SHOW TABLE STATUS LIKE 'city_huge_myisam' \G
SHOW TABLE STATUS LIKE 'city_huge_memory' \G

Appendix E-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.2 Chapter 6 – Statement Tuning

1. Create a clean version of the world database to remove any index structures that have been created.

DROP DATABASE world;

CREATE DATABASE world;
USE world;
SOURCE /home/oracle/Desktop/world.sql

Oracle University and BOS-it GmbH & Co.KG use only

2. Load the stored procedures that were deleted when creating a fresh copy of the world database:

SOURCE /labs/scripts/pt_stored_procedures.sql

3. Create the city_huge table needs to be recreated again also by issuing the following command:

CALL create_city_huge_no_index(200)

4. In an O/S terminal window, review the SQL statements that were slow. The following query was listed as
slow:

SELECT CountryCode, MAX(Population) FROM city_huge GROUP BY

CountryCode;

5. Improve the query in step 4 by including a combined index on the CountryCode and Population
columns in the city_huge table.

ALTER TABLE city_huge ADD INDEX Ccode_Pop (CountryCode,Population);

6. In the mysql client, execute the SQL statement from step 4 to see if the query execution time was
improved.

mysql> SELECT CountryCode, MAX(Population) FROM city_huge

-> GROUP BY CountryCode;

7. In the O/S terminal window, review another SQL statement that was slow. The following query was listed
as slow:

SELECT Country.Name FROM Country WHERE Code IN (SELECT CountryCode

FROM city_huge WHERE Population > 8000000);

8. Improve the query in step 7 by rewriting the query as a JOIN statement.

mysql> SELECT DISTINCT Country.Name FROM Country JOIN city_huge

-> ON (Country.Code = city_huge.CountryCode)
-> WHERE city_huge.Population > 800000;

9. In the mysql client, execute the SQL statement improved in step 8 to see if the query execution time was
improved.

Appendix E-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.3 Chapter 7 – Caching and MySQL

1. In the mysql client, execute the following SQL statement to reset the query cache statistics:

FLUSH STATUS;

2. Verify the query cache size is set to 4M. If there is a different value, set the query cache size to 4M.

SELECT @@query_cache_size;

Oracle University and BOS-it GmbH & Co.KG use only

3. Execute the Qcache-queries.sql file using the following mysqlslap benchmarking tool with one iteration:

mysqlslap -uroot -q /labs/scripts/Qcache-queries.sql -i 1

-–create-schema=world

4. Execute the same O/S command in step 3 again and then examine query cache statistics.

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql -i 1

-–create-schema=world

mysql> SHOW GLOBAL STATUS LIKE 'qcache%';

5. Reset the query cache statistics.

FLUSH STATUS;

6. Set the query cache size to 16M.

SET GLOBAL query_cache_size = 16 * 1024 * 1024;

7. Execute the same O/S command in step 3 two times and then examine the query cache statistics.

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 -–create-schema=world

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 -–create-schema=world

mysql> SHOW GLOBAL STATUS LIKE 'qcache%';

8. Reset the query cache statistics.

FLUSH STATUS;

9. Set the query cache size to 32M.

SET GLOBAL query_cache_size = 32 * 1024 * 1024;

Appendix E-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

10. Execute the same O/S command in step 3 two times and then examine the query cache statistics.

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 -–create-schema=world

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 –-create-schema=world

mysql> SHOW GLOBAL STATUS LIKE 'qcache%';

Oracle University and BOS-it GmbH & Co.KG use only

11. Reset the query cache statistics.

FLUSH STATUS;

12. Set the query cache size to 64M.

SET GLOBAL query_cache_size = 64 * 1024 * 1024;

13. Execute the same O/S command in step 3 two times and then examine the query cache statistics.

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 -–create-schema=world

shell> time mysqlslap -uroot -q /labs/scripts/qcache-queries.sql

-i 1 -–create-schema=world

mysql> SHOW GLOBAL STATUS LIKE 'qcache%';

14. Reset the query cache statistics and then turn off query caching.

FLUSH STATUS;
SET GLOBAL query_cache_size = 0;

15. Of the different query cache sizes (4M, 16M, 32M or 64M), which size seems the best? Why do you think
this is?
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
16. Execute the following query in the mysql client and record the time it took to execute:

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,

Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

Appendix E-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

17. Execute the query from step 16 to obtain the next 5 records (6-10) and record the time it took to execute.

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,

Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

18. Execute the query from step 16 to obtain the next 5 records (11-15) and record the time it took to execute.

Oracle University and BOS-it GmbH & Co.KG use only

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,
Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

19. Execute the query from step 16 to obtain the next 5 records (16-20) and record the time it took to execute.

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,

Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

20. Execute the query from step 16 to obtain the next 5 records (21-25) and record the time it took to execute.

SELECT MAX(city_huge.Population) AS city_pop, COUNT(*) AS num,

Continent, Region FROM Country, city_huge WHERE Country.Code =
city_huge.CountryCode GROUP BY Continent, Region LIMIT 0,5;

How many seconds did the query take to execute: _____________

21. Sum up the total time of all the previous 5 queries executed. How many seconds total did it take to run all 5
queries: _______________________
22. Using the query from step 16, create a MEMORY table called foo that contains the first 25 records using
the following SQL and record how many seconds it took to execute the query:

CREATE TABLE foo ENGINE=MEMORY SELECT MAX(city_huge.Population) AS

city_pop, COUNT(*) AS num, Continent, Region FROM Country, city_huge
WHERE Country.Code = city_huge.CountryCode GROUP BY Continent, Region
LIMIT 25;

How many seconds did the query take to execute: _____________

23. Select the first 5 records from the table created in step 22 and record how many seconds it took the query to
execute.

SELECT * FROM foo LIMIT 0,5;

How many seconds did the query take to execute: _____________

Appendix E-6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

24. Select the next 5 records (6-10) from the table created in step 22 and record how many seconds it took the
query to execute.

SELECT * FROM foo LIMIT 5,10;

How many seconds did the query take to execute: _____________

25. Select the next 5 records (11-15) from the table created in step 22 and record how many seconds it took the
query to execute.

Oracle University and BOS-it GmbH & Co.KG use only

SELECT * FROM foo LIMIT 10,5;

How many seconds did the query take to execute: _____________

26. Select the next 5 records (16-20) from the table created in step 22 and record how many seconds it took the
query to execute.

SELECT * FROM foo LIMIT 15,5;

How many seconds did the query take to execute: _____________

27. Select the next 5 records (21-25) from the table created in step 22 and record how many seconds it took the
query to execute.

SELECT * FROM foo LIMIT 20,5;

How many seconds did the query take to execute: _____________

28. Sum up the total time of all the previous 5 queries executed. How many seconds total did it take to run all 5
queries: _______________________.
How often would you need to have people go to extra pages to be useful?
__________________________________________________________________________________
__________________________________________________________________________________

Appendix E-7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.4 Chapter 8- MySQL Server Configuration

E.4.1 Section 8.3

1. Create 500 tables by passing the number 500 into the single parameter of the create_many_tables()
stored procedure.

CALL create_many_tables(500);

Oracle University and BOS-it GmbH & Co.KG use only

2. View the status of the table cache by looking at the associated status variables.

SHOW GLOBAL STATUS LIKE '%open%table%';

3. Set the two table cache server connection system variables to 256 each.

SET GLOBAL table_definition_cache=256;

SET GLOBAL table_open_cache=256;

4. In the mysql client, source the qry_many_tables.sql file to execute a query against each of the
tables created.

SOURCE /labs/scripts/qry_many_tables.sql

5. Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the tables created:

mysqlslap -uroot -q /labs/scripts/qry_many_tables.sql -i 10

-–create-schema=world

6. In the mysql client, monitor the change of the table cache statistics.

SHOW GLOBAL STATUS LIKE '%open%table%';

7. Set the two table cache server connection system variables to 1024 each.

SET GLOBAL table_definition_cache=1024;

SET GLOBAL table_open_cache=1024;

8. In the mysql client, source the qry_many_tables.sql file to execute a query against each of the
tables created again.

SOURCE /labs/scripts/qry_many_tables.sql

Appendix E-8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

9. Issue the following command in an O/S terminal window (command line) to run the mysqlslap
benchmarking tool against the tables created again:

mysqlslap -uroot -q /labs/scripts/qry_many_tables.sql -i 10

-–create-schema=world

10. Compare execution time by monitoring the change of the table cache statistics in the mysql client. Is there
a noticeable difference?

Oracle University and BOS-it GmbH & Co.KG use only

11. Remove the tables created by calling the drop_many_tables() stored procedure.

CALL drop_many_tables();

E.4.2 Section 8.4

1. Display the current values of the variables associated with the tmp_table_size system variable by
issuing the following command in the mysql client:

SHOW GLOBAL STATUS LIKE 'created%tmp%tables';

2. Execute the vmstat command in the O/S terminal window to display the current memory usage recorded.

shell> vmstat

3. Issue the following mysql and mysqlslap benchmark commands in the O/S terminal window to have
a noticeable load applied against the server and give us information for monitoring any future changes to
status variables:

mysql -uroot -e "CALL world.create_city_huge(200);"

mysqlslap -uroot -q "SELECT name, SUM(population) FROM city_huge
GROUP BY name ORDER BY name DESC;" -i 3 –-create-schema=world

4. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?

SHOW GLOBAL STATUS LIKE 'created%tmp%tables';

5. Increase the size of the tmp_table_size system variable to 8 *1024.

SET GLOBAL tmp_table_size = 8 * 1024;

6. Execute the benchmark command from step 3 again.

mysqlslap -uroot -q "SELECT name, SUM(population) FROM city_huge

GROUP BY name ORDER BY name DESC;" -i 3 -–create-schema=world

Appendix E-9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

7. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?

SHOW GLOBAL STATUS LIKE 'created%tmp%tables';

8. Increase the size of the tmp_table_size system variable to 8 *1024 * 1024.

SET GLOBAL tmp_table_size = 8 * 1024 & 1024;

Oracle University and BOS-it GmbH & Co.KG use only

9. Execute the benchmark command from step 3 again.

mysqlslap -uroot -q "SELECT name, SUM(population) FROM city_huge

GROUP BY name ORDER BY name DESC;" -i 3 –-create-schema=world

10. Display the current values of the variables associated with the tmp_table_size system variable. Was
there an increase in the values? Display the current memory usage. Was there an increase, decrease or no
change in the memory values?

SHOW GLOBAL STATUS LIKE 'created%tmp%tables';

Appendix E-10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.5 Chapter 9 – MyISAM Storage Engine

E.5.1 Section 9.9

1. Populate the city_huge table with approximately 800,000 records by passing an argument of 200 to the
create_city_huge stored procedure. This might take some time to run, so now might be a good time
for a break.

CALL create_city_huge(200);

Oracle University and BOS-it GmbH & Co.KG use only

2. Reset the key cache to a 16K key cache using the following SQL statement:

SET GLOBAL key_buffer_size = 16 * 1024;

3. Execute a large query against the city_huge table that performs a full index scan using the following SQL
statement:

SELECT SUM(Population) FROM city_huge;

Record the time it took to execute the query: _____________ (Query A-1)
4. Execute a FLUSH TABLES command to flush the query cache. This will ensure that the next command
gives an accurate result.
5. Execute a large query against the city_huge table that performs a full table scan (eliminating the file
system cache effect) using the following SQL statement:

SELECT MAX(District) FROM city_huge;

Record the time it took to execute the query: _____________ (Query A-2)
6. Reset the key cache to 75% of system memory. (Below is for a system with 2GB of memory)

SET GLOBAL key_buffer_size = 1549410000;

7. Execute a FLUSH TABLES command again to flush the query cache.

8. Execute the original full index scan query (step 3) against the city_huge table again.

SELECT SUM(Population) FROM city_huge;

Record the time it took to execute the query: _____________ (Query B-1)
9. Execute a FLUSH TABLES command again to flush the query cache.
10. Execute the original full table scan query (step 5) against the city_huge table again.

SELECT MAX(District) FROM city_huge;

Record the time it took to execute the query: _____________ (Query B-2)

Appendix E-11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

11. Execute a FLUSH TABLES command again to flush the query cache.
12. Execute the original full index scan query (step 3) against the city_huge table again.

SELECT SUM(Population) FROM city_huge;

Record the time it took to execute the query: _____________ (Query C-1)
13. Compare the three different queries (A, B and C) performance against each other.

Oracle University and BOS-it GmbH & Co.KG use only

14. Reset the key cache to 50M.

SET GLOBAL key_buffer_size = 52428800;

15. Execute the mid-point.sql file using the following O/S command to populate the cache with one
iteration:

mysqlslap -uroot -i 1 -q /lab/scripts/mid-point.sql

-–create_schema=world

16. In the mysql client, execute the following SQL statement to reset the key cache statistics:

FLUSH STATUS;

17. Execute the mid-point.sql file again using mysqlslap.

mysqlslap -uroot -i 1 -q /lab/scripts/mid-point.sql

-–create_schema=world

18. In the mysql client, determine the hit rate by reviewing the key cache statistics and using the following
formula:

100 - (Key_reads/Key_read_requests) * 100

What is the hit rate percentage: ______________ (hit rate A)

SHOW GLOBAL STATUS LIKE 'Key_read%';

19. Execute a FLUSH TABLES command again to flush the query cache.
20. Execute the original full index scan query (step 3) against the city_huge table again.

SELECT SUM(Population) FROM city_huge;

21. In the mysql client, determine the hit rate of key cache again.

100 - (Key_reads/Key_read_requests) * 100

Appendix E-12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

What is the hit rate percentage: ______________ (hit rate B)

SHOW GLOBAL STATUS LIKE 'Key_read%';

22. Reset the key cache statistics.

FLUSH STATUS;

23. Execute the mid-point.sql file again using the following O/S command to populate the cache with ten

Oracle University and BOS-it GmbH & Co.KG use only

iterations:

mysqlslap -uroot -i 1 -q /lab/scripts/mid-point.sql

-–create_schema=world

24. In the mysql client, determine the hit rate of key cache again.

100 - (Key_reads/Key_read_requests) * 100

What is the hit rate percentage: ______________ (hit rate C)

SHOW GLOBAL STATUS LIKE 'Key_read%';

25. Compare hit rate A and hit rate C.

26. Set the key_cache_division_limit system variable to 50 by issuing the following SQL statement:

SET GLOBAL key_cache_division_limit = 50;

27. Reset the key cache statistics.

28. Execute the mid-point.sql file again to populate the cache with ten iterations.
29. In the mysql client, determine the hit rate of key cache again.

100 - (Key_reads/Key_read_requests) * 100

What is the hit rate percentage: ______________ (hit rate D)

SHOW GLOBAL STATUS LIKE 'Key_read%';

30. Reset the key cache statistics.

FLUSH STATUS;

31. Execute a FLUSH TABLES command again to flush the query cache.
32. Execute the original full index scan query (step 3) against the city_huge table again.

SELECT SUM(Population) FROM city_huge;

Appendix E-13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

33. In the mysql client, determine the hit rate of key cache again.

100 - (Key_reads/Key_read_requests) * 100

What is the hit rate percentage: ______________ (hit rate E)

SHOW GLOBAL STATUS LIKE 'Key_read%';

34. Compare all the hit rates to determine the best use of the key cache.

Oracle University and BOS-it GmbH & Co.KG use only

E.5.2 Section 9.10
1. In the mysql client, export the city_huge table to a flat file using the following SQL statement:

SELECT * INTO OUTFILE '/tmp/city_huge.txt' FROM city_huge;

2. In an O/S terminal, setup up monitoring for the mysqladmin processlist by entering the flowing
statement:

watch mysqladmin -uroot pro

3. In another O/S terminal, setup up monitoring of disk usage by entering the following statement:

watch -d ls -l /usr/local/mysql/data/world/city_huge*

You may have to be logged in as root to access the data directory.

4. In another O/S terminal, setup up monitoring of CPU/disk usage by entering the following statement:

vmstat 1

5. In the mysql client, set the key cache to a small number (16K).

SET GLOBAL key_buffer_size = 16 * 1024;

6. Setting the myisam_sort_buffer to anything less than 4 will produce a warning and default to 4. Use
the following command to set the myisam_sort_buffer:

SET SESSION myisam_sort_buffer_size = 4;

7. Truncate the city_huge table, and reload the records exported in step 1.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/city_huge.txt' INTO TABLE city_huge;

Watch the monitors; how long did the import take: _____________ (Import A)

Appendix E-14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

8. In the mysql client, set the key cache to a big number (75% of system memory; Below is a 2G example).

SET GLOBAL key_buffer_size = 1549410000;

9. Truncate the city_huge table, and reload the records exported in step 1.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/city_huge.txt' INTO TABLE city_huge;

Oracle University and BOS-it GmbH & Co.KG use only

Watch the monitors; how long did the import take: _____________ (Import B)
10. In the mysql client, set the key cache to a small number (16K).

SET GLOBAL key_buffer_size = 16 * 1024;

11. Set the MyISAM sort buffer to 80% of system memory (Below is a 2G example).

SET GLOBAL myisam_sort_buffer_size = 1652704000;

12. Truncate the city_huge table, and reload the records exported in step 1.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/city_huge.txt' INTO TABLE city_huge;

Watch the monitors; how long did the import take: _____________ (Import C)
13. Compare the different times for each of the imports.
14. Shut down the other terminal windows opened during this lab exercise.

Appendix E-15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

E.6 Chapter 10 – InnoDB Storage Engine

1. In the mysql client, create a smaller city_huge database to complete the steps to follow by executing
the following SQL statement:

CALL create_city_huge(25)

ALTER TABLE city_huge ENGINE=INNODB;

Oracle University and BOS-it GmbH & Co.KG use only

2. In the mysql client, export the data from the city_huge table randomly to a flat file using the following
SQL statement:

SELECT * INTO OUTFILE '/tmp/innodb_rand.txt' FROM city_huge ORDER BY

RAND();

3. Export the data from the city_huge table ordered by the primary key to a flat file using the following
SQL statement:

SELECT * INTO OUTFILE '/tmp/innodb_pk_order.txt' FROM city_huge ORDER

BY id;

4. Truncate the city_huge table and import the random text file (created in step2) back into the table.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/innodb_rand.txt' INTO TABLE city_huge;

Record how long it took approximately to load the records back in: _________________
5. Truncate the city_huge table and import the primary key text file (created in step 3) back into the table.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/innodb_pk_order.txt' INTO TABLE city_huge;

Record how long it took approximately to load the records back in: _________________
6. Shut down the mysql client by executing the following command:

QUIT;

7. Edit the /etc/my.cnf file and add the following line:

innodb_buffer_pool_size = 1652704000;

This will increase the size of the innodb_buffer_pool_size system variable to approximately
80% of the system memory (Using 2GB as an example). Close and save the file.

Appendix E-16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix E - Further Practice Solutions

8. Restart the MySQL server by issuing the following commands:

shell> /etc/init.d/mysql stop

shell> /etc/init.d/mysql start

9. Restart the mysql client by issuing the following commands:

shell> mysql -uroot

Oracle University and BOS-it GmbH & Co.KG use only

10. Start up the mysql client again and issue the following SQL statement to disable (turn off) unique checks:

SET UNIQUE_CHECKS=0;

11. Using the world database, truncate the city_huge table and import the random text file (created in step
2) back into the table.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/innodb_rand.txt' INTO TABLE city_huge;

Record how long it took approximately to load the records back in this time: _________________
12. Truncate the city_huge table and import the primary key text file (created in step 2) back into the table.

TRUNCATE city_huge;
LOAD DATA INFILE '/tmp/innodb_pk_order.txt' INTO TABLE city_huge;

Record how long it took approximately to load the records back in this time: _______________

Appendix E-17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX F SUPERSMACK

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
MySQL Performance Tuning Appendix F: Super Smack

Super Smack
Running Super Smack
Once Super Smack installed, sample benchmarks are included in the /usr/share/smack directory:
select-key.smack
update-select.smack
These two files are example configuration files that will load a table with a large number of records and execute
a numerous queries against it. The following demonstrates how a files such as these can be used with Super
Smack:

Oracle University and BOS-it GmbH & Co.KG use only

shell> super-smack update-select.smack 40 8000

This script produces the equivalent of 40 concurrent users each running 8,000 iterations of the test queries
configured in the update-select.smack configuration file. This results in an output that would look
similar to the following example:

Query Barrel Report for client smacker

connect: max=54ms min=1ms avg= 3ms from 40 clients
Query_type num_queries max_time min_time q_per_s
select_index 320000 12 0 2654.32
update_index 320000 7 0 2654.32

The first time Super Smack is run, the following lines will be displayed which shows the application preparing
the MySQL server with the required SQL components (creating the schema and table) along with the loading of
data:

Error running query select count(*) from http_auth:Table 'test.http_auth'

doesn't exist
Creating table 'http_auth'
Loading data from file '/var/smack-data/words.dat' into table 'http_auth'

Query barrel report

The query barrel report is a set of queries run by Super Smack for each iteration that has been requested by the end
user. The first line of the report (connect:) produces statistical information that provides the overall connection
information to include the maximum, minimum and average time it took for the number of clients chosen to connect
to the MySQL server. The remaining lines identify each test statistical information to include the actual number of
queries that were executed for the test, the maximum and minimum time for the queries executed and the average
number of queries that were run each second. This information provides valuable statistics to begin the tuning
process and to monitor the effects of tuning decisions against the server.

F-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
MySQL Performance Tuning Appendix F: Super Smack

Test data preparation

The words.dat file is useful for initial testing purposes; however, it is necessary for each individual to
provide their own test data to ensure that the MySQL server is being tuned to work at optimal performance for
the data that it will be responsible for. Determining what type of data needs to be stored and how that data
should be accessed is based on the application that the MySQL server will be supporting and should be as
realistic as possible to obtain a proper benchmark. There are two ways to prepare the data file that will be used
with the application. The first way, which works when there is only one column of data that will be tested, is to
use the SELECT ... INTO OUTFILE command. An example of how this may look is below:

mysql> SELECT ssan INTO OUTFILE "/var/smack-data/hr_data.dat" FROM employees;

One column of data is the most optimal data file for use with Super Smack; however, if there is a need to test

Oracle University and BOS-it GmbH & Co.KG use only

multiple columns of data a delimiter separated file with quoted data must be created. The following shows an
example of how this can be accomplished by creating a comma-separated file:

mysql> SELECT ssan, last_name INTO OUTFILE "/var/smack-data/hr_data.dat"

> FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
> LINES TERMINATED BY "\n" FROM employees;

Super Smack does allow for the end user to identify the delimiter that would be used (when reading the file) but
it is common to use comma's as the delimiter in many programming languages and applications.
Configuration file
The heart of Super Smack is the configuration file that can be used to define a series of tests (a query barrel) to
run against the server along with the data and tables needed to support those tests. When running the tests,
Super Smack gives the user the ability to control how many concurrent clients will be simulated and how many
iterations of each test the clients will execute using command-line arguments. There are four components to the
configuration file that need to be created: clients, tables, dictionaries, and queries.
• Clients - This section requires the preparation of the client that will be used. It includes the user that
will be used to connect to the server, the host of the server, the database that will be used, the password
associated with the user and the location of the mysql socket file. In addition, if the MySQL server is running
on a non-standard port, that also needs to be identified (port "{port_num}"). The following is an
example of the client section of the Super Smack configuration file:

client "world"
{
user "root";
host "localhost";
db "world";
pass "";
socket "/var/share/mysql/mysql.sock";
}

Using passwords with Super Smack

For most applications that interact with the MySQL server, the server will prompt the user for a password when one
is not provided. Super Smack will not prompt the user to supply a password if one has not been provided. This will
result in errors if a password is needed for the user identified.

F-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
MySQL Performance Tuning Appendix F: Super Smack

• Tables - This section is responsible for preparing the table structure and the data that will be contained in
the table. In this section the client to utilize will be identified, the structural components of the table will be
listed, the minimum number of rows that is needed for the test, and the source of the data that will populate the
table. If there is not enough rows of data from the data source to meet the needs of the minimum number of
rows, a program called gen-data (which is included with Super Smack) can be used to fill in the missing rows.
The following example shows what a typical table configuration section may look like:

table "city_smack"
{

client "world";

Oracle University and BOS-it GmbH & Co.KG use only

create "CREATE TABLE city_smack
(ID INT NOT NULL PRIMARY KEY,
Name CHAR(35),
CountryCode CHAR(3),
District CHAR(20),
Population INT
);

min_rows "110000";
data_file "world.dat";
gen_data_file "gen-data -n 110000 -f %n,%10-35s%,%3-3s%,%10-20s%,%d";
}

Gen-Data
The actual options of the gen-data program are not documented anywhere in the actual Super Smack application;
however, there are a few common threads that can be read from this command line. First, the number of rows that is
needed is preceded by the -n option (-n 110000). Second, the data needed to be populated is preceded by the -f
to identify that there is a need to format the data which is then followed by the actual formatting which uses a printf-
style format string (%n - incremental integer, %#-#s% identifies a string with a minimum size and a maximum size,
and %d is a random numeric integer). This command can also be run from the operating system command line to test
it prior to using it in the configuration file.

F-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
MySQL Performance Tuning Appendix F: Super Smack

• Dictionary - This section of the configuration file is used to provide Super Smack with values to use
when creating the queries for testing purposes.
o type - In this section the method used to select the data is indentified (rand - for randomly
selected values, seq - for values to be selected sequentially, or unique - for values to be generated uniquely
using gen-data).
o source_type - The second component of this section is to identify where the values will be
coming from (file - which states that a file will be used to obtain the values, list - which states a comma-
separated list will be supplied in the configuration itself, or template - which is used with the unique method
and uses the same printf-style formatting).
o source - The next component is the actual source data; this can consist of either the file that is
to be used or the list using a formatting such as ("Kai","Sarah","Max","Patricia,"Tobias").

Oracle University and BOS-it GmbH & Co.KG use only

o delim - If a delimited file is being used, this component identifies the character that should be
used to delimit the separate columns of data.
The following provides an example of what a typical dictionary section of the Super Smack configuration
file will look like:

dictionary "world"
{
type "rand";
source_type "file";
source "world.dat";
delim ",";
}

• Queries - This section defines the queries that will be used and prepares them for use in the execution of
the Super Smack program. There are three components to this section: the query itself, the name of the query
and a flag to identify if the query will return a result set or not. In addition, if the query itself is going to use
values from the dictionary, the flag telling Super Smack that the query is parsed must be turned on (parsed =
"y"). The following is an example of two queries that could be used in the operation of the Super Smack
program:

query "select_city"
{
query "SELECT * FROM city_smack WHERE CountryCode = '$word'";
type "select_ctry_code";
has_result_set = "y";
parsed = "y";
}
query "update_city"
{
query "UPDATE city_smack SET CountryCode = '$word' WHERE District= '$word'";
type "update_ctry_code";
has_result_set = "n";
parsed = "y";
}

F-4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
MySQL Performance Tuning Appendix F: Super Smack

• Super Smack client - This section is the actual client that will perform the benchmark based on the
previous values that have been set. The difference between this client and the previous client (besides the
values being different) is the query_barrel syntax that defines the order and the number of clients that will be
executed during each iteration. The query_barrel syntax includes the number of times a query should be run
preceding the actual query name, so in the case of the example below, each query is run once for each iteration:

client "smacked"
{
user "test";
host "localhost";
db "test";
pass "";

Oracle University and BOS-it GmbH & Co.KG use only

socket "/var/share/mysql/mysql.sock";
query_barrel "1 select_city 1 update_city";
}

• Main section - This section is the final section of the Super Smack configuration file and contains the
actual flow that will be used to execute the application. The syntax for this section rarely changes and contains
the reading of the command-line arguments in shell-style numbered values ($1, $2, etc.) thus making it easy to
change the order of the arguments. The following is the order in which this section follows: (1) The smacked
client is initialized (2) The number of rounds that the application will execute is read in from the command line
argument and set in the script (3) The necessary threads are read in from command line argument and set in the
script (4) Each thread is connected to the server (5) The barrel of queries are unloaded/executed with the results
being stored for statistical purposes (6) The collect_threads is responsible for ensuring that all the clients have
returned statistics before moving on to the next iteration (7) The clients are disconnected and the statistics are
compiled and reported back to the end user.

Main
{
smacked.init();
smacked.set_num_rounds($2);
smacked.create_threads($1);
smacked.connect();
smacked.unload_query_barrel();
smacked.collect_threads();
smacked.disconnect();
}

F-5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

APPENDIX G MYSQL ENTERPRISE MONITOR

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix G - MySQL Enterprise Monitor

G.1 MySQL Enterprise Monitor

The MySQL Enterprise Monitor is a web-based monitoring and advising system. The Enterprise Monitor helps
MySQL DBAs manage more MySQL servers in a scale-out environment, tune their current MySQL servers and find
and fix problems with their MySQL database applications before they can become serious problems or costly
outages. Running completely within the corporate firewall, the Enterprise Monitor pro-actively monitors enterprise
database environments and provides expert advice on how MySQL can tighten security, optimize performance and
reduce downtime of their MySQL powered systems. EM accomplishes all this while reducing DBA time and effort.
EM provides a rich GUI Enterprise Dashboard that contains “pages” which allow the DBA to have an immediate,

Oracle University and BOS-it GmbH & Co.KG use only

graphic view of administrative tasks, server and database status, and advisory information.
The principle features of Enterprise Monitor:
● Enterprise Dashboard
○ Manage all MySQL servers from a consolidated console
● Server or Server-group management
○ Auto detection, grouping and monitoring of replication and scale-out topologies
● Monitoring page
○ “At a glance” global health check of key systems
● MySQL provided Advisors and Advisor Rules
○ Enforce MySQL Best Practices
● Advisor Rule Scheduler
○ Schedule unattended operations
● Customizable Thresholds and Alerts
○ Identify Advisor Rule violations
● Custom Advisor Rule
○ User defined advisor rules
● Events and Alert history
○ Lists all Advisor Rules executions
● Specialized Scale-Out assistance

Appendix G-1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix G - MySQL Enterprise Monitor

G.1.1 Enterprise Dashboard

Graphs Page Tabs
(custom)

ED Control
/Help

Oracle University and BOS-it GmbH & Co.KG use only

Explorer
Navigation
(all serve rs)

Heat Chart
(all serve rs)
Critical Events
(per Server)

G.1.2 Enterprise Advisors

The MySQL advisors are divided into the following categories (and specific functions):

Appendix G-2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Tuning Appendix G - MySQL Enterprise Monitor

G.1.3 Enterprise Monitor Subscription Levels

A “Silver” or higher level is required to get Enterprise Monitor. Some features are only available with the “Gold”
and “Platinum” levels, respectively. A list of features per level is shown below:

Basic Silver Gold Platinum

Enterprise Dashboard

Oracle University and BOS-it GmbH & Co.KG use only

Notifications and Alerts

Custom Advisor

Upgrade Advisor

Administration Advisor

Security Advisor

Replication Monitor

Replication Advisor

Query Analyaer

Memory Usage Advisor

Schema Advisor

Performance Advisor

Appendix G-3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

<Insert Picture Here>

Better Performance, Scalability, and Recovery

Oracle University and BOS-it GmbH & Co.KG use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Server 5.5

• InnoDB default storage engine

• InnoDB performance improvements
• MySQL performance improvements
• Improved availability
• Improved usability

Oracle University and BOS-it GmbH & Co.KG use only

• Improved scalability

CONFIDENTIAL – HIGHLY RESTRICTED

MySQL 5.5 introduces a re-architected InnoDB that includes many performance

and scalability features in addition to what is available in the previous versions.
MySQL Server 5.5 optimizes MySQL server performance and scalability across
heterogeneous platforms, specifically on multi-CPU/core, hyper-threaded
architecture.

MySQL for Database Administrators Appendix H - 2

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

InnoDB: Default Storage Engine

From MySQL 5.5, InnoDB is the default storage

engine.
• ACID Transactions
• FKs

Oracle University and BOS-it GmbH & Co.KG use only

• Crash Recovery

CONFIDENTIAL – HIGHLY RESTRICTED

Oracle’s InnoDB provides highly efficient ACID-compliant transactional

capabilities, and includes unique architectural elements that assure high
performance and scalability. This storage engine is structurally designed to handle
transactional applications that require crash recovery, referential integrity, high
levels of user concurrency, and SLA exceeding response times.

MySQL for Database Administrators Appendix H - 3

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

InnoDB Performance Improvements

• Multiple buffer pool instances

• Multiple rollback segments
• Extended change buffering
• Improved recovery times

Oracle University and BOS-it GmbH & Co.KG use only

CONFIDENTIAL – HIGHLY RESTRICTED

• No more waiting when a thread is updating the buffer pool.

• Multiple rollback segments – now support of 128K concurrent transactions
• Change buffering includes delete buffering and purge buffering.
• From MySQL 5.5, there are default optimizations designed to speed up the
scanning and applying of redo logs, so the next restart is faster. Users who
previously sized redo logs artificially low because of slow recovery times
can now increase the log file size without concern.

MySQL for Database Administrators Appendix H - 4

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL Performance Improvements

• Better metadata locking within transactions

• Split LOCK_open mutex
• Eliminated LOCK_alarm mutex as bottleneck
• Eliminated LOCK_thread_count as bottleneck
• Improved performance/scale on Win32, 64

Oracle University and BOS-it GmbH & Co.KG use only

CONFIDENTIAL – HIGHLY RESTRICTED

Transaction serialization avoids that one session performs a data definition

language (DDL) statement on a table that is used in an incomplete transaction in
another session.
Windows-specific features for speeding up and scaling up
• Windows API calls for much of the I/O done inside MySQL
• Ability to build engines and other plug-ins as DLLs on Windows
• Network support for autodetecting the MAC address
• Much cleanup and simplifying of threading code

MySQL for Database Administrators Appendix H - 5

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Improved Availability

• Semi-synchronous replication
• Replication heartbeat
• Replication slave fsync options
• Automatic relay log recovery

Oracle University and BOS-it GmbH & Co.KG use only

CONFIDENTIAL – HIGHLY RESTRICTED

From MySQL 5.5, replication has been enhanced to:

• Ensure data consistency between master and slave
• Immediately detect if replication is not working
• Allow a crashed slave to automatically recover from the master relay log
• Allow users to filter events for specific servers
• Correctly convert data types between masters and slaves

MySQL for Database Administrators Appendix H - 6

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Improved Usability

• SIGNAL/RESIGNAL
• More partitioning options
• Replication server filtering
• Replication slave side type conversions
• Individual log flushing

Oracle University and BOS-it GmbH & Co.KG use only

• Pluggable external authentication, audit interfaces
• PERFORMANCE_SCHEMA

CONFIDENTIAL – HIGHLY RESTRICTED

• MySQL 5.5 extends the usability of the stored objects and table/index
partitioning features.
• SIGNAL and RESIGNAL statements allow you to implement exception-
handling logic in your stored procedures, stored functions, triggers, events,
and database applications.
• With the RANGE COLUMNS and LIST COLUMNS clauses of the CREATE
TABLE statement, partitioning is more flexible and can optimize queries
better.
• Performance schema provides tables that let you see performance data in
real time, or historically.

MySQL for Database Administrators Appendix H - 7

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Improved Scalability

• Scales on multi-core with InnoDB 1.0 and even

further with InnoDB built-in
• Improved log sys mutex
• Separate flush list mutex
• Improved purge scheduling

Oracle University and BOS-it GmbH & Co.KG use only

CONFIDENTIAL – HIGHLY RESTRICTED

• Transactions per second continue to increase beyond 16 coresThe internal

processing happens with less waiting and less blocking of other operations
involving the buffer pool. You do not need to do any additional.
• Separate flush list mutex means reduced contention with buffer pool
operations and faster InnoDB. You do not need to do any additional.
• From MySQL 5.5, purge can run in its own thread, thereby permitting more
concurrency.
• Generally, improved performance levels at higher transactions per second
and user connection loads, so applications remain responsive even when the
physical server resources are saturated.
(Mutexes are in-memory structures that prevent different threads from
interfering with each others’ changes to important memory areas such as the
buffer pool.)

MySQL for Database Administrators Appendix H - 8

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 Scales on Multi Core

SysBench Read Write

MySQL 5.5.4

MySQL 5.5.3
Transactions/Second

Oracle University and BOS-it GmbH & Co.KG use only

MySQL 5.1

AMD Opteron 7160 (Magny-Cours) @2100 MHz

64 GB memory
2 x Intel X25E SSD drives
OS is Oracle Enterprise Linux with the Enterprise Kernel
CONFIDENTIAL – HIGHLY RESTRICTED 4 sockets with a total of 48 cores.

MySQL for Database Administrators Appendix H - 9

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 SysBench Benchmarks

Linux

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50

Oracle University and BOS-it GmbH & Co.KG use only

(InnoDB Plug-in)

MySQL 5.1.50
(InnoDB built-in)

Intel Xeon X7460 x86_64

4 CPU x 6 Cores/CPU
2.66 GHz, 32GB RAM
Fedora 10

CONFIDENTIAL – HIGHLY RESTRICTED

MySQL for Database Administrators Appendix H - 10

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 SysBench Benchmarks

Linux

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50

Oracle University and BOS-it GmbH & Co.KG use only

(InnoDB Plug-in)

MySQL 5.1.50
(InnoDB built-in)

Intel Xeon X7460 x86_64

4 CPU x 6 Cores/CPU
2.66 GHz, 32GB RAM
Fedora 10

CONFIDENTIAL – HIGHLY RESTRICTED

MySQL for Database Administrators Appendix H - 11

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 Scales on Multi Core

SysBench Read Write

Sysbench OLTP: Read / Write

Performance

10000

Oracle University and BOS-it GmbH & Co.KG use only

Transactions Per Second

8000

6000
InnoDB
4000
MyISAM
2000

0
6 12 18 24 30 36

CPU Cores
AMD Opteron 7160 (Magny-Cours) @2100 MHz
64 GB memory
2 x Intel X25E SSD drives
OS is Oracle Enterprise Linux with the Enterprise Kernel
CONFIDENTIAL – HIGHLY RESTRICTED 4 sockets with a total of 48 cores.

MySQL for Database Administrators Appendix H - 12

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 SysBench Benchmarks

Windows

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50

Oracle University and BOS-it GmbH & Co.KG use only

(InnoDB Plug-in)

MySQL 5.1.50
(InnoDB built-in)

Intel x86_64
4 CPU x 2 Cores/CPU
3.166 GHz, 8GB RAM
Windows Server 2008

CONFIDENTIAL – HIGHLY RESTRICTED

MySQL for Database Administrators Appendix H - 13

13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

MySQL 5.5 SysBench Benchmarks

Windows

MySQL 5.5.6
(New InnoDB)

MySQL 5.1.50

Oracle University and BOS-it GmbH & Co.KG use only

(InnoDB Plug-in)

MySQL 5.1.50
(InnoDB built-in)

Intel x86_64
4 CPU x 2 Cores/CPU
3.166 GHz, 8GB RAM
Windows Server 2008

CONFIDENTIAL – HIGHLY RESTRICTED

MySQL for Database Administrators Appendix H - 14

14