MySQL 优化
MySQL 优化
1
Monday, March 7, 2011
Show of Hands
InnoDB? MyISAM? XtraDB? MySQL 4.x? MySQL 5.0.x? MySQL 5.1? MySQL 5.5? Something else?
2
Monday, March 7, 2011
Q: Would you like to talk about MySQL optimization? A: Sure. Optimization can be made at many levels. Im going to show 2-3 parts of the puzzle:
Query Optimization. How InnoDB works. (Time Permitting) Our work at Percona on improving MySQL.
3
Monday, March 7, 2011
Bookmark this manual page: http://dev.mysql.com/doc/refman/5.1/en/usingexplain.html It is the best source for anyone getting started.
4
Monday, March 7, 2011
IMDB database loaded into InnoDB tables (~5G). Download it and import it for yourself using imdbpy2sql.py: http://imdbpy.sourceforge.net/
5
Monday, March 7, 2011
First Example
6
Monday, March 7, 2011
3.09s
ALL means tablescan Anticipated number of rows to be examined In this case a sort is required because of the order by
8
Monday, March 7, 2011
We must revisit...
0.00s
Using = for comparison, but not primary key lookup. Identified title as a candidate index, chose to use it. Size of the index used (in bytes) Anticipated number of rows to be examined dropped considerably. 9
Monday, March 7, 2011
0.00s
At most one We can Better type of const. matching row. stop when we find one row. In InnoDB the primary key is often much faster than all other keys.
10
Monday, March 7, 2011
And..
0.00s
Type is range. BETWEEN, IN() and < > are also ranges.
Ignore the time with EXPLAIN. Only look at the time for a query.
11
Monday, March 7, 2011
We're looking for titles between BambA and BambZ* When we say index in MySQL, we mean trees.
That is, B-Tree/B+Tree/T-Tree. Pretend they're all the same (for simplication). There's no radically different indexing methods in MySQL unless you play storage engine Bingo**.
12
* In reality the range is a little wider ** The memory storage engine supports hash indexes
Whats that?
13
Monday, March 7, 2011
3.2s
14
Monday, March 7, 2011
15
Monday, March 7, 2011
LIKE Z%
0.05s
16
Monday, March 7, 2011
LIKE T%
3.13s
17
Monday, March 7, 2011
LIKE The %
3.07s
18
Monday, March 7, 2011
It dynamically samples the data to choose which is the better choice - or in some cases uses static statistics*. This helps the optimizer choose:
Which indexes will be useful. Which indexes should be avoided. Which is the better index when there is more than one.
Use an index if you want just a few rows. Scan cover-to-cover if you want a large percentage.
20
Monday, March 7, 2011
21
Monday, March 7, 2011
Development environments should contain sample data exported from production systems. Between two seemingly identical queries, execution plans may be very different.
22
3.41s
23
Monday, March 7, 2011
24
Monday, March 7, 2011
Index on production_year
3.53s
25
Monday, March 7, 2011
0.92s
26
Monday, March 7, 2011
Index on title(50)
0.02s
27
Monday, March 7, 2011
mysql> EXPLAIN SELECT * from title WHERE title = 'Pilot' AND production_year BETWEEN 2006 and 2009\G
28
Monday, March 7, 2011
Composite Indexes.
What is better?
29
Monday, March 7, 2011
Index on py_t
0.02s
30
http://www.mysqlperformanceblog.com/2010/01/09/getting-aroundoptimizer-limitations-with-an-in-list/
Index on t_py
0.00s
31
Monday, March 7, 2011
Recommendations
GOOD: Only some pilots made between 2006-2009. BAD: All pilots made between 2006-2009.
32
Monday, March 7, 2011
Recommendations (cont.)
RULE: Think how to lter the fastest. Use that order left to right. EXCEPTION: If there's a range (><, BETWEEN, %). Those always go to the RIGHT.
33
http://www.mysqlperformanceblog.com/2010/01/09/getting-aroundoptimizer-limitations-with-an-in-list/
Query Tuning
36
About Disks.
This is about the average for a 7200RPM drive. The variation is because disks are mechanical. We can much write faster sequentially than randomly.
37
Monday, March 7, 2011
When you write to a le, heres what happens in the Operating System:
Block 9, 10, 1, 4, 200, 5.
38
Monday, March 7, 2011
$ man fsync
Synopsis #include <unistd.h> int fsync(int fd); int fdatasync(int fd); Hint: MyISAM just writes to the OS buffer and has no durability.
Description fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) where that file resides. The call blocks until the device reports that the transfer has completed. It also flushes metadata information associated with the file (see stat(2)).
39
http://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/
Knowing this:
InnoDB wants to try and reduce random IO. It can not (safely) rely on the operating systems write buffering and be ACID compliant.
40
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
41
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
42
Monday, March 7, 2011
This is an optimization!
The log le IO is sequential and much cheaper than live updates. The IO for the eventual updates to the tablespace can be optimized as well.
Provided that you saved enough to recover, this shouldnt matter should it?
43
Monday, March 7, 2011
More on Logs...
Log Files
To gure out which pages need to be evicted we have two lists - the ush list and the LRU. Log activities are all assigned a LSN (log sequence number).
44
Monday, March 7, 2011
The background process of syncing dirty pages is normally referred to as a Checkpoint. InnoDB has fuzzy checkpointing.
45
Monday, March 7, 2011
Log Writing
You can change increase innodb_log_file_size. This will allow InnoDB to smooth out background IO for longer.
Tip: Optionally you could change innodb_log_files_in_group as well. Be aware that your effective log le is innodb_log_file_size * innodb_log_files_in_group
46
Monday, March 7, 2011
You can also change innodb_flush_log_at_trx_commit to 0 or 2 to reduce the durability requirements of this write.
innodb_log_buffer_size may also help buffer changes for longer before writing to the logs.
Very workload dependent - tends to be more helpful for writing big TEXT/BLOB changes.
47
Monday, March 7, 2011
In summary
What visibility do we have into these operations? How do we decide how much background work to do per second? What happens if we fall behind in background work?
48
Monday, March 7, 2011
Terminology
Oracle Product MySQL Server License GPL Percona Equivalent Product Percona Server The XtraDB Storage Engine XtraBackup License GPL GPL GPL
The InnoDB Storage GPL Engine (Plugin edition) InnoDB Hot Backup Commercial
50
Monday, March 7, 2011
Performance Improvements
Improved Buffer Pool Scalability Insert buffer controls Separate purge thread Completely disable the query cache. Faster page checksums* Strip comments before using query cache Improved Rollback Segment Scalability* Separate location of double write buffer* Improved IO Path + adaptive checkpointing Transaction logs larger than 4G supported.
Data dictionary memory consumption controls Support for different page sizes*
51
52
Monday, March 7, 2011
Once an InnoDB table is opened it is never freed from the in-memory data dictionary (which is unlimited in size). XtraDB introduces:
--innodb_dict_size_limit - a conguration item in bytes. innodb_dict_tables - a status variable for number entries in the cache.
53
Monday, March 7, 2011
Undo Slots
This means the number of open transactions is limited to 1023. See: http://bugs.mysql.com/bug.php?id=26590 Some statements require 2 undo slots.
54
Rollback Segments
In XtraDB, its also possible to have more than one rollback segment.
Conguration is --innodb-extra-rsegments=N This has the added effect of reducing mutex contention on the rollback segment:
55
Fast Checksums
The InnoDB page checksum computation is slower than it needs to be. XtraDB has the option to use a faster checksum format.
56
XtraDB now has support for different page sizes - 4K, 8K, 16K.
57
58
Usability Enhancements
Show contents of the buffer pool Save index statistics between restarts Import / export of buffer pool contents Show data dictionary User / Index / Table statistics Disable automatic statistics regeneration Transactional Replication Advise in processlist when waiting on Query cache mutex. Improved slow query log Log connection errors Import / export of innodb_file_per_table tables Deadlock counter Retain query response time distribution.
59
Monday, March 7, 2011
60
Export the contents of the buffer pool to a le called ib_lru_dump in the data directory:
SELECT * FROM information_schema.XTRADB_ADMIN_COMMAND /*! XTRA_LRU_DUMP*/; SELECT * FROM information_schema.XTRADB_ADMIN_COMMAND /*! XTRA_LRU_RESTORE*/;
Restored ib_lru_dump:
61
Note: Not the actual contents - it takes 8 bytes to remember the address of a 16K page.
Import/Export tables
Because --innodb-file-per-table still has information (data dictionary, undo) in the global tablespace you cant just back it up by itself. With a new setting, --innodb_expand_import=1, this is no longer the case. Tip: The import/export still has to be done with XtraBackup. Documentation available here:
http://www.percona.com/docs/wiki/percona-xtradb:patch:innodb_expand_import
62
Monday, March 7, 2011
$ mysql -e SET GLOBAL log_slow_verbosity = full; $ tail /var/log/mysql.slow Percona Server .. # Time: 100924 13:58:47 # User@Host: root[root] @ localhost [] # Thread_id: 10 Schema: imdb Last_errno: 0 Killed: 0 # Query_time: 399.563977 Lock_time: 0.000110 Rows_sent: 1 Rows_examined: 46313608 Rows_affected: 0 Rows_read: 1 # Bytes_sent: 131 Tmp_tables: 1 Tmp_disk_tables: 1 Tmp_table_sizes: 25194923 # InnoDB_trx_id: 1403 # QC_Hit: No Full_scan: Yes Full_join: No Tmp_table: Yes Tmp_table_on_disk: Yes # Filesort: Yes Filesort_on_disk: Yes Merge_passes: 5 # InnoDB_IO_r_ops: 1064749 InnoDB_IO_r_bytes: 17444847616 InnoDB_IO_r_wait: 26.935662 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 65329 SET timestamp=1285336727; select STRAIGHT_JOIN count(*) as c, person_id FROM cast_info FORCE INDEX(person_id) INNER JOIN title ON (cast_info.movie_id=title.id) WHERE title.kind_id = 1 GROUP BY cast_info.person_id ORDER by c DESC LIMIT 1;
63
Monday, March 7, 2011
User Statistics
( Not Possible )
MySQL Server
mysql> SET GLOBAL userstat_running = 1; Percona mysql> SELECT DISTINCT s.TABLE_SCHEMA, s.TABLE_NAME, s.INDEX_NAME FROM information_schema.statistics `s` LEFT JOIN information_schema.index_statistics IS ON (s.TABLE_SCHEMA = IS.TABLE_SCHEMA AND s.TABLE_NAME=IS.TABLE_NAME AND s.INDEX_NAME=IS.INDEX_NAME) WHERE IS.TABLE_SCHEMA IS NULL; +--------------+---------------------------+-----------------+ | TABLE_SCHEMA | TABLE_NAME | INDEX_NAME | +--------------+---------------------------+-----------------+ | art100 | article100 | ext_key | | art100 | article100 | site_id | | art100 | article100 | hash | | art100 | article100 | forum_id_2 | | art100 | article100 | published | | art100 | article100 | inserted | | art100 | article100 | site_id_2 | | art100 | author100 | PRIMARY | | art100 | author100 | site_id | ... +--------------+---------------------------+-----------------+ 1150 rows IN SET (1 min 44.23 sec)
Server
64
Monday, March 7, 2011
65
http://www.mysqlperformanceblog.com/2009/09/14/statisticsof-innodb-tables-and-indexes-available-in-xtrabackup/
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
UPDATE City SET name = 'Morgansville' WHERE name = 'Brisbane' AND CountryCode='AUS'
Buffer Pool
Tablespace
66
Monday, March 7, 2011
Log Files
Buffer Pool
Tablespace
66
Monday, March 7, 2011
The End.
Questions?
67
Monday, March 7, 2011