Mysql query optimization
3 likes•1,139 views
The document discusses MySQL query optimization. It covers the query optimizer, principles of optimization like using EXPLAIN and profiling, indexes, JOIN optimization, and ORDER BY/GROUP BY optimization. The key points are to identify bottlenecks, use indexes on frequently filtered fields, avoid indexes on fields that change often or contain many duplicates, and consider composite indexes to cover multiple queries.
![Use Index
SELECT col1, col2 FROM
sort
a ORDER BY [sort]
SELECT col1, col2 FROM
a WHERE colX=value (colx, sort)
ORDER BY [sort]
SELECT * FROM a
WHERE uid=1 ORDER BY (uid, x, y)
x, y
SELECT * FROM a
won’t use index
ORDER BY YEAR(date)
...... ......](https://image.slidesharecdn.com/mysqlqueryoptimization-100709050517-phpapp01/85/Mysql-query-optimization-69-320.jpg)
![Use Index
SELECT col1, col2 FROM
sort
a ORDER BY [sort]
SELECT col1, col2 FROM
a WHERE colX=value (colx, sort)
ORDER BY [sort]
SELECT * FROM a
WHERE uid=1 ORDER BY (uid, x, y)
x, y
SELECT * FROM a
won’t use index
ORDER BY YEAR(date)
...... ......](https://image.slidesharecdn.com/mysqlqueryoptimization-100709050517-phpapp01/85/Mysql-query-optimization-70-320.jpg)
![Use Index
SELECT col1, col2 FROM
sort
a ORDER BY [sort]
SELECT col1, col2 FROM
a WHERE colX=value (colx, sort)
ORDER BY [sort]
SELECT * FROM a
WHERE uid=1 ORDER BY (uid, x, y)
x, y
SELECT * FROM a
won’t use index
ORDER BY YEAR(date)
...... ......](https://image.slidesharecdn.com/mysqlqueryoptimization-100709050517-phpapp01/85/Mysql-query-optimization-71-320.jpg)
![Use Index
SELECT col1, col2 FROM
sort
a ORDER BY [sort]
SELECT col1, col2 FROM
a WHERE colX=value (colx, sort)
ORDER BY [sort]
SELECT * FROM a
WHERE uid=1 ORDER BY (uid, x, y)
x, y
SELECT * FROM a
won’t use index
ORDER BY YEAR(date)
...... ......](https://image.slidesharecdn.com/mysqlqueryoptimization-100709050517-phpapp01/85/Mysql-query-optimization-72-320.jpg)
![Use Index
SELECT col1, col2 FROM
sort
a ORDER BY [sort]
SELECT col1, col2 FROM
a WHERE colX=value (colx, sort)
ORDER BY [sort]
SELECT * FROM a
WHERE uid=1 ORDER BY (uid, x, y)
x, y
SELECT * FROM a
won’t use index
ORDER BY YEAR(date)
...... ......](https://image.slidesharecdn.com/mysqlqueryoptimization-100709050517-phpapp01/85/Mysql-query-optimization-73-320.jpg)
Mysql query optimization
- 1. MySQL Query Optimization 2010.07.09 Cai Baohua
- 2. Agenda • What the query optimizer is • The principles of the optimization • Explain and Profiling • Use Index • JOIN Optimization • ORDER BY, GROUP BY Optimization
- 4. MySQl Query Optimizer Parser Table Table Optimizer Modification Maintenance ..... Module Module Access Control Module
- 5. MySQl Query Optimizer • Not only CBO but aslo CBO + RBO • Cost Base Optimizer • Rule Base Optimizer
- 6. The Principles Of the Optimization
- 8. • Optimizing the query which need more optimization
- 9. • Optimizing the query which need more optimization • Identify the performance bottleneck
- 10. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects
- 11. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often
- 12. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often • Always using the small result set to drive the large result set
- 13. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often • Always using the small result set to drive the large result set • Complete the sequencing in the index as much as possible
- 14. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often • Always using the small result set to drive the large result set • Complete the sequencing in the index as much as possible • Fetch the only fields that we need
- 15. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often • Always using the small result set to drive the large result set • Complete the sequencing in the index as much as possible • Fetch the only fields that we need • Only use the most effective conditions of the filter
- 16. • Optimizing the query which need more optimization • Identify the performance bottleneck • Find clear optimization objects • Start with Explain and use Profile more often • Always using the small result set to drive the large result set • Complete the sequencing in the index as much as possible • Fetch the only fields that we need • Only use the most effective conditions of the filter • Avoid the complex Join and sub queries as far as possible
- 18. Use Explain and Profiling Explain tells you: • In which order the tables are read • What types of read operations that are made • Which indexes could have been used • Which indexes are used • How the tables refer to each other • How many rows the optimizer estimates to retrieve from each table
- 19. Use Explain and Profiling
- 20. Use Explain and Profiling Explain Types Different join types. system !"#$%&'(#$"&)$*+(,$*+#$-*.$ const /%$%"#$0*)%$*+#$0&%1"2+3$-*.4$%-#&%#5$&)$&$ 1*+)%&+%$ eq_ref 6+#$-*.$7#-$-*.$8-*0$7-#92*:)$%&'(#)$ ref ;#9#-&($-*.)$.2%"$0&%1"2+3$2+5#<$9&(:#$ ref_or_null =2>#$-#84$7(:)$?@==$9&(:#)$ index_merge ;#9#-&($2+5#<$)#&-1"#)$&-#$0#-3#5$ unique_subquery ;&0#$&)$-#8$8*-$)*0#$):'A:#-2#)$ index_subquery /)$&'*9#$8*-$+*+B:+2A:#$2+5#<#)$ range /$-&+3#$2+5#<$)1&+$ index !"#$."*(#$2+5#<$2)$)1&++#5$ ALL /$8:(($%&'(#$)1&+$
- 21. Use Explain and Profiling Explain Types Different join types. system !"#$%&'(#$"&)$*+(,$*+#$-*.$ good const /%$%"#$0*)%$*+#$0&%1"2+3$-*.4$%-#&%#5$&)$&$ 1*+)%&+%$ eq_ref 6+#$-*.$7#-$-*.$8-*0$7-#92*:)$%&'(#)$ ref ;#9#-&($-*.)$.2%"$0&%1"2+3$2+5#<$9&(:#$ ref_or_null =2>#$-#84$7(:)$?@==$9&(:#)$ index_merge ;#9#-&($2+5#<$)#&-1"#)$&-#$0#-3#5$ unique_subquery ;&0#$&)$-#8$8*-$)*0#$):'A:#-2#)$ index_subquery /)$&'*9#$8*-$+*+B:+2A:#$2+5#<#)$ range /$-&+3#$2+5#<$)1&+$ index !"#$."*(#$2+5#<$2)$)1&++#5$ ALL /$8:(($%&'(#$)1&+$ bad
- 22. Use Explain and Profiling Explain Extra This column contains additional information about how MySQL resolves the query. Using index !"#$%#&'()$*&$+%#,)#-$&)%,*.")$/%01$)"#$*2-#3$ Using where 40)$,(($%05&$,%#$'&#-$*2$)"#$%#&'()$ Distinct 62(7$,$&*2.(#$%05$*&$%#,-$8#%$%05$+019*2,:02$ Not exists ;$LEFT JOIN$1*&&*2.$%05&$08:1*<,:02$*&$'&#-$ Using filesort ;2$#3)%,$%05$&0%:2.$&)#8$*&$-02#$ Using temporary ;$)#180%,%7$),9(#$*&$'&#-$ Range checked !"#$%#,-$)78#$*&$08:1*<#-$*2-*=*-',((7$/0%$#,+"$ for each record +019*2,:02$0/$%05&$/%01$)"#$8%#=*0'&$),9(#&$
- 23. Use Explain and Profiling • Open / Close Query Profiler mysql> set profiling = 1 (close: 0)
- 24. Use Explain and Profiling Show profiles
- 25. Use Explain and Profiling SHOW PROFILE • ALL - displays all information • BLOCK IO - displays counts for block input and output operations • CONTEXT SWITCHES - displays counts for voluntary and involuntary context switches • IPC - displays counts for messages sent and received • MEMORY - is not currently implemented • PAGE FAULTS - displays counts for major and minor page faults • SOURCE - displays the names of functions from the source code, together with the name and line number of the file in which the function occurs • SWAPS - displays swap count
- 26. Use Explain and Profiling Show more info
- 27. Use Index
- 28. Index Types
- 29. Index Types • Balance-Tree • Primary Key • Secondary Index • InnoDB, MyISAM often use
- 30. Index Types • Balance-Tree • Primary Key • Secondary Index • InnoDB, MyISAM often use • Hash • Memory, NDB Cluster • “=”, “IN”, “<=>” not > < between != like • not work for ORDER BY
- 31. Index Types • Balance-Tree • Primary Key • Secondary Index • InnoDB, MyISAM often use • Hash • Memory, NDB Cluster • “=”, “IN”, “<=>” not > < between != like • not work for ORDER BY • Fulltext • CHAR,VARCHAR and TEXT • Uses it instead of LIKE ‘%*****%’, more efficient
- 32. Index Types • Balance-Tree • Primary Key • Secondary Index • InnoDB, MyISAM often use • Hash • Memory, NDB Cluster • “=”, “IN”, “<=>” not > < between != like • not work for ORDER BY • Fulltext • CHAR,VARCHAR and TEXT • Uses it instead of LIKE ‘%*****%’, more efficient • R-Tree • to solve the problem of spatial data retrieval • only data type: GEOMETRY
- 33. Pros and Cons of Index
- 34. Pros and Cons of Index • Pros • Improve the efficiency of data retrieval • Reduce the cost of database I/O • Reduce the cost of data sorting
- 35. Pros and Cons of Index • Pros • Improve the efficiency of data retrieval • Reduce the cost of database I/O • Reduce the cost of data sorting • Cons • index will take more disk space • slow the speed of updating table (insert, update, delete)
- 36. When Use Index?
- 37. When Use Index? • Field used in WHERE more frequently, use index
- 38. When Use Index? • Field used in WHERE more frequently, use index • Field like status or type, no index
- 39. When Use Index? • Field used in WHERE more frequently, use index • Field like status or type, no index • Contain too many records records, which bring too many random I/O, to many duplicate I/O
- 40. When Use Index? • Field used in WHERE more frequently, use index • Field like status or type, no index • Contain too many records records, which bring too many random I/O, to many duplicate I/O • Field updated too often, no index
- 41. When Use Index? • Field used in WHERE more frequently, use index • Field like status or type, no index • Contain too many records records, which bring too many random I/O, to many duplicate I/O • Field updated too often, no index • Field not in WHERE, no index
- 42. 1 or N-Columns Index
- 43. 1 or N-Columns Index • No absolute conclusion
- 44. 1 or N-Columns Index • No absolute conclusion • When a filter field can filter data more than 90% and the other filter fields will be updated often, which we can try to use composite index
- 45. 1 or N-Columns Index • No absolute conclusion • When a filter field can filter data more than 90% and the other filter fields will be updated often, which we can try to use composite index • Reduce the cost of index updating and disk space of index
- 46. 1 or N-Columns Index • No absolute conclusion • When a filter field can filter data more than 90% and the other filter fields will be updated often, which we can try to use composite index • Reduce the cost of index updating and disk space of index • let one index used in different quries
- 47. 1 or N-Columns Index • No absolute conclusion • When a filter field can filter data more than 90% and the other filter fields will be updated often, which we can try to use composite index • Reduce the cost of index updating and disk space of index • let one index used in different quries • Don’t over index
- 48. Index Prefixes
- 49. Index Prefixes • Index prefixes of CHAR,VARCHAR, BINARY, VARBINARY, BLOB, and TEXT columns
- 50. Index Prefixes • Index prefixes of CHAR,VARCHAR, BINARY, VARBINARY, BLOB, and TEXT columns • name char (200)
- 51. Index Prefixes • Index prefixes of CHAR,VARCHAR, BINARY, VARBINARY, BLOB, and TEXT columns • name char (200) • most value are unique within the first 10-20
- 52. Index Prefixes • Index prefixes of CHAR,VARCHAR, BINARY, VARBINARY, BLOB, and TEXT columns • name char (200) • most value are unique within the first 10-20 • CREATE INDEX part_of_name ON customer (name(10));
- 53. Index Prefixes • Index prefixes of CHAR,VARCHAR, BINARY, VARBINARY, BLOB, and TEXT columns • name char (200) • most value are unique within the first 10-20 • CREATE INDEX part_of_name ON customer (name(10)); • faster query and disk I/O reduction
- 54. Limitation of Mysql Index • MyISAM - the total length of index <= 1000 bytes • BLOB and TEXT only create Index Prefix • Mysql not support Function Index • “!=” or “<>”, won’t use index • abs(column) etc, won’t use index • Join (a.city = b.city). If the filter fileds’ type are not the same, mysql won’t use index • Like ‘%abc’, won’t use index • Hash index only can be used when “=”, “<=>”, “IN”
- 55. Join
- 56. Principle • Nested Loop Join
- 57. Example
- 59. users_group(g) index ref scan
- 60. Nested Loop (ref) g.group_id=m.group_id users_group(g) group_message(m) index ref scan index ref scan
- 61. Nested Loop (ref) Nested Loop (ref) g.group_id=m.group_id m.id=c.group_msg_id Result Set Output users_group(g) group_message(m) group_message_content() index ref scan index ref scan index ref scan
- 62. Ideas for optimization • Minimize the number of Nested Loop • Give priority to optimizing the inner loop • Indexing filter fields • ... FROM A, B WHERE B.group_id = A.group_id • Join Buffer size, type is All, index, range, index_merge
- 63. Order By, Group By
- 64. How Satisfy Order By
- 65. How Satisfy Order By • Use Index, without doing a any extra sorting
- 66. How Satisfy Order By • Use Index, without doing a any extra sorting • Use filesort algorithms
- 67. Use Index
- 68. Use Index
- 69. Use Index SELECT col1, col2 FROM sort a ORDER BY [sort] SELECT col1, col2 FROM a WHERE colX=value (colx, sort) ORDER BY [sort] SELECT * FROM a WHERE uid=1 ORDER BY (uid, x, y) x, y SELECT * FROM a won’t use index ORDER BY YEAR(date) ...... ......
- 70. Use Index SELECT col1, col2 FROM sort a ORDER BY [sort] SELECT col1, col2 FROM a WHERE colX=value (colx, sort) ORDER BY [sort] SELECT * FROM a WHERE uid=1 ORDER BY (uid, x, y) x, y SELECT * FROM a won’t use index ORDER BY YEAR(date) ...... ......
- 71. Use Index SELECT col1, col2 FROM sort a ORDER BY [sort] SELECT col1, col2 FROM a WHERE colX=value (colx, sort) ORDER BY [sort] SELECT * FROM a WHERE uid=1 ORDER BY (uid, x, y) x, y SELECT * FROM a won’t use index ORDER BY YEAR(date) ...... ......
- 72. Use Index SELECT col1, col2 FROM sort a ORDER BY [sort] SELECT col1, col2 FROM a WHERE colX=value (colx, sort) ORDER BY [sort] SELECT * FROM a WHERE uid=1 ORDER BY (uid, x, y) x, y SELECT * FROM a won’t use index ORDER BY YEAR(date) ...... ......
- 73. Use Index SELECT col1, col2 FROM sort a ORDER BY [sort] SELECT col1, col2 FROM a WHERE colX=value (colx, sort) ORDER BY [sort] SELECT * FROM a WHERE uid=1 ORDER BY (uid, x, y) x, y SELECT * FROM a won’t use index ORDER BY YEAR(date) ...... ......
- 74. Use Filesort • increase max_length_for_stort_data • remove return fields which are not necessary • increase sort_buffer_size
- 75. How Satisfy Group By • Loose Index Scan • Tight Index Scan
- 76. Loose Index Scan
- 77. Loose Index Scan
- 78. Loose Index Scan Conditions Example The query is over a SELECT c1, c2 FROM t1 single table GROUP BY c1, c2; only columns that form a leftmost prefix of the index on (c1,c2,c3) index and no other • GROUP BY c1, c2 • CROUP BY c2, c3 columns. only can use aggregate SELECT c1, MIN(c2) FROM functions like MAX, t1 GROUP BY c1; MIN
- 79. Loose Index Scan Conditions Example The query is over a SELECT c1, c2 FROM t1 single table GROUP BY c1, c2; only columns that form a leftmost prefix of the index on (c1,c2,c3) index and no other • GROUP BY c1, c2 • CROUP BY c2, c3 columns. only can use aggregate SELECT c1, MIN(c2) FROM functions like MAX, t1 GROUP BY c1; MIN
- 80. Loose Index Scan Conditions Example The query is over a SELECT c1, c2 FROM t1 single table GROUP BY c1, c2; only columns that form a leftmost prefix of the index on (c1,c2,c3) index and no other • GROUP BY c1, c2 • CROUP BY c2, c3 columns. only can use aggregate SELECT c1, MIN(c2) FROM functions like MAX, t1 GROUP BY c1; MIN
- 81. Loose Index Scan Conditions Example The query is over a SELECT c1, c2 FROM t1 single table GROUP BY c1, c2; only columns that form a leftmost prefix of the index on (c1,c2,c3) index and no other • GROUP BY c1, c2 • CROUP BY c2, c3 columns. only can use aggregate SELECT c1, MIN(c2) FROM functions like MAX, t1 GROUP BY c1; MIN
- 82. Loose Index Scan
- 83. Loose Index Scan
- 84. Loose Index Scan SELECT c1, c2 FROM t1 WHERE c1 < const Any other parts of the GROUP BY c1, c2; index than those from the GROUP BY SELECT MAX(c3), MIN referenced in the query (c3), c1, c2 FROM t1 must be constants WHERE c2 > const GROUP BY c1, c2; Prefix index cannot be col VARCHAR(20), used for loose index scan INDEX (col(10))
- 85. Loose Index Scan SELECT c1, c2 FROM t1 WHERE c1 < const Any other parts of the GROUP BY c1, c2; index than those from the GROUP BY SELECT MAX(c3), MIN referenced in the query (c3), c1, c2 FROM t1 must be constants WHERE c2 > const GROUP BY c1, c2; Prefix index cannot be col VARCHAR(20), used for loose index scan INDEX (col(10))
- 86. Loose Index Scan If loose index scan is applicable to a query, the EXPLAIN output shows Using index for group-by in the Extra column.
- 88. Tight Index Scan • MySQL Query Optimizer • If loose index scan are not met, then try tight index scan • Different with loose, tight • After finding all index keys in WHERE conditions, then MySQL do the grouping operation
- 89. Tight Index Scan
- 90. Tight Index Scan idx(c1,c2,c3) on table t1(c1,c2,c3,c4)
- 91. Tight Index Scan idx(c1,c2,c3) on table t1(c1,c2,c3,c4) • A gap in the GROUP BY • SELECT c1, c2, c3 FROM t1 WHERE c2 = 'a' GROUP BY c1, c3;
- 92. Tight Index Scan idx(c1,c2,c3) on table t1(c1,c2,c3,c4) • A gap in the GROUP BY • SELECT c1, c2, c3 FROM t1 WHERE c2 = 'a' GROUP BY c1, c3; • not the first part of the key • SELECT c1, c2, c3 FROM t1 WHERE c1 = 'a' GROUP BY c2, c3;
- 93. More... • Books • <<MySQL >>, Author: • Web Sites • http://dev.mysql.com/doc/refman/5.1/en/ optimization.html • http://www.slideshare.net/
- 94. Q and A