Scribd
Upload
41 views

Table Partitioning:: Secret Weapon For Big Data Problems

Table partitioning is a technique for managing large datasets that can help with performance and maintenance tasks. It works by dividing tables into multiple partitions that can each be maintained independently. This allows tasks like backups, index rebuilds, statistics updates, purging and archiving to be performed on individual partitions rather than entire tables. It also enables query optimization techniques like partition elimination and skip-scans to improve performance for queries that filter on the partition key column.

Uploaded by

power
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
41 views

Table Partitioning:: Secret Weapon For Big Data Problems

Table partitioning is a technique for managing large datasets that can help with performance and maintenance tasks. It works by dividing tables into multiple partitions that can each be maintained independently. This allows tasks like backups, index rebuilds, statistics updates, purging and archiving to be performed on individual partitions rather than entire tables. It also enables query optimization techniques like partition elimination and skip-scans to improve performance for queries that filter on the partition key column.

Uploaded by

power
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 46

Table Partitioning:

Secret Weapon for Big Data Problems

October 15-18, 2013 John Sterrett, Sr. Database Admin Advisor


Charlotte, NC DELL
About John

2
How to contact me?

http://johnsterrett.com/go/partition

Blog: http://JohnSterrett.com/
Twitter: @JohnSterrett
LinkedIn: http://linkedin.com/in/johnsterrett

3
Agenda

Big Data starting to slow you down? Data growth putting your
maintenance tasks in danger of not meeting your SLAs? Wish you
could archive old data with minimal impact to your tables during the
archive process or that you could eliminate most of the data in your
tables when you query them? If so, it’s time you consider implementing
table partitioning to help with general performance and reduce your
window for completing maintenance tasks.

•  Learn if Partitioning can help you manage big data.


•  Understand how partitioning works.
•  Learn how to maintain table partitioning with an automated
sliding window.

4
Assumptions…

•  You are not familiar with Table Partitioning and want


to see how it works.
•  You have Enterprise Edition
•  Highly Transactional Tables with over 50 GB of data
•  General Maintenance Tasks are in danger of not meeting
your SLA
•  Purge and/or Archive process is slowing you down.

5
The Big Question…

HOW CAN TABLE PARTITIONG MAKE MY LIFE


EASIER?

6
How partitioning helps me?

•  Reduce Maintenance Tasks

7
How partitioning helps me?

•  Reduce Maintenance Tasks


•  Improves Purging and/or Archiving

8
How partitioning helps me?

•  Reduce Maintenance Tasks


•  Improves Purging and/or Archiving
•  Improves Performance

9
Big Question…

HOW DOES TABLE PARTITIONING WORK?

10
High Level…

11
High Level…

Partition  Table

Partition  Scheme

Partition  Function

12
Selecting The Partition Column

Only get one column, use it wisely!


Column should be a highly used filter.
•  Review index usage statistics
•  Review Missing index statistics
•  Review Queries and Talk to developers ;-)
Column must be part of clustered index

13
Partition Function

Defines the data type used to distribute data into partitions.


Assigns boundary values to split data between partitions.
Assigns the RANGE for boundary values
Partitions = Boundary values + 1
NULL values go to left most partition

14
Partition Function

Partition functions are not static. They can change over


time with SPLIT and MERGE statements.

Range LEFT is used by default.

15
Partition Function - Range

{min….100}, {101…200}, {201...300}, {301…max}

{min…99}, {100…199}, {200…299}, {300…max}

16
Partition Scheme

Assigns a partition function to a partition scheme


Assigns filegroups to partitions

17
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC


Big Question…

HOW DOES TABLE PARTITIONING IMPROVE MY


MAINTENANCE TASKS?

19
Improving Maintenance Tasks

Backup and restore filegroups based on business priorities.


Index Maintenance by partition
New Features in SQL 2014
•  Rebuild index online by partition
•  Incremental Statistics by partition (SQL 2014 CTP 2)

20
Incremental Statistics

Just added in SQL Server 2014 CTP2

From Books Online:


“When ON, the statistics are recreated as per partition
statistics.”

Cannot be used for the following:


•  Statistics created with indexes that are not partition-
aligned with the base table
•  Filtered Indexes

21
HOW DOES TABLE PARTITIONING IMPROVE
PERFORMANCE?

22
Partition Elimination

23
Skip-Scan: Seek keys

24
Database Compression by Partition

25
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC


HOW DOES PARTITIONING IMPROVE ARCHIVING AND
PURGING?

27
Real World Example:

28
Quickest way to move billions of rows

Meta data swap is quickest way to move billions of rows


assuming you can get a schema lock.

29
Sliding Window Goals

SPLIT and MERGE with empty partitions


Use SWITCH to do meta-data swaps
Minimize all physical data movement

30
Range Right - MERGE

31
Range Left - MERGE

32
Range Right - SPLIT

33
Range Left - SPLIT

34
Visual Sliding Window Example

35
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4 Partiton5


(EMPTY) (DATA) (DATA) (DATA) (DATA)

FG1 FG2 FG3 FG4 FG5


Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4 Partiton5


(EMPTY) (DATA) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5

Staging  
Table    
(EMPTY)

FG2
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4 Partiton5


(EMPTY) (EMPTY) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5


SWAP

Staging  
Table    (DATA)

FG2
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4 Partiton5


(EMPTY) MERGE (EMPTY) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5

Staging  
Table    (DATA)

FG2
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4


(EMPTY) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5

Staging  
Table    (DATA)

FG2
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4


NEXT  USED
(EMPTY) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5

Staging  
Table    
(EMPTY)

FG2
Visual Sliding Window Example

Partiton1 Partiton2 Partiton3 Partiton4 Partiton5


(EMPTY) (DATA) (DATA) (DATA) (DATA)

FG1 FG3 FG4 FG5 FG2


Sliding Window Steps...
Create partition swap table (if it doesn’t exist)
1.  Insert partition swap meta data
2.  Create staging table
3.  Meta-data swap (partition 2 with staging table)
4.  Merge Partitions #1 and #2
5.  Mark next used partition
6.  Split to create new partition
7.  Update processed partition swap meta data.

43
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC


Questions…

http://johnsterrett.com/go/partition

Blog: http://JohnSterrett.com/
Twitter: @JohnSterrett
LinkedIn: http://linkedin.com/in/johnsterrett

45
Thank you
for attending this session and the
2013 PASS Summit in Charlotte, NC

46
October 15-18, 2013 | Charlotte, NC

You might also like