05 Bufferpool
05 Bufferpool
05 Bufferpool
ADMINISTRIVIA
D ATA B A S E W O R K L O A D S
B I F U R C AT E D E N V I R O N M E N T
Extract
Transform
Load
B I F U R C AT E D E N V I R O N M E N T
Transactions
Analytical Queries
Extract
Transform
Load
D ATA B A S E S T O R A G E
D ATA B A S E S T O R A G E
Spatial Control:
→ Where to write pages on disk.
→ The goal is to keep pages that are used together often as
physically close together as possible on disk.
Temporal Control:
→ When to read pages into memory, and when to write
them to disk.
→ The goal is minimize the number of stalls from having to
read data from disk.
D I S K- O R I E N T E D D B M S
Get page #2 Execution
Engine
Buffer Pool
Directory Header
Pointer to page #2
2
Memory
Database File
1 2 3 4 5 … Pages
Disk
CMU 15-445/645 (Fall 2019)
8
T O D AY ' S A G E N D A
B U F F E R P O O L O R G A N I Z AT I O N
Buffer
Memory region organized as an array Pool
of fixed-size pages.
frame1
An array entry is called a frame.
frame2
frame3
When the DBMS requests a page, an
frame4
exact copy is placed into one of these
frames.
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L O R G A N I Z AT I O N
Buffer
Memory region organized as an array Pool
of fixed-size pages.
page1
frame1
An array entry is called a frame.
frame2
frame3
When the DBMS requests a page, an
frame4
exact copy is placed into one of these
frames.
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L O R G A N I Z AT I O N
Buffer
Memory region organized as an array Pool
of fixed-size pages.
page1
frame1
An array entry is called a frame.
frame2
page3
frame3
When the DBMS requests a page, an
frame4
exact copy is placed into one of these
frames.
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L M E TA - D ATA
Page Buffer
The page table keeps track of pages Table Pool
that are currently in memory.
page1 frame1
page1
page3 frame2
page3
Also maintains additional meta-data
frame3
per page:
→ Dirty Flag frame4
→ Pin/Reference Counter
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L M E TA - D ATA
Page Buffer
The page table keeps track of pages Table Pool
that are currently in memory.
page1 frame1
page1
page3 frame2
page3
Also maintains additional meta-data
frame3
per page:
→ Dirty Flag frame4
→ Pin/Reference Counter
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L M E TA - D ATA
Page Buffer
The page table keeps track of pages Table Pool
that are currently in memory.
page1 frame1
page1
page3 frame2
page3
Also maintains additional meta-data
frame3
per page:
→ Dirty Flag frame4
→ Pin/Reference Counter
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L M E TA - D ATA
Page Buffer
The page table keeps track of pages Table Pool
that are currently in memory.
page1 frame1
page1
page3 frame2
page3
Also maintains additional meta-data
page2
frame3
per page:
→ Dirty Flag frame4
→ Pin/Reference Counter
page1 page2 page3 page4
On-Disk File
B U F F E R P O O L M E TA - D ATA
Page Buffer
The page table keeps track of pages Table Pool
that are currently in memory.
page1 frame1
page1
page3 frame2
page3
Also maintains additional meta-data
page2
frame3
per page:
→ Dirty Flag page2 frame4
→ Pin/Reference Counter
page1 page2 page3 page4
On-Disk File
L O C K S V S . L AT C H E S
Locks:
→ Protects the database's logical contents from other
transactions.
→ Held for transaction duration.
→ Need to be able to rollback changes.
Latches:
→ Protects the critical sections of the DBMS's internal data
structure from other threads. ←Mutex
→ Held for operation duration.
→ Do not need to be able to rollback changes.
PA G E TA B L E V S . PA G E D I R E C T O R Y
A L L O C AT I O N P O L I C I E S
Global Policies:
→ Make decisions for all active txns.
Local Policies:
→ Allocate frames to a specific txn without considering the
behavior of concurrent txns.
→ Still need to support sharing pages.
B U F F E R P O O L O P T I M I Z AT I O N S
M U LT I P L E B U F F E R P O O L S
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. Q1 page0
→ Sequential Scans
→ Index Scans page1
page0 page3
page4
page5
CMU 15-445/645 (Fall 2019)
17
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. page0
→ Sequential Scans
→ Index Scans Q1 page1
page0 page3
page1 page4
page5
CMU 15-445/645 (Fall 2019)
17
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. page0
→ Sequential Scans
→ Index Scans Q1 page1
page0 page3
page1 page4
page5
CMU 15-445/645 (Fall 2019)
17
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. page0
→ Sequential Scans
→ Index Scans Q1 page1
page3 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
17
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. page0
→ Sequential Scans
→ Index Scans page1
page3 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
17
P R E- F E T C H I N G
Disk Pages
The DBMS can also prefetch pages
based on a query plan. page0
→ Sequential Scans
→ Index Scans page1
page3 page3
page4 page4
page5 Q1 page5
CMU 15-445/645 (Fall 2019)
18
P R E- F E T C H I N G
Disk Pages
Q1 SELECT * FROM A
index-page0
WHERE val BETWEEN 100 AND 250
index-page1
index-page3
index-page4
index-page5
CMU 15-445/645 (Fall 2019)
18
P R E- F E T C H I N G
index-page0
Disk Pages
index-page1 index-page4 index-page0
index-page2 index-page3 index-page5 index-page6
index-page1
0 99 100 199 200 299 300 399
index-page3
index-page4
index-page5
CMU 15-445/645 (Fall 2019)
18
P R E- F E T C H I N G
index-page0
Disk Pages
index-page1 index-page4
Q1 index-page0
index-page2 index-page3 index-page5 index-page6
index-page1
0 99 100 199 200 299 300 399
index-page0 index-page3
index-page4
index-page5
CMU 15-445/645 (Fall 2019)
18
P R E- F E T C H I N G
index-page0
Disk Pages
index-page1 index-page4 index-page0
index-page2 index-page3 index-page5 index-page6
Q1 index-page1
0 99 100 199 200 299 300 399
index-page0 index-page3
index-page1 index-page4
index-page5
CMU 15-445/645 (Fall 2019)
18
P R E- F E T C H I N G
index-page0
Disk Pages
index-page1 index-page4 index-page0
index-page2 index-page3 index-page5 index-page6
Q1 index-page1
0 99 100 199 200 299 300 399
index-page0 index-page3
index-page1 index-page4
index-page5
CMU 15-445/645 (Fall 2019)
19
SCAN SHARING
SCAN SHARING
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
Q1 page0
page1
page0 page3
page4
page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
page1
page0 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
page1
page0 Q1 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
page1
page3 Q1 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
Q2 page0
Q2 SELECT AVG(val) FROM A
page1
page3 Q1 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
Q2 SELECT AVG(val) FROM A
page1
page3 Q2 Q1 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
Q2 SELECT AVG(val) FROM A
page1
page3 page3
page4 page4
page5 Q2 Q1 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
Q2 page0
Q2 SELECT AVG(val) FROM A
page1
page3 page3
page4 page4
page5 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
Q2 SELECT AVG(val) FROM A
page1
page0 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
21
SCAN SHARING
Disk Pages
Q1 SELECT SUM(val) FROM A
page0
Q2 SELECT AVG(val) FROM A LIMIT 100
page1
page0 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
22
B U F F E R P O O L B Y PA S S
O S PA G E C A C H E
Demo: Postgres
CMU 15-445/645 (Fall 2019)
24
Goals:
→ Correctness
→ Accuracy
→ Speed
→ Meta-data overhead
L E A S T- R E C E N T LY U S E D
CLOCK
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
CLOCK
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
CLOCK
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
CLOCK
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
CLOCK
X
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
CLOCK
page.
→ Each page has a reference bit. ref=0 ref=0
→ When a page is accessed, set to 1. page4 page2
page5
CLOCK
page.
→ Each page has a reference bit. ref=1 ref=0
→ When a page is accessed, set to 1. page4 page2
page5
CLOCK
ref=0
→ When a page is accessed, set to 1. page4 page2
page5
PROBLEMS
SEQUENTIAL FLOODING
page1
page0 page3
page4
page5
CMU 15-445/645 (Fall 2019)
28
SEQUENTIAL FLOODING
page1
page0 page3
page4
page5
CMU 15-445/645 (Fall 2019)
28
SEQUENTIAL FLOODING
page1
page0 Q2 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
28
SEQUENTIAL FLOODING
page1
page3 Q2 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
28
SEQUENTIAL FLOODING
page3 Q2 page3
page1 page4
page2 page5
CMU 15-445/645 (Fall 2019)
29
B E T T E R P O L I C I E S : L O C A L I Z AT I O N
index-page0
Q1 INSERT INTO A VALUES (id++)
index-page1 index-page4
MIN id MAX
CMU 15-445/645 (Fall 2019)
31
index-page0
Q1 INSERT INTO A VALUES (id++)
index-page1 index-page4
MIN id MAX
CMU 15-445/645 (Fall 2019)
31
index-page0
Q1 INSERT INTO A VALUES (id++)
index-page1 index-page4
Q2 SELECT * FROM A WHERE id = ?
index-page2 index-page3 index-page5 index-page6
MIN id MAX
CMU 15-445/645 (Fall 2019)
32
D I R T Y PA G E S
B A C KG R O U N D W R I T I N G
CONCLUSION
PROJECT #1
Due Date:
Friday Sept 27th @ 11:59pm
General Hints:
→ Your ClockReplacer needs to check the "pinned" status
of a Page.
→ If there are no pages touched since last sweep, then
return the lowest page id.
General Hints:
→ Make sure you get the order of operations
correct when pinning.
CMU 15-445/645 (Fall 2019)
39
G E T T I N G S TA R T E D
THINGS TO NOTE
CODE QUALIT Y
P L A G I A R I S M WA R N I N G
NEXT CLASS
HASH TABLES!