PostgreSQL Performance

for Application Developers

Because There is no Magic Button

Zafin, Ottawa
19-Feb-2025
Introducing PostgreSQL

● Object-relational database
● Open source with a liberal license
● SQL Standard compliant
● ACID compliant (atomicity, consistency, isolation, durability)
● Supports structured as well unstructured data
● Highly extensible
A bit of a history lesson …

● 1986 - The Berkeley POSTGRES Project

○ Implementation started at University of California Berkeley
○ Led by Professor Michael Stonebraker
○ Sponsored by
■ Defense Advanced Research Projects Agency
■ Army Research Office
■ National Science Foundation
■ ESL, Inc
● 1994 - Postgres95
○ SQL interpreter added
○ Code made open source
○ 30-50% faster than its predecessor
● 1996 - PostgreSQL
Abstract

● PostgreSQL adoption is exploding and the move to the cloud is fueling it

● The difference between kicking things off and scaling in production
● The four areas of focus for scaling PostgreSQL
○ Query & SQL Optimization
○ Architectural Improvements
○ Performance Features
○ Parameter Tuning
So - what do you do
when you need to
scale PostgreSQL in
the cloud?
Scale by credit card!
 Well, not really …
You are only delaying the inevitable
You tested your application here
… and this is what production looks like
If you find this button, please let me know!
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
pg_stat_statement is your friend

● PostgreSQL extension, included in distribution

○ Provides a view in PostgreSQL
● Logs statistics about SQL statements
● Easy stats to watch out for
○ Long running (mean_exec_time )
○ Most frequent (calls)
○ Standard deviation in execution time (stddev_exec_time )
○ I/O intensive (blk_read_time , blk_write_time )
What to watch out for

● pg_stat_statements is off by default

● The data is aggregated with time windows not available
● Small ~4% CPU overhead
Some tooling for stats and visualization

● pgAdmin
● pgBadger
● Prometheus with Grafana
● Commercial tools
○ Datadog
○ DBeaver
○ New Relic
EXPLAIN plan is your friend

● ‘Cost’ based query planner of PostgreSQL

○ Configuration parameters
○ Database statistics
● Key metrics in the EXPLAIN plan
○ Cost
○ Rows
○ Width
● Key outputs of the EXPLAIN plan
○ Scan
○ Join
○ Aggregate & grouping
○ Sort
● EXPLAIN vs EXPLAIN ANALYZE
Example
We want the top 5 products by total sales in 2024 from a database with tables: orders, order_details, & products

SELECT p.product_name, SUM(od.quantity * od.unit_price) AS

total_sales
FROM products p
JOIN order_details od ON p.product_id = od.product_id
JOIN orders o ON od.order_id = o.order_id
WHERE o.order_date >= '2024-01-01' AND o.order_date <= '2024-12-31'
GROUP BY p.product_name
ORDER BY total_sales DESC
LIMIT 5;
Example
Limit (cost=1000.43..1000.45 rows=5 width=40)
-> Sort (cost=1000.43..1005.43 rows=2000 width=40)
Sort Key: (sum((od.quantity * od.unit_price))) DESC
-> HashAggregate (cost=950.00..975.00 rows=2000 width=40)
Group Key: p.product_name
-> Hash Join (cost=500.00..900.00 rows=10000 width=20)
Hash Cond: (od.product_id = p.product_id)
-> Hash Join (cost=250.00..600.00 rows=10000 width=16)
Hash Cond: (od.order_id = o.order_id)
-> Seq Scan on order_details od (cost=0.00..300.00 rows=10000 width=16)
-> Hash (cost=225.00..225.00 rows=2000 width=8)
-> Seq Scan on orders o (cost=0.00..225.00 rows=2000 width=8)
Filter: (order_date >= '2024-01-01' AND order_date <= '2024-12-31')
-> Hash (cost=150.00..150.00 rows=5000 width=12)
-> Seq Scan on products p (cost=0.00..150.00 rows=5000 width=12)
Helping the query planner do its job

● Up to date database statistics

○ Run ANALYZE periodically
● Accurate cost calculations
○ Tune database parameters
Watch out for locks!
Session 1 Session 2

BEGIN; UPDATE foo SET … WHERE id = 1;

UPDATE foo SET … WHERE id = 1; (waits)

UPDATE foo SET … WHERE id = 2;

UPDATE foo SET … WHERE id = 3;

COMMIT; Locks
 Query & SQL Optimization
Key takeaways:
● Monitor your queries
● Analyze the execution
● Code to avoid locks
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
Load Balancing

● Distributing database queries across multiple servers to optimize

resource use and improve performance
● Benefits
○ Prevents any single server from becoming a bottleneck.
○ Facilitates horizontal scaling by adding more replicas.
○ Enhances system's resilience against server failures.
Load balancing

Application Application

Reporting & Primary

Analytics
Reporting &
Analytics

Standby 1 Standby 2

Write
Read
Replicate
Load balancing
Single Node SELECTs Load Balanced 3-node Cluster
transaction type: <builtin: select only> transaction type: <builtin: select only>
scaling factor: 10 scaling factor: 10
query mode: simple query mode: simple
number of clients: 25 number of clients: 25
number of threads: 1 number of threads: 1
maximum number of tries: 1 maximum number of tries: 1
duration: 60 s duration: 60 s
number of transactions actually processed: 19139 number of transactions actually processed: 24885
number of failed transactions: 0 (0.000%) number of failed transactions: 0 (0.000%)
latency average = 67.215 ms latency average = 51.449 ms
initial connection time = 8620.897 ms initial connection time = 8896.110 ms
tps = 371.939402 (without initial connection time) tps = 485.918972 (without initial connection time)

+30%
Partitioning

● Dividing a large table into smaller, more manageable pieces, called

partitions, based on certain criteria (e.g., date ranges, geographic
location).
● Benefits
○ Performance: Queries that access only a subset of data can run faster.
○ Maintenance: Operations like backups, deletes, and archiving can be performed on
individual partitions.
○ Indexing: Smaller, partition-specific indexes are faster to update and search.
Partitioning

Application Application

Jan Feb Mar

Apr May Jun
Jul Aug Sep
Oct Nov Dec

Q1 Q2 Q3 Q4
Partitioning
select * from foo where month = ‘Aug’ select * from foo where month = ‘Aug’

Application Application

Jan Feb Mar

Apr May Jun
Jul Aug Sep
Oct Nov Dec

Q1 Q2 Q3 Q4
 Architectural Improvements
Key takeaway:
● Don’t overload a single node!
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
Indexes

● B-Tree - default index

○ Default index that structure data into a balanced tree
○ Composite - multi column
■ Useful in cases where queries filter based on multiple columns
○ Partial - conditional index on subset of data
■ Defined by a conditional expression, making the index smaller in size
○ Covering - includes an additional column
■ Index-only lookup, created using an INCLUDE clause
● Hash - equality checks
○ Uses hash of the key for very fast equality access
○ Creates a hash table with O(1) complexity
● BRIN (block range index) - space efficient for sorted tables
○ Stores min and max values of a block only
Indexes - Not a one-size-fits-all!

Don’t use indexes when:

● You need all or most of the data any ways

● Your workload is WRITE or UPDATE heavy with little READs
● Data bloat is a concern (‘over’ indexing)
● Your table is too small
Many performance features ‘just work’

A few examples …

● Parallel queries
○ The query planner decides if it can use multiple CPU cores to execute a single query
○ There are tuning parameters that you can adjust
● Heap-Only Tuples (HOT)
○ Avoids index updates if changes don’t impact an indexed column
● Incremental sort
○ Don’t start from scratch, sort only what is not yet sorted
● Autovacuum
○ Gets rid of dead tuples to clear out the table bloat
○ There are tuning parameters that you can adjust
 Performance Features
Key takeaways:
● Indexes are a powerful ally
● … but you shouldn’t overuse them
● Let PostgreSQL do its job
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
Parameter tuning

● Two broad categories of parameters

○ Allocation parameters
○ Cost definitions
Easily tuned database parameters - Allocation

● shared_buffers
○ Cache for frequently accessed data
○ Default is 128MB
○ Recommended is between 25% and 40% of system memory
● wal_buffers
○ Shared memory not yet written to disk
○ Default is 3% of shared_buffers
○ A value of up to 16MB can improve performance in high concurrency commits
● work_mem
○ Memory available for a query operation
○ Default is 4MB
○ High I/O activity for a query is an indicator that an increase in work_mem can help
○ Each parallel operation is allowed to use memory up to this value
Easily tuned database parameters - Costs

● cpu_tuple_cost
○ Cost of processing a single row of data, including operations like WHERE and JOIN
○ Default is 0.01
○ Lower values encourage query planner to process more rows, helpful for I/O bound operations
○ Higher values encourage query planner to process less rows, helpful for CPU bound operations
● random_page_cost
○ Cost of non-sequential disk page access
○ Default is 4.0
○ Lower values imply low cost for random access, encouraging index scans
○ Higher values imply high cost for random access, encouraging sequential scans
● effective_cache_size
○ Expected size of database cache, including shared buffers and OS cache
○ Default is 4GB (this is not an allocation)
○ Higher values imply more data in cache, encouraging index scans
○ Lower values imply less data in cache, encouraging sequential scans
 Parameter Tuning
Key takeaways:
● Tweak configuration parameters based on your
hardware and workload
● This will require some experimentation till you nail it
down
● Makes an ideal candidate for AI-based tuning
 2!
ke
Ta

PostgreSQL Performance
for Application Developers
Because There is no Magic Button

Zafin, Ottawa
03-Mar-2025
 Scaling
● Query & SQL Optimization
PostgreSQL ● Architectural Improvements
● Performance Features
Some tips for developers ● Parameter Tuning
 ● Vacuum and Dead Tuples
● Best practices for JOINs
Tips for Zafin ● INSERT performance
● Reading EXPLAIN plans
 ● Vacuum and Dead Tuples
Tips for Zafin ●
●
Best practices for JOINs
INSERT performance
● Reading EXPLAIN plans
The Vacuum Process in PostgreSQL

● For every update, a new row is

created and old row is marked
deleted
● Transactions hold snapshots that
are released when the transaction
completes
● The rows marked deleted are
called Dead Tuples
● Dead tuples are ‘cleaned up’ by
the vacuum process
● Autovacuum also runs ANALYZE
 Vacuum and Dead Tuples - Visualized

Credit: https://www.cs.cmu.edu/~pavlo/blog/2023/04/the-part-of-postgresql-we-hate-the-most.html
Mismanaging the Autovacuum Process

● Table bloat
● Poor performance
● Storage creep
● Inaccurate database statistics
Vacuum and Dead Tuples
Key takeaways:
● Never turn autovacuum off
● ‘Idle in transaction’ holds a lock that prevents vacuum
● Long running transactions are a killer for vacuum
● There is a problem in autovacuum configuration if:
○ Vacuum process is taking too long to complete
○ Vacuum process is taking up too many resources
 ● Vacuum and Dead Tuples
● Best practices for JOINs
Tips for Zafin ● INSERT performance
● Reading EXPLAIN plans
JOINs Explained
Common Pitfalls in JOINs

● Filter early to avoid nested loops

● Cartesian product is your enemy
 ● Vacuum and Dead Tuples
● Best practices for JOINs
Tips for Zafin ● INSERT performance
● Reading EXPLAIN plans
When INSERTing Lots of Data …

● COPY > Batch Inserts > INSERT

● https://www.timescale.com/learn/testing-postgres-ingest-insert-vs-batch
-insert-vs-copy
 ● Vacuum and Dead Tuples
● Best practices for JOINs
Tips for Zafin ● INSERT performance
● Reading EXPLAIN plans
The Query
insert
into
rp_s_event (id_,
entityidentifier_,
entitytype_,
eventtype_,
eventdate_,
groupingkey_)
select
nextval ('seq_rp_s_event') as id_ ,
at.accountidentifier_,
at.producttype_,
'ACCOUNT',
to_date('2020-07-31',
'YYYY-MM-DD'),
'DEFAULT' as groupingkey_
from
(
select
t.accountidentifier_,
t.producttype_
from
rd_r_transaction t
join rd_r_account a on
t.accountIdentifier_ = a.accountIdentifier_
join rd_r_account a2 on
coalesce(a.chargingaccount_,
a.accountidentifier_) = a2.accountidentifier_
join rd_r_billing_period bp on
a2.accountidentifier_ = bp.accountidentifier_
where
t.actualingestiondate_ between bp.startDate_ and to_date('2020-07-31',
'YYYY-MM-DD')
and a2.billingplancode_ is not null
and t.ingestiondate_< t.actualingestiondate_
and t.ingestiondate_ <= bp.endDate_
and bp.triggerDate_ = to_date('2020-07-31',
'YYYY-MM-DD')
and a.processgroupcode_ = 'DEFAULT'
group by
t.accountidentifier_,
t.producttype_) at;
The EXPLAIN Plan
Insert on rp_s_event (cost=2483104.21..2500256.04 rows=902728 width=1144) (actual time=5260010.099..5260010.103 rows=0 loops=1)
Buffers: shared hit=15609957 read=2579616
-> Subquery Scan on "*SELECT*" (cost=2483104.21..2500256.04 rows=902728 width=1144) (actual time=5260010.097..5260010.100 rows=0 loops=1)
Buffers: shared hit=15609957 read=2579616
-> Subquery Scan on at (cost=2483104.21..2493034.22 rows=902728 width=124) (actual time=5260010.097..5260010.099 rows=0 loops=1)
Buffers: shared hit=15609957 read=2579616
-> HashAggregate (cost=2483104.21..2485812.39 rows=902728 width=48) (actual time=5260010.096..5260010.099 rows=0 loops=1)
Group Key: t.accountidentifier_, t.producttype_
Batches: 1 Memory Usage: 24601kB
Buffers: shared hit=15609957 read=2579616
-> Nested Loop (cost=127249.43..2478590.57 rows=902728 width=48) (actual time=5260006.819..5260006.822 rows=0 loops=1)
Join Filter: ((t.actualingestiondate_ >= bp.startdate_) AND (t.ingestiondate_ <= bp.enddate_))
Buffers: shared hit=15609957 read=2579616
-> Nested Loop (cost=127248.85..893946.78 rows=273910 width=49) (actual time=17759.886..74895.989 rows=170541 loops=1)
Join Filter: ((bp.accountidentifier_)::text = (a2.accountidentifier_)::text)
Buffers: shared hit=2613198 read=429620
-> Hash Join (cost=127248.30..533230.84 rows=540391 width=130) (actual time=17734.325..30932.830 rows=520219 loops=1)
Hash Cond: ((COALESCE(a.chargingaccount_, a.accountidentifier_))::text = (bp.accountidentifier_)::text)
Buffers: shared hit=3439 read=415039
-> Seq Scan on rd_r_account a (cost=0.00..393199.18 rows=4869851 width=82) (actual time=584.713..11365.997 rows=4853552 loops=1)
Filter: ((processgroupcode_)::text = 'DEFAULT'::text)
Buffers: shared hit=3438 read=362977
-> Hash (cost=124505.70..124505.70 rows=498653 width=48) (actual time=17148.393..17148.394 rows=503699 loops=1)
Buckets: 524288 Batches: 1 Memory Usage: 44196kB
Buffers: shared hit=1 read=52062
-> Index Scan using idx_billing_prd_trggr_dt on rd_r_billing_period bp (cost=0.43..124505.70 rows=498653 width=48) (actual time=19.836..16986.981 rows=503699 loops=1)
Index Cond: (triggerdate_ = to_date('2020-07-31'::text, 'YYYY-MM-DD'::text))
Buffers: shared hit=1 read=52062
-> Index Scan using rd_r_account_pk on rd_r_account a2 (cost=0.56..0.66 rows=1 width=41) (actual time=0.084..0.084 rows=0 loops=520219)
Index Cond: ((accountidentifier_)::text = (COALESCE(a.chargingaccount_, a.accountidentifier_))::text)
Filter: (billingplancode_ IS NOT NULL)
Rows Removed by Filter: 1
Buffers: shared hit=2607539 read=14581
-> Index Scan using rd_r_txn_acct_actingdt_idx on rd_r_transaction t (cost=0.57..5.35 rows=55 width=56) (actual time=30.402..30.402 rows=0 loops=170541)
Index Cond: (((accountidentifier_)::text = (a.accountidentifier_)::text) AND (actualingestiondate_ <= to_date('2020-07-31'::text, 'YYYY-MM-DD'::text)))
Filter: (ingestiondate_ < actualingestiondate_)
Rows Removed by Filter: 85
Buffers: shared hit=12996759 read=2149996
Planning:
Buffers: shared hit=457 read=117
Planning Time: 493.580 ms
JIT:
Functions: 41
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 4.740 ms, Inlining 36.579 ms, Optimization 337.047 ms, Emission 204.066 ms, Total 582.432 ms
Execution Time: 5260052.101 ms
… or you could just use the tools available

1. https://explain.depesz.com/
2. https://explain.dalibo.com/
Conclusion
Database performance involves a
lot of variables. Optimize how
data is accessed before scaling
by credit card!
Questions?

pg_umair