Future Postgres Challenges

BRUCE MOMJIAN

This presentation explores possible challenges to Postgres’s success in the coming years.

https://momjian.us/presentations Creative Commons Attribution License

Last updated: April 2023

1 / 25
 Outline

1. Current status
2. Project challenges
3. Competitive challenges
4. Technical challenges

2 / 25
1. Current Status

https://www.flickr.com/photos/snikologiannis/

3 / 25
 Consistent Development

• 35+ years of development

• 25+ years of annual major releases
• ~180 features per major release
• Quarterly minor releases

4 / 25
 Healthy Community Structure

• BSD license guarantees software will be available forever, including for proprietary
use.
• Development and leadership is diversified geographically, culturally, and is
multi-company.

5 / 25
Strong Diversified Assistance

Sarah Conway, EDB

6 / 25
Innovative Features

Keith Alsheimer, EDB, 2013

7 / 25
Most Loved Relational Database in 2020

https://insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-databases-loved4

8 / 25
2. Project Challenges

https://www.flickr.com/photos/croydonclicker/

9 / 25
 Leadership Disruption

• Gimp was abandoned by its lead developers, later resurrected

• Red Hat took over CentOS, changed stability

10 / 25
 Poor Reputation

• Security flaws
• Buggy releases
• Instability
• Poor performance
• Data corruption

11 / 25
 Patent Attacks

• Developer with patents, Rambus

• Competitor with patents, Microsoft
• Patent trolls, Rothschild Patent Imaging LLC
• Good news
• Open Invention Network
• Unified Patents
• Project Jengo at Cloudflare

12 / 25
 Identity Challenges

• Domain name
• Website
• Trademark

13 / 25
 Cloud Vendor Starvation

• Cloud vendors use open source as upsell

• Already have infrastructure-as-a-service relationship with customers
• Company-controlled open source already impacted, changed licenses
• Red Hat challenged by cloud vendors

https://momjian.us/main/blogs/pgblog/2020.html#September_25_2020

https://momjian.us/main/blogs/pgblog/2020.html#September_28_2020

14 / 25
3. Competitive Challenges

https://www.flickr.com/photos/oui-ennui/

15 / 25
 Decline of Relational

• Relational storage was proposed by E. F. Codd in

1970
• 50+ years still in use
• Very flexible
• Resisted challenges
• XML databases
• Object databases
• NOSQL

https://en.wikipedia.org/wiki/Edgar_F._Codd

16 / 25
 Other Solutions

• Relational (e.g., MariaDB)

• Embedded (SQLite)
• Document (MongoDB)
• Columnar (ClickHouse)
• Data warehouse (Hadoop)
• Full text search (Elasticsearch)
• Time series (InfluxDB)

17 / 25
 The Rise of Forks

• Forks of Postgres go back to the early 1990’s with

Illustra
• Popular fork goals
• cloud customization
• horizontal scaling
• data warehouse
• BSD split into FreeBSD, NetBSD, and OpenBSD
• Egcs forked gcc, became popular, later became the
new gcc

https://raw.github.com/daamien/artwork/master/inkscape/PostgreSQL_timeline/timeline_postgresql.png
https://wiki.postgresql.org/wiki/PostgreSQL_derived_databases
18 / 25
4. Technical Challenges

https://www.flickr.com/photos/afc16/

19 / 25
 Write Amplification

• Non-HOT updates can cause massive index updates

• Dead and old row version cleanup can become expensive for certain workloads
• Writes cause full page image and hint WAL writes
• Freezing of old transaction ids
• Incremental improvements
• Are radical improvements needed?

20 / 25
 Cluster File Encryption, TDE

• Newer versions of the PCI DSS specification make storage-only encryption less
acceptable
• This is a check-box requirement for many financial deployments
• Development is in progress

https://wiki.postgresql.org/wiki/Transparent_Data_Encryption

21 / 25
 Horizontal Scaling

SQL Queries

• Allows data storage larger than possible on a single

server
• Allows write scaling PG FDW

• Enables large CPU and memory scaling SQL Queries

• Development is in progress
Foreign Server Foreign Server Foreign Server

https://momjian.us/main/writings/pgsql/sharding.pdf

22 / 25
 Obsolete Toolchain

Difficulty replacing obsolete or abandoned:

• Programming languages
• Support libraries
• Testing frameworks

23 / 25
 Drastic Technology Changes

• New language, architecture, or storage that are difficult for Postgres to adopt
• Technology changes have happened before
• SSDs, added random_page_cost to tablespaces
• virtual machines, containers, cloud

24 / 25
 Conclusion

https://momjian.us/presentations https://www.flickr.com/photos/91451979@N00/

25 / 25