2ndQuadrant
PostgreSQL Solutions for the Enterprise
14th Feb 2019
Nikhil Sontakke
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
pglogical
PostgreSQL to PostgreSQL replication
● Logical replication (one direction only)
● De-facto tool for
○ Data Movement
○ Migration
○ Low Downtime Upgrade
● Pglogical 2.2 available as binary and also now
available on Amazon RDS by popular demand
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Pglogical
Releases and Plans
● Pglogical 2.2 available now
○ Supports PostgreSQL 11 and earlier
○ Will continue to be supported for 3 years
○ Much community interest and patches - thanks!
● Pglogical 3
○ Future architecture for PostgreSQL 12+
○ Many new features
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Pglogical 3
● New architecture for performance
○ Split Receiver/Apply processes, so now 2 procs
○ Similar to WALReceiver/Startup process in
physical streaming replication
● Pluggable Architecture
○ Allows heterogenous Source(s)/Target(s)
○ Kafka, RabbitMQ targets available
○ Bulk-mode for high volume data loads
● Parallel initial copy on roadmap for pglogical 3
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Postgres-BDR3
Future of multi-master database for PostgreSQL
● BDR is Bi-Directional Replication (BDR)
2-way/Multi-way logical replication
● Postgres-BDR3 runs on PgLogical3
● Coordinated code bases, operating as a full stack
● Streamlines feature development and support
● New feature releases quarterly
● BDR and pglogical 3.5.1 now available
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Postgres-BDR
Trusted multi-master database for PostgreSQL
● Postgres-BDR 3
○ Future architecture
● Postgres-BDR 2
○ Upgrade path to BDR3 due 3Q2019
○ De-supported by mid 2020
● Postgres-BDR 1
○ Upgrade path to BDR3 due 1Q2019
○ De-supported by end 2019
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Postgres-BDR3
Future of multi-master database for PostgreSQL
● Postgres-BDR Standard Edition
○ Runs as an Extension on
Community PostgreSQL 10+
● Postgres-BDR Enterprise Edition
○ Runs as an Extension,
using minimal additional features in
2ndQuadrant PostgreSQL 11+
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Using Postgres-BDR
Transparent multi-master database for PostgreSQL
● Connect to any node
● Read and Write SQL like standard PostgreSQL
● Create/Manage tables like standard PostgreSQL
● Options to control which tables are replicated
● Some minor restrictions on DDL
● Design considerations for distributed database
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Writing to Postgres-BDR
Distributed database options
● Post-Commit Synchronization
Resolve issues after COMMIT
○ Conflict-Free Custom Datatypes (CRDTs)
○ Row-level Conflict Handling by default
○ Column-level Conflict Handling option
○ Logging and resolution of issues
○ Conflict Triggers
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Writing to Postgres-BDR
Distributed database options
● Pre-Commit Synchronization
Eager Replication avoids conflicts
○ All Nodes
● Avoids issues at COMMIT
○ Additional latency not desirable in many cases
○ Some transaction aborts in conflict cases
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Cluster Management
Enhancements for Distributed PostgreSQL
● Node Roles
○ Send/Receive Node (default)
○ Receive-Only Node
● Rep Sets (node subsets)
● Rolling Cross-Version Upgrades
○ One-by-one upgrades with zero downtime
○ Minimal APIs designed to allow upgrades
○ Across BDR releases and Postgres major releases
○ Fully tested to ensure compatibility
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Security
Cloud and High Security use cases
● BDR changes are applied as the table-owning user
● BDR functions do not need to be executed as
superuser, except for installation
● Default roles allow principle of least privilege
● Secure access to BDR catalog tables
for non-admin users
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Robustness and Supportability
When it’s critical, you can count on us!
● Active-Active architecture offers Very High Availability
● Each master has multiple protection options
○ Logical Standby(s)
○ Physical Standby(s) (or both at once)
● Synchronous and asynchronous
Replication options
● Commit At Most Once option
● Trace facility for investigation of
complex bugs in distributed system
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Application Maintenance
DevOps Ease of Use
● Transparent handling of DDL for partitioned tables
● Optional modes for very detailed specification of node
targets, allowing DDL even with down nodes
● Allows differing indexes on different nodes,
allowing nodes for Business Intelligence and
New Application-Version Testing
● Stream Triggers for filtering and transformation
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Operations
Practical experience
● Clearly identify node state, even when disconnected
● Monitor current lag time and lag bytes, even for down nodes
● Options for controlling down-node effects
● Error handling and controls
● Postgres-BDR plugin for
OmniDB also available
● Postgres Cloud Manager
with cluster visualization
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Postgres-BDR Plugin for OmniDB
Visual Administration
● Clearly identify node state, even when disconnected
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Performance
Real-World Production Performance
● Efficient logical replication
● Streaming of large transactions
● Efficient distributed sequences
● Fast node addition
● Choice of options for selecting appropriate
robustness and performance trade-offs
● Put data where its needed,
keep it fresh easily and automatically
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
BDR Multi-node Query
Consistency and Performance
● Timestamp-based snapshots
● Allow consistent queries across nodes
even with real-time replication of data
● Data verification between nodes
● Multi-node parallel query (MPP) across
○ Local clusters with remote DR nodes
○ Geo-distributed clusters
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Full Stack Integration
Working together as one
● Fully integrated stack
○ Full automatic testing of operations and upgradability
○ 258 pages of detailed documentation
● Postgres-BDR, pglogical, 2QPG
● Repmgr, Barman
● PCISecurityPack and other tools
● OmniDB
● TPAexec Cloud/On-Premise Orchestration
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
Postgres-BDR3 Summary
Major new functionality, born out of production
experience
● Cluster Management
● Security
● Robustness and Supportability
● Application Maintenance
● Operations
● Performance
● Rapid, agile development from many user requests
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise
2ndQuadrant
PostgreSQL Solutions
Website https://www.2ndquadrant.com/
Blog https://blog.2ndquadrant.com/
Email [email protected]
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
