Scribd
Upload
307 views22 pages

Postgres BDR PDF

The document discusses pglogical and Postgres-BDR, open source tools for logical replication and multi-master replication in PostgreSQL. It provides information on recently released versions 3.5.1, upcoming features in pglogical 3 and Postgres-BDR 3, how Postgres-BDR allows for transparent multi-master databases, and integration of the tools and support from 2ndQuadrant.

Uploaded by

ANDALOUS
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
307 views22 pages

Postgres BDR PDF

The document discusses pglogical and Postgres-BDR, open source tools for logical replication and multi-master replication in PostgreSQL. It provides information on recently released versions 3.5.1, upcoming features in pglogical 3 and Postgres-BDR 3, how Postgres-BDR allows for transparent multi-master databases, and integration of the tools and support from 2ndQuadrant.

Uploaded by

ANDALOUS
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

2ndQuadrant

PostgreSQL Solutions for the Enterprise

14th Feb 2019

Nikhil Sontakke

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

pglogical
PostgreSQL to PostgreSQL replication

● Logical replication (one direction only)


● De-facto tool for
○ Data Movement
○ Migration
○ Low Downtime Upgrade
● Pglogical 2.2 available as binary and also now
available on Amazon RDS by popular demand

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Pglogical
Releases and Plans

● Pglogical 2.2 available now


○ Supports PostgreSQL 11 and earlier
○ Will continue to be supported for 3 years
○ Much community interest and patches - thanks!

● Pglogical 3
○ Future architecture for PostgreSQL 12+
○ Many new features

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Pglogical 3

● New architecture for performance


○ Split Receiver/Apply processes, so now 2 procs
○ Similar to WALReceiver/Startup process in
physical streaming replication
● Pluggable Architecture
○ Allows heterogenous Source(s)/Target(s)
○ Kafka, RabbitMQ targets available
○ Bulk-mode for high volume data loads
● Parallel initial copy on roadmap for pglogical 3

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Postgres-BDR3
Future of multi-master database for PostgreSQL

● BDR is Bi-Directional Replication (BDR)


2-way/Multi-way logical replication

● Postgres-BDR3 runs on PgLogical3


● Coordinated code bases, operating as a full stack
● Streamlines feature development and support
● New feature releases quarterly

● BDR and pglogical 3.5.1 now available


Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise

Postgres-BDR
Trusted multi-master database for PostgreSQL

● Postgres-BDR 3
○ Future architecture

● Postgres-BDR 2
○ Upgrade path to BDR3 due 3Q2019
○ De-supported by mid 2020
● Postgres-BDR 1
○ Upgrade path to BDR3 due 1Q2019
○ De-supported by end 2019

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Postgres-BDR3
Future of multi-master database for PostgreSQL

● Postgres-BDR Standard Edition


○ Runs as an Extension on
Community PostgreSQL 10+

● Postgres-BDR Enterprise Edition


○ Runs as an Extension,
using minimal additional features in
2ndQuadrant PostgreSQL 11+

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Using Postgres-BDR
Transparent multi-master database for PostgreSQL

● Connect to any node


● Read and Write SQL like standard PostgreSQL
● Create/Manage tables like standard PostgreSQL

● Options to control which tables are replicated


● Some minor restrictions on DDL
● Design considerations for distributed database

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Writing to Postgres-BDR
Distributed database options

● Post-Commit Synchronization
Resolve issues after COMMIT
○ Conflict-Free Custom Datatypes (CRDTs)
○ Row-level Conflict Handling by default
○ Column-level Conflict Handling option
○ Logging and resolution of issues
○ Conflict Triggers

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Writing to Postgres-BDR
Distributed database options

● Pre-Commit Synchronization
Eager Replication avoids conflicts
○ All Nodes

● Avoids issues at COMMIT


○ Additional latency not desirable in many cases
○ Some transaction aborts in conflict cases

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Cluster Management


Enhancements for Distributed PostgreSQL

● Node Roles
○ Send/Receive Node (default)
○ Receive-Only Node
● Rep Sets (node subsets)
● Rolling Cross-Version Upgrades
○ One-by-one upgrades with zero downtime
○ Minimal APIs designed to allow upgrades
○ Across BDR releases and Postgres major releases
○ Fully tested to ensure compatibility

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Security
Cloud and High Security use cases
● BDR changes are applied as the table-owning user
● BDR functions do not need to be executed as
superuser, except for installation
● Default roles allow principle of least privilege
● Secure access to BDR catalog tables
for non-admin users

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Robustness and Supportability


When it’s critical, you can count on us!

● Active-Active architecture offers Very High Availability


● Each master has multiple protection options
○ Logical Standby(s)
○ Physical Standby(s) (or both at once)
● Synchronous and asynchronous
Replication options
● Commit At Most Once option
● Trace facility for investigation of
complex bugs in distributed system

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Application Maintenance


DevOps Ease of Use
● Transparent handling of DDL for partitioned tables
● Optional modes for very detailed specification of node
targets, allowing DDL even with down nodes
● Allows differing indexes on different nodes,
allowing nodes for Business Intelligence and
New Application-Version Testing
● Stream Triggers for filtering and transformation

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Operations
Practical experience
● Clearly identify node state, even when disconnected
● Monitor current lag time and lag bytes, even for down nodes
● Options for controlling down-node effects
● Error handling and controls

● Postgres-BDR plugin for


OmniDB also available

● Postgres Cloud Manager


with cluster visualization
Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise

Postgres-BDR Plugin for OmniDB


Visual Administration
● Clearly identify node state, even when disconnected

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Performance
Real-World Production Performance

● Efficient logical replication


● Streaming of large transactions
● Efficient distributed sequences
● Fast node addition
● Choice of options for selecting appropriate
robustness and performance trade-offs

● Put data where its needed,


keep it fresh easily and automatically

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

BDR Multi-node Query


Consistency and Performance

● Timestamp-based snapshots
● Allow consistent queries across nodes
even with real-time replication of data
● Data verification between nodes

● Multi-node parallel query (MPP) across


○ Local clusters with remote DR nodes
○ Geo-distributed clusters

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Full Stack Integration


Working together as one
● Fully integrated stack
○ Full automatic testing of operations and upgradability
○ 258 pages of detailed documentation
● Postgres-BDR, pglogical, 2QPG
● Repmgr, Barman
● PCISecurityPack and other tools
● OmniDB

● TPAexec Cloud/On-Premise Orchestration

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com


2ndQuadrant
PostgreSQL Solutions for the Enterprise

Postgres-BDR3 Summary
Major new functionality, born out of production
experience

● Cluster Management
● Security
● Robustness and Supportability
● Application Maintenance
● Operations
● Performance

● Rapid, agile development from many user requests


Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com
2ndQuadrant
PostgreSQL Solutions for the Enterprise

2ndQuadrant
PostgreSQL Solutions

Website https://www.2ndquadrant.com/
Blog https://blog.2ndquadrant.com/
Email [email protected]

Pglogical and Postgres-BDR 3.5.1 https://www.2ndQuadrant.com

You might also like