Scribd
Upload
119 views

Postgres Database

This document discusses PostgreSQL database replication options and projects. It summarizes four main PostgreSQL replication projects: Usogres, eRServer, PostgreSQL Replicator, and Postgres-R. It then outlines the goals and roadmap of the PGReplication project, which aims to provide an open-source replication solution for PostgreSQL by building on theories from Postgres-R and other projects.

Uploaded by

Erick Pinenka
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
119 views

Postgres Database

This document discusses PostgreSQL database replication options and projects. It summarizes four main PostgreSQL replication projects: Usogres, eRServer, PostgreSQL Replicator, and Postgres-R. It then outlines the goals and roadmap of the PGReplication project, which aims to provide an open-source replication solution for PostgreSQL by building on theories from Postgres-R and other projects.

Uploaded by

Erick Pinenka
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 29

PostgreSQL Database

Replication Options

Darren Johnson
darren@up.hrcoxmail.com
The PGReplication Project
Agenda

Characterizing Replication
Replication Scenarios
PostgreSQL Replication
Postgres-R Concepts
PGReplication Project
Transaction Processing

• Transaction- a group of SQL commands whose


result will be made visible to the rest of the system as
a unit when the transaction commits--or not at all, if
the transaction aborts.

• Transaction Processing Application- a collection


of transaction programs designed to automate a
given business activity.

3
Critical Properties of a Transaction
ACID

• Atomicity-successful transaction commit; failed


transaction abort.
• Consistency-each transaction is programmed to
preserve database consistency.
• Isolation-each transaction is executed as if it were
running alone.
• Durability-the result of a committed transaction is
guaranteed to be on stable storage.

4
Updating the Database
w(x)
• Read Only (Primary
read- read-
copy / master - slave) only only

w(x) w(x) w(x)


• Peer to Peer (Update
everywhere / multi-
master)

5
Propagating Updates

• Asynchronous (Lazy / Store and Forward) - Post


commit sends information to all other systems
involved in the distribution.
– Trade Off: Data synchronization and conflict resolution

• Synchronous (Eager) – Pre commit sends


information to all other systems in the distribution and
verifies a commit or roll back on each transaction for
the entire distribution.
– Trade Off: Performance and scalability

6
More Characteristics

• Propagated as – SQL vs. Parsed (tuple)


• Event driven – Logs vs. Triggers
• What to replicate – Partial vs. Full
• Architecture - Embedded vs. External
• Where to replicate – Pre vs. Post

7
Scenario 1 (Hot Fail Over)

• The ability to fail a database from a


primary standalone server to a
secondary server.
– Usually done locally
– Hardware/Software solutions
– Multi-ported RAID
– Heartbeat
– WAL services must exist

8
Scenario 2 (Long Distance)

• Servers on different
coasts, and data needs
to be consistent San New
between them Diego York
– Fast local access
– Catastrophic failure
Orlando
– Data partitioned by
region

9
Scenario 3 (Cluster)

• Several servers in a
cluster instead of a
big mainframe
– Load balancing
– Fault resilience
– Scalability

10
Scenario 4 (Software Solution)

• Here the database has


little or no action in
the replication process
– Data warehouse
– Mobile users
– Reporting

11
PostgreSQL Replication Projects

• Usogres (Tesuichi Hosokawa)


• eRServer (Vadim Mikheev and Thomas
Lockhart @ PostgreSQL, Inc.)
• PostgreSQL Replicator (Matteo Cavalleri,
Rocco Prudentino @ IRCCS)
• Postgres-R (Bettina Kemme, Win Bausch,
Gustavo Alonso, Michael Baumer, Ignaz
Bachman, and others @ ETH Zurich)

12
Usogres

• Type: Full, Read Only, Pre-Process


• Location: usogres.good-day.net/
• Description:
– Real-time backup utility
– Replicates data via pre-postmaster
– Supports only one main server and backup
server

13
eRserver

• Type: Partial, Read Only, Transactional


• Location: http://www.erserver.com/
• Description:
– Supports snapshots (bundles changes)
– Uses SPI, Perl and PG_Perl interface
– SyncIDs are used to keep track of the
slave data updates

14
eRServer Desctiption Cont…

– Uses a replication table on master to


capture updates via trigger then
synchronization can be manual via
command line or by number of snapshots
– Only one slave and no fail over support

15
PostgreSQL Replicator

• Type: Partial, Peer-to-Peer, Async


• Location: pgreplicator.sourceforge.net
• Description:
– Robust update conflict detection and resolution
mechanism
– Creates a set of Replication Schema Tables
(RST) to store the replication rules
– Uses SP to dynamically capture triggers and
auxiliary tables, and act as an interface between
the DBA and the the replication engine
16
PostgreSQL Replicator Cont...

– TCL replication daemon running on each


system in the distribution allows database
synchronization to be started from any site
at any time.
– Uses triggers written in PL/TCL. The TCL
daemon uses PostgreSQL connectivity API
and TCL/DP libraries for communication
over TCP/IP

17
Postgres-R

• Type: Embedded, Peer-to-Peer, Sync


• Location: gborg.postgresql.org
• Description:
– Uses a “total order” group communication system
(multicasting updates)
– Shadow copies are used to enforce isolation
– Propagates tuple changes to decrease processing
on remote or query strings if too many tuples
changed

18
Postgres-R Cont…

• Implemented on PostgreSQL 6.4.2


• 4 branches of code (partial and
recovery)
• Only replicates one database

19
Postgres-R Goals

• To develop and apply appropriate


techniques in order to avoid previous
limitations of synchronous peer-to-peer
solutions
– Good performance (response time +
throughput)
– Consistent and fault tolerant
– Non-intrusive integration
20
Using Group Communication System

• Multicast
• Delivery order (FIFO, casual, total,etc.)
• Reliable delivery: all nodes vs. all
available nodes
• Membership control
• ISIS, Totem, Transis, Horus, Ensemble,
Spread
21
Ordering Transactions

• Two phase commit • Total order

22
Architecture of Postgres-R
original Client Client
Client PostgreSQL Client

Server Server
Postmaster

Local BE Local BE
Group
Communication Client
System

Replication Mgr Grp Comm Mgr

Server
Remote Remote Remote
BE BE BE
23
Basic Protocol
SITE A
• All operations are first
w(y)
r(z) performed locally at a
single site.
Local Phase
w(z) • Writes are sent in one
message to all sites at
w(z) Group the end of transaction.
Send Phase Communication • Write messages are
totally ordered.
Serial Phase l(z) • Serialization order
obeys total order (same
serialization order at all
Serial Phase l(y) sites).
• When receiving remote
writes, they must be
Write Phase w(y) applied

24
Basic Protocol - Conflict
SITE A SITE B
r(z) r(z) • Concurrency
Control must be
w(z) slightly modified.
w(z)
w(z)
w(z) Group
Communication
l(z) l(z) detect conflict
abort yellow
l(z) w(z)
detect conflict
abort yellow

25
PGReplication Project

• Goal – to provide a replication solution for


PostgreSQL which will meet most needs of
users and applications alike.
• Location – gborg.postgresql.org
• Provide information on current research
• Openly discuss all replication projects
and share ideas
• Invitation to combine efforts
26
PGReplication Roadmap

• Take the theories of Postgres-R and


implement them into the current version
of PostgreSQL using Spread as GCS
• Phase 0 – Full, Read Only, many slaves
• Phase I – Partial, Read Only, Recovery
• Phase II – Peer-to-peer, DDL

27
Improving PGReplication

• WAL for marshalling


• Using recovery play back mechanism
for PITR
• Asynchronous and data partitioning
• Bulk inserts
• Sequences
• Different architectures
28
Conclusions

• Understanding replication
characteristics can help determine the
correct solution to a problem.
• Many solutions for replication not one
will fit all needs, so we need to be able
to support all of them.

29

You might also like