postgres-xc-developers Mailing List for Postgres-XC

On Fri, Sep 14, 2012 at 1:46 AM, David E. Wheeler <da...@ju...>wrote:

> > In the case of a Postgres-XC cluster, you need a GTM node to feed each
> master node of cluster with GXIDs and snapshot, so it is highly advised to
> keep GTM node at the same place as the other masters to avoid having a
> master having its request going from one data center to another when
> getting global snapshot and GXID. This will impact performance depending on
> the network speed between the 2 centers.
>
> Yeah. For multi-data center configuration, I think some way of having GTMs
> clustered would be useful. That is, say I have 3 data centers, with
> coordinators and data nodes in each. It'd be awesome to also have GTMs in
> each, and let the GTMs coordinate between themselves.
>
In this configuration case, you will still need a common point for all the
GTMs to be sure that they share the same global view.
By the way, you can already use GTM-Proxy to group the messages sent to
GTM. This will reduce the amount of data exchanged between the data
centers, at least for the stuff related to MVCC.


>
> > But the amount of data exchanged between nodes and GTM is limited with
> only GXID, snapshots, timestamps and sequence values, so this is limited
> amount of data compared of the possible quantity of table data exchanged
> between nodes for a JOIN done on local nodes.
>
> With the system I am designing now, all the data would be in both data
> centers, so there would be no need to join across data centers. So maybe
> GTM traffic on a dedicated link would be adequate? I'm assuming that the
> replication between the replicated data nodes would be much higher.
>
If you do not need to join data among centers, so yes XC would perform well
it is more adapted for OLTP type applications now.
We have ideas to extend that though.
Also, you can use the preferred node feature to relocate the read on a
replicated table on a node that you would like to point to in priority.
In the case of your application, I would imagine that you got, for example,
1 Coordinator and 1 Datanode on each data center (perhaps more), but you
can force a Coordinator to read data of a replicated table on a node that
is located in the same data center.

You can set up that with "ALTER NODE nodename WITH (PREFERRED)".
This will be a real gain in your case.


> So if I understood correctly your case, and that the masters are located
> on different places, well it will be OK.
>
> But XC was not designed with that in mind, right?
>
Not that much. Like an Oracle RAC application, all the servers should be
located at the same place.
But this is not mandatory to my mind depending on the applications used,
especially OLTP without write-scalability needs.
-- 
Michael Paquier
http://michael.otacoo.com

On Sep 12, 2012, at 11:47 PM, Michael Paquier <mic...@gm...> wrote:

> OK, so your masters will be located on different data centers with their standbys?

Yes.

> In the case where all the masters are located at the same place, and all the standbys are located on a different place, well that will make it as all the write operations will happen at the same place.

Yes, but if I was going to do that, I wouldn't bother with PGXC for this app. The database will be relatively small, does not need partitioning or sharding. I would just set up a PostgreSQL master in one data center and a standby in the other.

> In the case of a Postgres-XC cluster, you need a GTM node to feed each master node of cluster with GXIDs and snapshot, so it is highly advised to keep GTM node at the same place as the other masters to avoid having a master having its request going from one data center to another when getting global snapshot and GXID. This will impact performance depending on the network speed between the 2 centers.

Yeah. For multi-data center configuration, I think some way of having GTMs clustered would be useful. That is, say I have 3 data centers, with coordinators and data nodes in each. It'd be awesome to also have GTMs in each, and let the GTMs coordinate between themselves.

> But the amount of data exchanged between nodes and GTM is limited with only GXID, snapshots, timestamps and sequence values, so this is limited amount of data compared of the possible quantity of table data exchanged between nodes for a JOIN done on local nodes.

With the system I am designing now, all the data would be in both data centers, so there would be no need to join across data centers. So maybe GTM traffic on a dedicated link would be adequate? I'm assuming that the replication between the replicated data nodes would be much higher.

> So if I understood correctly your case, and that the masters are located on different places, well it will be OK.

But XC was not designed with that in mind, right?

> But if the masters are located on different places, low network load would help in having a good performance deal but I am worrying on the impact of GTM/node data exchange.

Yeah. We have a dedicated link, but still.

> What is also important in your case is that the amount of table data exchanged is small, so this is really good for XC btw.

What is good for XC here?

BTW, I am still hoping to one day see something like Cassandra's distributed model running on PostgreSQL. Maybe someday…

Thanks for your reply, much appreciated!

Best,

David

> Quick question. We are looking at building a database cluster with
> multiple masters to span two data centers, each with a master and a
> standby. Not a whole lot of traffic, less than 100G of data, no need for
> write scaling. What do you think? Would PGXC be good for such a thing, or
> should I go with Bucardo for now?
>
OK, so your masters will be located on different data centers with their
standbys?

In the case where all the masters are located at the same place, and all
the standbys are located on a different place, well that will make it as
all the write operations will happen at the same place.

In the case of a Postgres-XC cluster, you need a GTM node to feed each
master node of cluster with GXIDs and snapshot, so it is highly advised to
keep GTM node at the same place as the other masters to avoid having a
master having its request going from one data center to another when
getting global snapshot and GXID. This will impact performance depending on
the network speed between the 2 centers.
But the amount of data exchanged between nodes and GTM is limited with only
GXID, snapshots, timestamps and sequence values, so this is limited amount
of data compared of the possible quantity of table data exchanged between
nodes for a JOIN done on local nodes.

So if I understood correctly your case, and that the masters are located on
different places, well it will be OK.
But if the masters are located on different places, low network load would
help in having a good performance deal but I am worrying on the impact of
GTM/node data exchange.

What is also important in your case is that the amount of table data
exchanged is small, so this is really good for XC btw.
-- 
Michael Paquier
http://michael.otacoo.com