postgres-xc-general Mailing List for Postgres-XC

ah, thank you. that explains a lot. I am including the steps required to
ADD NODE and DELETE NODE from each table replica set as required.

so what happens if DISTRIBUTE BY REPLICATION is set for a table, and then
one of those datanodes loses connectivity? my initial testing indicates
that the rest of the cluster doesn't seem to notice a change, and data is
still accessible on a cluster-wide level (at which point it could be
possible to cleanly remove a malfunctioning node). is this not correct?

On Thu, Jan 8, 2015 at 5:01 PM, 鈴木 幸市 <ko...@in...> wrote:

>  Generally, XC itself does not provide a feature to deal with node
> failure.   You should configure slave of each node based upon log shipping.
>
>  Adding slaves can be done with “add node” command of pgxc_ctl.
>
>  Even with “distribute by replication”, XC requires all the replica are
> alive.
>  ---
> Koichi Suzuki
> NTT DATA Intellilink Corp.
> ko...@in...
>
>  2015/01/09 8:59、WilliamKray QFor <wil...@gm...> のメール：
>
>  Hello,
>
>  Apologies if these things have been discussed thousands of times before,
> but I'm feeling a bit frustrated.
>
>  Background:
> I'm running pgxc 1.2.1, and attempting to automate deploying a cluster in
> AWS. AWS, as we know, likes to drop servers without a lot of warning, and
> all kinds of bad things happen, so my main goal is having consistent data
> in the cluster in the event of a node failure, as well as easily adding a
> new node to replace the dropped one, or expand the cluster.
>
>  QUESTION 1:
> I'm under the impression that every table created in the database must be
> created with the DISTRIBUTE BY REPLICATION statement, if I want all data to
> be replicated, with no option to set that as the default method of
> distribution. Is this correct?
>
>
>  Please see the above comment.
>
>
>  QUESTION 2:
> I've also (in testing) created a database using some commands like the
> following:
> CREATE TABLE test (id int, data char(100));
> ALTER TABLE test DISTRIBUTE BY REPLICATION;
> misc. insert query here;
>
>  I then go to another node, and I can *access* all the data, but it is *not
> replicated to that node*. If I run:
> ALTER TABLE TO NODE (nodename);
>
>  it then appears to have data replicated to that node (as in, if I run EXECUTE
> DIRECT queries on that datanode, I can now see the content living on that
> node). BUT, when I remove any node from the cluster by running DROP NODE,
> suddenly none of the nodes have any data and the table is completely empty!
> HELP!
>
>
>  DROP NODE does not take care of the data.   It simply drops the node from
> pgxc_node system catalog.   If the dropped node is involved in any table,
> it will be suffered from catalog inconsistency, that is, pgxc_class
> contains dangling node information.   Before you issue DROP NODE, you
> should have issued ALTER TABLE DELETE NODE against each table dropping node
> is involved.
>
>   Are there more steps required to remove a node from a cluster? and if
> so, how do I configure my cluster to be resilient to node failure or
> unexpected server termination?
>
>  QUESTION 3:
> I've been looking into Postgres-XL, this appears to have a couple more
> features than PGXC, and more recent activity on their website, but also
> looks like it's the same codebase. Is XC still being maintained, or has
> development shifted to XL? Should I switch?
>
>
>  XL is, in fact, spin-off from XC.   Most of the code basis is the same.
> They’re still based upon PG 9.2 and does not have some new feature in 9.3.
> Anyway, they’re almost the same.
>
>
>  Thank you so much for your input. If this email is too long, please let
> me know and I'll break it up into more manageable chunks.
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming! The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is
> your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now.
> http://goparallel.sourceforge.net_______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>

Generally, XC itself does not provide a feature to deal with node failure.   You should configure slave of each node based upon log shipping.

Adding slaves can be done with “add node” command of pgxc_ctl.

Even with “distribute by replication”, XC requires all the replica are alive.
---
Koichi Suzuki
NTT DATA Intellilink Corp.
ko...@in...<mailto:ko...@in...>.jp

2015/01/09 8:59、WilliamKray QFor <wil...@gm...<mailto:wil...@gm...>> のメール：

Hello,

Apologies if these things have been discussed thousands of times before, but I'm feeling a bit frustrated.

Background:
I'm running pgxc 1.2.1, and attempting to automate deploying a cluster in AWS. AWS, as we know, likes to drop servers without a lot of warning, and all kinds of bad things happen, so my main goal is having consistent data in the cluster in the event of a node failure, as well as easily adding a new node to replace the dropped one, or expand the cluster.

QUESTION 1:
I'm under the impression that every table created in the database must be created with the DISTRIBUTE BY REPLICATION statement, if I want all data to be replicated, with no option to set that as the default method of distribution. Is this correct?

Please see the above comment.


QUESTION 2:
I've also (in testing) created a database using some commands like the following:
CREATE TABLE test (id int, data char(100));
ALTER TABLE test DISTRIBUTE BY REPLICATION;
misc. insert query here;

I then go to another node, and I can access all the data, but it is not replicated to that node. If I run:
ALTER TABLE TO NODE (nodename);

it then appears to have data replicated to that node (as in, if I run EXECUTE DIRECT queries on that datanode, I can now see the content living on that node). BUT, when I remove any node from the cluster by running DROP NODE, suddenly none of the nodes have any data and the table is completely empty! HELP!

DROP NODE does not take care of the data.   It simply drops the node from pgxc_node system catalog.   If the dropped node is involved in any table, it will be suffered from catalog inconsistency, that is, pgxc_class contains dangling node information.   Before you issue DROP NODE, you should have issued ALTER TABLE DELETE NODE against each table dropping node is involved.

Are there more steps required to remove a node from a cluster? and if so, how do I configure my cluster to be resilient to node failure or unexpected server termination?

QUESTION 3:
I've been looking into Postgres-XL, this appears to have a couple more features than PGXC, and more recent activity on their website, but also looks like it's the same codebase. Is XC still being maintained, or has development shifted to XL? Should I switch?

XL is, in fact, spin-off from XC.   Most of the code basis is the same.   They’re still based upon PG 9.2 and does not have some new feature in 9.3.
Anyway, they’re almost the same.


Thank you so much for your input. If this email is too long, please let me know and I'll break it up into more manageable chunks.
------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hello,

Apologies if these things have been discussed thousands of times before,
but I'm feeling a bit frustrated.

Background:
I'm running pgxc 1.2.1, and attempting to automate deploying a cluster in
AWS. AWS, as we know, likes to drop servers without a lot of warning, and
all kinds of bad things happen, so my main goal is having consistent data
in the cluster in the event of a node failure, as well as easily adding a
new node to replace the dropped one, or expand the cluster.

QUESTION 1:
I'm under the impression that every table created in the database must be
created with the DISTRIBUTE BY REPLICATION statement, if I want all data to
be replicated, with no option to set that as the default method of
distribution. Is this correct?

QUESTION 2:
I've also (in testing) created a database using some commands like the
following:
CREATE TABLE test (id int, data char(100));
ALTER TABLE test DISTRIBUTE BY REPLICATION;
misc. insert query here;

I then go to another node, and I can *access* all the data, but it is *not
replicated to that node*. If I run:
ALTER TABLE TO NODE (nodename);

it then appears to have data replicated to that node (as in, if I run EXECUTE
DIRECT queries on that datanode, I can now see the content living on that
node). BUT, when I remove any node from the cluster by running DROP NODE,
suddenly none of the nodes have any data and the table is completely empty!
HELP!

Are there more steps required to remove a node from a cluster? and if so,
how do I configure my cluster to be resilient to node failure or unexpected
server termination?

QUESTION 3:
I've been looking into Postgres-XL, this appears to have a couple more
features than PGXC, and more recent activity on their website, but also
looks like it's the same codebase. Is XC still being maintained, or has
development shifted to XL? Should I switch?

Thank you so much for your input. If this email is too long, please let me
know and I'll break it up into more manageable chunks.