postgres-xc-general Mailing List for Postgres-XC

After configuring and installing pg-xc from git trunk by
./configure --prefix=$HOME/pgxc && make && make install

I try to run the GTM:

filip@srv:~$ pgxc/bin/gtm_ctl -Z gtm -D pgxcdata/gtm0 start
server starting
filip@srv:~$ sh: 1: gtm: not found



pg_ctl does not have this problem. It knows that it should call
$HOME/pgxc/bin/postgres



PS. Please advise  - is this approppriate to file a bug report in
sourceforge for such minor issue?

On Thu, Jan 31, 2013 at 3:12 AM, Michael Paquier
<mic...@gm...>wrote:

> On Wed, Jan 30, 2013 at 8:01 PM, Christophe Le Roux <
> chr...@fr...> wrote:
>
>>                  Can we distribute and replicate a table ? (A table
>> distributed on 3 datanodes (for performance) and replicated on 3 others
>> datanode(for availability without down time) – I’m not talking about hot
>> standby or streaming replication included in standard postgresql)
>>
>  This is not supported directly, but you could try something like that,
> perhaps it will work that's just an idea. This idea uses as a base the
> internal Postgres partitioning mechanism:
> CREATE TABLE tab (col1 int, col2 int) DISTRIBUTE BY REPLICATION;
> CREATE TABLE tab1 (CHECK col1 >= 100 AND CHECK col1 < 200) INHERITS (tab)
> *DISTRIBUTE BY HASH(col1);*
> CREATE TABLE tab2 (CHECK col1 >= 200 AND CHECK col1 < 300) INHERITS (tab)
> *DISTRIBUTE BY REPLICATION;*
> Putting an index on column col1 would be good also. Here the secret is to
> have the parent table replicated.
>
I forgot something here:
You will need also to create a rule or a trigger to redirect
INSERT/UPDATE/DELETE to the correct subtable. Have a rule at the postgres
documentation here:
http://www.postgresql.org/docs/9.2/static/ddl-partitioning.html.

-- 
Michael Paquier
http://michael.otacoo.com

On Wed, Jan 30, 2013 at 8:01 PM, Christophe Le Roux <
chr...@fr...> wrote:

>                  Can we distribute and replicate a table ? (A table
> distributed on 3 datanodes (for performance) and replicated on 3 others
> datanode(for availability without down time) – I’m not talking about hot
> standby or streaming replication included in standard postgresql)
>
 This is not supported directly, but you could try something like that,
perhaps it will work that's just an idea. This idea uses as a base the
internal Postgres partitioning mechanism:
CREATE TABLE tab (col1 int, col2 int) DISTRIBUTE BY REPLICATION;
CREATE TABLE tab1 (CHECK col1 >= 100 AND CHECK col1 < 200) INHERITS
(tab) *DISTRIBUTE
BY HASH(col1);*
CREATE TABLE tab2 (CHECK col1 >= 200 AND CHECK col1 < 300) INHERITS
(tab) *DISTRIBUTE
BY REPLICATION;*
Putting an index on column col1 would be good also. Here the secret is to
have the parent table replicated.
-- 
Michael Paquier
http://michael.otacoo.com

Big up for questions that Kriss_fr asked.

At our company we are considering a very similar setup.
One of main advantages that draws us into even considering Postgres-XC, is
possibility of having 2 datacenters in "active-active" layout.

Thanks,
Filip



On Wed, Jan 30, 2013 at 7:13 AM, Christophe Le Roux <
chr...@fr...> wrote:

> Hi Pavan,
>
> The network between 2 DC is 1Gb/s with low latency ( like a LAN ). So this
> is not an issue.
>
> Ashutosh say if B or C fail (all table of all databases are "DISTRIBUTE by
> REPLICATION" - B and C should have same data), postgres-XC fail (for this
> release). It is on the roadmap ? or already on the master branch of github ?
>
> Thanks.
>
> Kriss_fr
>
> -----Message d'origine-----
> De : Pavan Deolasee [mailto:pav...@gm...]
> Envoyé : mercredi 30 janvier 2013 13:56
> À : Christophe Le Roux
> Cc : Ashutosh Bapat; pos...@li...
> Objet : Re: [Postgres-xc-general] Table distribution/replication and other
> things
>
> On Wed, Jan 30, 2013 at 5:46 PM, Christophe Le Roux <
> chr...@fr...> wrote:
> > Thank you for your answers.
> >
> >
> >
> > About the third question, I don't want to use standby server for now
> > (except for GM standby).
> >
> > The cluster would be 4 servers :
> >
> >
> >
> > A - GTM server
> >
> > B - GTM proxy, coordinator and datanode server
> >
> > C - GTM proxy, coordinator and datanode server
> >
> > D - GTM standby server
> >
> >
> >
> > All tables of all databases are "DISTRIBUTE by REPLICATION"
> > (postgres-XC capability). So we have same data at the same time on
> datanodes B and C.
> >
> > So if B server fail , C server will do the job.
> >
> > If C server fail, B server will do the job.
> >
> > B and C server talk to A server (for GxId, sequence,..) but if A
> > server fail, we can promote D server to the master GTM.
> >
> >
> >
> > Now, we suppose that : server A and B are located in DC1, and serveur
> > C and D are located in DC2.
> >
> > B and C will do the jobs and talk to A server (for GxId, sequence,..).
> >
>
> If I read correctly, B and C will be in different DCs which typically mean
> that they will connected on a slow WAN link. Since for replication,
> coordinator needs to send a lot of information to/for the datanode, I am
> not sure if this would work well. But then it depends on how much workload
> you are really putting on the server. It will also depend on what kind of
> queries you would be running. I don't think anyone has tried (yet) to keep
> the datanodes on different DCs, but give it a try.
>
> Thanks,
> Pavan
> --
> Pavan Deolasee
> http://www.linkedin.com/in/pavandeolasee
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_jan
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

Hi Pavan,

The network between 2 DC is 1Gb/s with low latency ( like a LAN ). So this is not an issue.

Ashutosh say if B or C fail (all table of all databases are "DISTRIBUTE by REPLICATION" - B and C should have same data), postgres-XC fail (for this release). It is on the roadmap ? or already on the master branch of github ?

Thanks.

Kriss_fr

-----Message d'origine-----
De : Pavan Deolasee [mailto:pav...@gm...] 
Envoyé : mercredi 30 janvier 2013 13:56
À : Christophe Le Roux
Cc : Ashutosh Bapat; pos...@li...
Objet : Re: [Postgres-xc-general] Table distribution/replication and other things

On Wed, Jan 30, 2013 at 5:46 PM, Christophe Le Roux <chr...@fr...> wrote:
> Thank you for your answers.
>
>
>
> About the third question, I don't want to use standby server for now 
> (except for GM standby).
>
> The cluster would be 4 servers :
>
>
>
> A - GTM server
>
> B - GTM proxy, coordinator and datanode server
>
> C - GTM proxy, coordinator and datanode server
>
> D - GTM standby server
>
>
>
> All tables of all databases are "DISTRIBUTE by REPLICATION" 
> (postgres-XC capability). So we have same data at the same time on datanodes B and C.
>
> So if B server fail , C server will do the job.
>
> If C server fail, B server will do the job.
>
> B and C server talk to A server (for GxId, sequence,..) but if A 
> server fail, we can promote D server to the master GTM.
>
>
>
> Now, we suppose that : server A and B are located in DC1, and serveur 
> C and D are located in DC2.
>
> B and C will do the jobs and talk to A server (for GxId, sequence,..).
>

If I read correctly, B and C will be in different DCs which typically mean that they will connected on a slow WAN link. Since for replication, coordinator needs to send a lot of information to/for the datanode, I am not sure if this would work well. But then it depends on how much workload you are really putting on the server. It will also depend on what kind of queries you would be running. I don't think anyone has tried (yet) to keep the datanodes on different DCs, but give it a try.

Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

On Mon, Jan 28, 2013 at 7:23 PM, Koichi Suzuki <koi...@gm...>wrote:

> I have not looked into it very in detail.   So this is just a guess.
>
> XC's datanode cluster contains three more system catalogues,
> pgxc_class, pgxc_node and pgxc_nodegroup and also contains many more
> build-in functions which have to be hard-coded.
>


Only adding new objects is fine. I think it does not prevent this kind of
migration.

I can think of creating new objects in pg_catalog just before switching to
Postgres-XC.

Does postgres-xc modify any existing postgres catalog tables? (adding
columns, changing types, any structure changes)?

Does postgres-xc modify any existing postgres catalog functions? (function
signatures and/or internal logic)

Do you think it's possible to map/translate existing pg_class to
pgxc_class, existing TXIDs to GXIDs, etc?

I understand that pointing "empty" GTMs and Coordinators to this new
special datanode would simply not work?
We'd have to initialize them in a special way?



>
> Even though other part of the database cluster has no serious
> difference, the above could be very serious.
>

answers to above questions would help me understand _why_ it is serious.



> It looks a good idea to allow XC to start with only one node
> configuration, and then add nodes as needed.
>
>
Yes, that's my idea also - I see there are ALTER TABLE commands which allow
to replicate/redistribute.



> Other than that, you can use pg_dump and pg_restore to move all your
> data from PostgreSQL to Postgres-XC.
>
>
I know it's a standard procedure - but in my case this would take tens of
hours, and I'm looking for other ways.


Thank you,
Filip

On Wed, Jan 30, 2013 at 5:46 PM, Christophe Le Roux
<chr...@fr...> wrote:
> Thank you for your answers.
>
>
>
> About the third question, I don’t want to use standby server for now (except
> for GM standby).
>
> The cluster would be 4 servers :
>
>
>
> A – GTM server
>
> B – GTM proxy, coordinator and datanode server
>
> C – GTM proxy, coordinator and datanode server
>
> D – GTM standby server
>
>
>
> All tables of all databases are “DISTRIBUTE by REPLICATION” (postgres-XC
> capability). So we have same data at the same time on datanodes B and C.
>
> So if B server fail , C server will do the job.
>
> If C server fail, B server will do the job.
>
> B and C server talk to A server (for GxId, sequence,..) but if A server
> fail, we can promote D server to the master GTM.
>
>
>
> Now, we suppose that : server A and B are located in DC1, and serveur C and
> D are located in DC2.
>
> B and C will do the jobs and talk to A server (for GxId, sequence,..).
>

If I read correctly, B and C will be in different DCs which typically
mean that they will connected on a slow WAN link. Since for
replication, coordinator needs to send a lot of information to/for the
datanode, I am not sure if this would work well. But then it depends
on how much workload you are really putting on the server. It will
also depend on what kind of queries you would be running. I don't
think anyone has tried (yet) to keep the datanodes on different DCs,
but give it a try.

Thanks,
Pavan
-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

On Wed, Jan 30, 2013 at 5:46 PM, Christophe Le Roux <
chr...@fr...> wrote:

>  Thank you for your answers.****
>
> ** **
>
> About the third question, I don’t want to use standby server for now
> (except for GM standby).****
>
> The cluster would be 4 servers :****
>
> ** **
>
> A – GTM server****
>
> B – GTM proxy, coordinator and datanode server****
>
> C – GTM proxy, coordinator and datanode server****
>
> D – GTM standby server****
>
> ** **
>
> All tables of all databases are “DISTRIBUTE by REPLICATION” (postgres-XC
> capability). So we have same data at the same time on datanodes B and C.**
> **
>
> So if B server fail , C server will do the job.****
>
> If C server fail, B server will do the job.****
>
> B and C server talk to A server (for GxId, sequence,..) but if A server
> fail, we can promote D server to the master GTM.****
>
> **
>

Thanks for elaborate example. Now it's clear. Here's a suggestion. You
don't need separate servers for GTM or GTM standby. They can share servers
with coordinator or datanode. But having them on separate servers would
boost the performance. If one of B or C fails, Postgres-XC fails (for this
release). But you can use coordinators on B and C simultaneously.


> **
>
> Now, we suppose that : server A and B are located in DC1, and serveur C
> and D are located in DC2.
>

Postgres-XC is not tolerant to network failures (right now). You will need
to test your deployment for this scenario.


> ****
>
> B and C will do the jobs and talk to A server (for GxId, sequence,..).****
>
> If DC2 fail, B (and A) servers will do the job.****
>
> If DC1 fail, we can promote D server to the master GTM and C server will
> do the job.****
>
> ** **
>
> ** **
>
> Is it all right ?****
>
> ** **
>
> ** **
>
> ** **
>
> Kriss_fr****
>
> ** **
>
> *De :* Ashutosh Bapat [mailto:ash...@en...]
> *Envoyé :* mercredi 30 janvier 2013 12:45
> *À :* Christophe Le Roux
> *Cc :* pos...@li...
> *Objet :* Re: [Postgres-xc-general] Table distribution/replication and
> other things****
>
> ** **
>
> Hi Christophe,
> Thanks for showing interest in Postgres-XC. Please find the answers below,
> inlined
>
> ****
>
> On Wed, Jan 30, 2013 at 4:31 PM, Christophe Le Roux <
> chr...@fr...> wrote:****
>
> Hi the team,****
>
>  ****
>
> First of all thank you for your work on postgresql XC. It’s a very cool
> features product, but I need more informations.****
>
>  ****
>
> My first question is about table distribution/replication :****
>
>                 Can we distribute and replicate a table ? (A table
> distributed on 3 datanodes (for performance) and replicated on 3 others
> datanode(for availability without down time) – I’m not talking about hot
> standby or streaming replication included in standard postgresql)****
>
>                 Is it in the roadmap to add this capability ?****
>
>  ****
>
>
> No, we can't have mixed distribution. You can use standbys for datanodes
> and coordinators using the streaming replication and hot standby
> capabilities of PostgreSQL. We do not have mixed distribution on our
> roadmap, right now.
>  ****
>
>  My second question is about GTM :****
>
>                 During how many time (30 seconds,5 min, 15 min, 1h ...)
> the GTM serveur can be down without impact the service (and without promote
> a GTM stanby to master) ?****
>
>               ****
>
>
> GTM provides the global transaction ids to datanodes and coordinators, so
> downtime allowed depends upon the transaction load, or rate of incoming
> transactions to the server. But, we don't have precise numbers here.
>  ****
>
>  My third question is about global architecture :****
>
>                 we have 2 datacenters with an extra low latency link (like
> LAN).****
>
> I want to put in the first DC one node with GTM and another with GTM
> proxy, coordinator and datanode and in the second DC one node with GTM
> standby and another node with GTM proxy,coordinator and datanode.****
>
> All tables of all databases will be replicated (DISTRIBUTE BY REPLICATION).
> ****
>
> Web server are in first and second DC.****
>
>  ****
>
> The goal is :****
>
> service is delivred by first and second DC at the same time****
>
> if one DC fail the service continue the other****
>
>  ****
>
>
> If I understand your deployment correctly, you are actually replicating
> the Postgres-XC setup at two DC's using standby servers. That may not work.
> We haven't tested this configuration.
>  ****
>
>                  What do you think about this ?****
>
>                 Can it be possible ?****
>
>  ****
>
>  ****
>
> Thank you****
>
>  ****
>
> Kriss_fr                                                   ****
>
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_jan
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general****
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company****
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Thank you for your answers.

About the third question, I don't want to use standby server for now (except for GM standby).
The cluster would be 4 servers :

A - GTM server
B - GTM proxy, coordinator and datanode server
C - GTM proxy, coordinator and datanode server
D - GTM standby server

All tables of all databases are "DISTRIBUTE by REPLICATION" (postgres-XC capability). So we have same data at the same time on datanodes B and C.
So if B server fail , C server will do the job.
If C server fail, B server will do the job.
B and C server talk to A server (for GxId, sequence,..) but if A server fail, we can promote D server to the master GTM.

Now, we suppose that : server A and B are located in DC1, and serveur C and D are located in DC2.
B and C will do the jobs and talk to A server (for GxId, sequence,..).
If DC2 fail, B (and A) servers will do the job.
If DC1 fail, we can promote D server to the master GTM and C server will do the job.


Is it all right ?



Kriss_fr

De : Ashutosh Bapat [mailto:ash...@en...]
Envoyé : mercredi 30 janvier 2013 12:45
À : Christophe Le Roux
Cc : pos...@li...
Objet : Re: [Postgres-xc-general] Table distribution/replication and other things

Hi Christophe,
Thanks for showing interest in Postgres-XC. Please find the answers below, inlined

On Wed, Jan 30, 2013 at 4:31 PM, Christophe Le Roux <chr...@fr...<mailto:chr...@fr...>> wrote:
Hi the team,

First of all thank you for your work on postgresql XC. It's a very cool features product, but I need more informations.

My first question is about table distribution/replication :
                Can we distribute and replicate a table ? (A table distributed on 3 datanodes (for performance) and replicated on 3 others datanode(for availability without down time) - I'm not talking about hot standby or streaming replication included in standard postgresql)
                Is it in the roadmap to add this capability ?


No, we can't have mixed distribution. You can use standbys for datanodes and coordinators using the streaming replication and hot standby capabilities of PostgreSQL. We do not have mixed distribution on our roadmap, right now.

My second question is about GTM :
                During how many time (30 seconds,5 min, 15 min, 1h ...) the GTM serveur can be down without impact the service (and without promote a GTM stanby to master) ?


GTM provides the global transaction ids to datanodes and coordinators, so downtime allowed depends upon the transaction load, or rate of incoming transactions to the server. But, we don't have precise numbers here.

My third question is about global architecture :
                we have 2 datacenters with an extra low latency link (like LAN).
I want to put in the first DC one node with GTM and another with GTM proxy, coordinator and datanode and in the second DC one node with GTM standby and another node with GTM proxy,coordinator and datanode.
All tables of all databases will be replicated (DISTRIBUTE BY REPLICATION).
Web server are in first and second DC.

The goal is :
service is delivred by first and second DC at the same time
if one DC fail the service continue the other


If I understand your deployment correctly, you are actually replicating the Postgres-XC setup at two DC's using standby servers. That may not work. We haven't tested this configuration.

                What do you think about this ?
                Can it be possible ?


Thank you

Kriss_fr

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general



--
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi Christophe,
Thanks for showing interest in Postgres-XC. Please find the answers below,
inlined


On Wed, Jan 30, 2013 at 4:31 PM, Christophe Le Roux <
chr...@fr...> wrote:

>  Hi the team,****
>
> ** **
>
> First of all thank you for your work on postgresql XC. It’s a very cool
> features product, but I need more informations.****
>
> ** **
>
> My first question is about table distribution/replication :****
>
>                 Can we distribute and replicate a table ? (A table
> distributed on 3 datanodes (for performance) and replicated on 3 others
> datanode(for availability without down time) – I’m not talking about hot
> standby or streaming replication included in standard postgresql)****
>
>                 Is it in the roadmap to add this capability ?****
>
> **
>

No, we can't have mixed distribution. You can use standbys for datanodes
and coordinators using the streaming replication and hot standby
capabilities of PostgreSQL. We do not have mixed distribution on our
roadmap, right now.


> **
>
> My second question is about GTM :****
>
>                 During how many time (30 seconds,5 min, 15 min, 1h ...)
> the GTM serveur can be down without impact the service (and without promote
> a GTM stanby to master) ?****
>
>
>

GTM provides the global transaction ids to datanodes and coordinators, so
downtime allowed depends upon the transaction load, or rate of incoming
transactions to the server. But, we don't have precise numbers here.


> My third question is about global architecture :****
>
>                 we have 2 datacenters with an extra low latency link (like
> LAN).****
>
> I want to put in the first DC one node with GTM and another with GTM
> proxy, coordinator and datanode and in the second DC one node with GTM
> standby and another node with GTM proxy,coordinator and datanode.****
>
> All tables of all databases will be replicated (DISTRIBUTE BY REPLICATION).
> ****
>
> Web server are in first and second DC.****
>
> ** **
>
> The goal is :****
>
> service is delivred by first and second DC at the same time****
>
> if one DC fail the service continue the other****
>
> **
>

If I understand your deployment correctly, you are actually replicating the
Postgres-XC setup at two DC's using standby servers. That may not work. We
haven't tested this configuration.


> **
>
>                 What do you think about this ?****
>
>                 Can it be possible ?****
>
> ** **
>
> ** **
>
> Thank you****
>
> ** **
>
> Kriss_fr                                                   ****
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_jan
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi the team,

First of all thank you for your work on postgresql XC. It's a very cool features product, but I need more informations.

My first question is about table distribution/replication :
                Can we distribute and replicate a table ? (A table distributed on 3 datanodes (for performance) and replicated on 3 others datanode(for availability without down time) - I'm not talking about hot standby or streaming replication included in standard postgresql)
                Is it in the roadmap to add this capability ?

My second question is about GTM :
                During how many time (30 seconds,5 min, 15 min, 1h ...) the GTM serveur can be down without impact the service (and without promote a GTM stanby to master) ?

My third question is about global architecture :
                we have 2 datacenters with an extra low latency link (like LAN).
I want to put in the first DC one node with GTM and another with GTM proxy, coordinator and datanode and in the second DC one node with GTM standby and another node with GTM proxy,coordinator and datanode.
All tables of all databases will be replicated (DISTRIBUTE BY REPLICATION).
Web server are in first and second DC.

The goal is :
service is delivred by first and second DC at the same time
if one DC fail the service continue the other

                What do you think about this ?
                Can it be possible ?


Thank you

Kriss_fr