postgres-xc-general Mailing List for Postgres-XC (Page 4)

My apologies if this seems far too simple.

I'm looking at Postgres-XC 1.2beta to build out a datastore.  I've been through the documentation several times and I built out what I believed was a reasonable first step, with a GTM, single coordinator and two data nodes on an amazon i2 instance.

I start all four instances (gtm, coord, data_node_1 and data_node_2), add the nodes to the coordinator and build my schema - to this point, everything is scripted.  At this point, I already see failures to find a procs in the database logs. The failures match transactions that are coming in and are as follows:

STATEMENT:  COMMIT PREPARED 'T10010'
LOG:  failed to find proc 0x7f6bcbc0a7c0 in ProcArray
STATEMENT:  COMMIT PREPARED 'T10012'
LOG:  failed to find proc 0x7f6bcbc0a7c0 in ProcArray
STATEMENT:  COMMIT PREPARED 'T10014'
LOG:  failed to find proc 0x7f6bcbc0a7c0 in ProcArray
STATEMENT:  COMMIT PREPARED 'T10016'

As these appear to be mostly benign (I'm sure they're not).  I begin building the tables in my schema and this is usually about the point I begin to experience a breakdown, usually resulting with the coordinator reporting that the database is in recovery mode.  There is nothing special about the DDL - for example, it can be as simple as the following:

DROP TABLE IF EXISTS Foo.Bar;
CREATE TABLE Foo.Bar( Foo int NOT NULL, Bar smallint NOT NULL ) DISTRIBUTE HASH( Foo );

The first message to come back is "PANIC: sorry, too many clients already"
 - followed shortly thereafter by "FATAL: the database system is in recovery mode"

The configurations were built using initdb or initgtm directly.

gtm/gtm.conf
----------------------------------------
nodename = 'one'
port = 6666

data_coord/postgresql.conf
----------------------------------------
max_connections = 100
shared_buffers = 128MB
max_prepared_transactions = 20
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_timezone = 'UTC'
datestyle = 'iso, mdy'
timezone = 'UTC'
lc_messages = 'en_US.UTF-8'
lc_monetary = 'en_US.UTF-8'
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8'
default_text_search_config = 'pg_catalog.english'
min_pool_size = 20
max_pool_size = 200
pgxc_node_name = 'coord_1'

data_node_1/postgresql.conf
----------------------------------------
port = 15432
max_connections = 100
shared_buffers = 128MB
max_prepared_transactions = 100
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_timezone = 'UTC'
datestyle = 'iso, mdy'
timezone = 'UTC'
lc_messages = 'en_US.UTF-8'
lc_monetary = 'en_US.UTF-8'
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8'
default_text_search_config = 'pg_catalog.english'
min_pool_size = 1
max_pool_size = 100
pgxc_node_name = 'node_1'

data_node_2/postgresql.conf
----------------------------------------
port = 15433
max_connections = 100
shared_buffers = 128MB
max_prepared_transactions = 100
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_timezone = 'UTC'
datestyle = 'iso, mdy'
timezone = 'UTC'
lc_messages = 'en_US.UTF-8'
lc_monetary = 'en_US.UTF-8'
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8'
default_text_search_config = 'pg_catalog.english'
min_pool_size = 1
max_pool_size = 100
pgxc_node_name = 'node_2'

Any insight would be helpful in understanding what I've done wrong here.

Thank you,

Aaron

Hi. I decided to modify my configuration like this:

Node A: 1 gtm master
Node B: 1 gtm_proxy,

1 coordinator master, 1 coordinator slave (for node C coordinator),
1 datanode master, 1 datanode slave (for node C data node)

Node C: 1 gtm_proxy,

1 coordinator master, 1 coordinator slave (for node B coordinator),
1 datanode master, 1 datanode slave (for node B data node)

Node D: 1 gtm slave

I added following lines in postgresql.conf of each data node slave.
pgxc_ctl script didn't add them.

archive_mode = off
archive_command = ''
max_wal_senders = 0
wal_level = minimal

In normal operation, everything worked fine.

Fist I killed data node master in Node B, I promoted corresponding data
node slave to master with pgxc_ctl. I got the following error:

ERROR:  PGXC node datanode1: two nodes cannot be primary
 pgxc_pool_reload
------------------
 t
(1 row)

ERROR:  PGXC node datanode1: two nodes cannot be primary
 pgxc_pool_reload
------------------
 t
(1 row)

In both master coordinators, I queried pgxc_node and the failed datanode
still pointed to node B. I had to execute ALTER NODE for change host and
primary property. After that, everything worked fine.

Then, I killed coordinator master in Node B, I promoted corresponding
coordinator slave to master. I got the following results:

ALTER NODE
 pgxc_pool_reload
------------------
 t
(1 row)

ERROR:  Failed to get pooled connections
CONTEXT:  SQL statement "EXECUTE DIRECT ON (coord1) 'SELECT
pg_catalog.pg_try_advisory_xact_lock_shared(65535, 0)'"
 pgxc_pool_reload
------------------
 t
(1 row)

So in new master coordinator, when I queried pgxc_node the failed
coordinator pointed correctly to node C. But in the original master
coordinator of node C, when I queried pgxc_node the failed coordinator
pointed to node B. In this coordinator I tried to modify failed coordinator
with ALTER NODE but I got the same error Failed to get pooled connections.

Finally when I execute ALTER TALBE … DELETE NODE for a table, in the new
master coordinator it executed ok. But in the original master coordinator
of node C I got the same error Failed to get pooled connections.

Is this behaviour normal? What can i do?


2014-04-02 23:19 GMT-05:00 鈴木 幸市 <ko...@in...>:

>  Before removing a datanode, you should drop the datanode from tables
> before you drop the datanode.   You can do this with ALTER TALBE … DELETE
> NODE as seen in
> http://postgres-xc.sourceforge.net/docs/1_1/sql-altertable.html.   This
> will extract all the rows from the node and redistribute them (as well as
> others) to the new set of nodes.
>
>  List of the nodes where a table is distributed/replicated in pgxc_class.
>
>  Regards;
> ---
> Koichi Suzuki
>
>  2014/04/02 15:06、Sergio Sinuco <ser...@da...> のメール：
>
>  I use pgxc_ctl.
>
>  My configuration is:
>
>  Node A: 1 gtm master
> Node B: 1 gtm_proxy, 1 coordinator, 1 datanode
> Node C: 1 gtm_proxy, 1 coordinator, 1 datanode
>  Node D: 1 gtm slave
>
>  First I stopped coordinator and data node from Node C. Then I executed*
> pgxc_ctl remove datanode.... *and *pgxc_ctl remove coordinator...* were
> ok. But I got "Failed to get pooled connections" message when I tried an
> insert or update in Node B. I also ran pgxc_pool_reload and restarted the
> cluster. If I run "SELECT * FROM pgxc_node" in Node B I only have one
> coordinator and one data node.
>
>  The inserts and update commands was ran in a DISTRIBUTE BY REPLICATION
> table.
>
>
>
> 2014-04-02 0:32 GMT-05:00 鈴木 幸市 <ko...@in...>:
>
>> Did you change your cluster configuration?   Are you using pgxc_ctl or
>> doing configuration/operation manually?
>>
>>  They help to see what’s going on.
>>
>>  Thank you;
>> ---
>> Koichi Suzuki
>>
>>  2014/04/02 14:02、Sergio Sinuco <ser...@da...> のメール：
>>
>>   Hi. I dropped a coordinator node and a data node from a cluster, but
>> when i try to make an insert or update I have a "Failed to get pooled
>> connections" message. I executed pgxc_pool_reload() but it didn't work.
>> What can I do? I use pgxc 1.1.
>>
>> --
>> Sergio E. Sinuco Leon
>> Arquitecto de soluciones
>> Datatraffic S.A.S.
>> Móvil: (57) 310 884 26 50
>> Fijo (+571) 7426160 Ext 115
>> Calle 93 # 15-27 Ofc. 502
>> Calle 29 # 6 - 94 Ofc. 601
>> Bogotá, Colombia.
>> www.datatraffic.com.co
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>>
>
>
>  --
> Sergio E. Sinuco Leon
> Arquitecto de soluciones
> Datatraffic S.A.S.
> Móvil: (57) 310 884 26 50
> Fijo (+571) 7426160 Ext 115
> Calle 93 # 15-27 Ofc. 502
> Calle 29 # 6 - 94 Ofc. 601
> Bogotá, Colombia.
> www.datatraffic.com.co
>
>
>


-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co

Hi Tim,


On Tue, Apr 8, 2014 at 12:28 AM, Tim Uckun <tim...@gm...> wrote:

> It would be ideal if shards could be distributed amongst the data nodes in
> a redundant fashion. Perhaps with limits like at least three nodes, no more
> than five nodes etc.
>

As mentioned, there is no built-in HA.  While some tables can be
"replicated" there are not redundant copies of the other tables that are
"distributed" (partiioned/sharded). The way to achieve this is analogous to
HA in PostgreSQL; have a redundant copy of each node. In addition,
Postgres-XC adds a a GTM Standby. My company has used Corosync/Pacemaker
for failing over the components for our StormDB branch of Postgres-XC. What
we do is have a replica of node1 on node2 and node3, have a replica of
node2 on node3 and node4, etc.

What you are describing above (shard redundancy policies), is actually what
my employer, TransLattice, offers in its multi-master TED product. It
decouples sharding from the nodes so that it is not a 1-1 mapping. It is
not open source, however.

Good luck!

Regards,


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

It would be ideal if shards could be distributed amongst the data nodes in
a redundant fashion. Perhaps with limits like at least three nodes, no more
than five nodes etc.

Thanks.


On Tue, Apr 8, 2014 at 12:59 PM, 鈴木 幸市 <ko...@in...> wrote:

>  Not thorough document of everything on it.   So far, HA configuration
> needs integration with other software like Pacemaker and has far more
> variety of configuration to describe in a simple document.
>
>  To configure XC with slaves and to failover the nodes, following
> material will be helpful.
>
>  Pgxc_ctl: Postgres-XC configuration and operation tool.   Reference
> maulal will be at
> http://postgres-xc.sourceforge.net/docs/1_2_1/pgxc-ctl.html
> pgxc_ctl sample configuration and sample operation scenario will be at
> ---
> https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013
>
>  Regards;
> ---
> Koichi Suzuki
>
>  2014/04/08 8:20、Tim Uckun <tim...@gm...> のメール：
>
>  Is there a document someplace which explains how to achieve high
> availability with PGXC?  For example do you set up two clusters and
> replicate between them? Do you replicate every data node separately?  Is
> there a way to "raid" the data so that it exists in multiple nodes so you
> can safely lose a data node?
>
>  I know you can specify that a table be present in all the nodes but I am
> guessing it won't really get you much if you choose every table to be on
> every node.
>
>  Thanks.
> ------------------------------------------------------------------------------
> Put Bad Developers to Shame
> Dominate Development with Jenkins Continuous Integration
> Continuously Automate Build, Test & Deployment
> Start a new project now. Try Jenkins in the cloud.
>
> http://p.sf.net/sfu/13600_Cloudbees_______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>

Not thorough document of everything on it.   So far, HA configuration needs integration with other software like Pacemaker and has far more variety of configuration to describe in a simple document.

To configure XC with slaves and to failover the nodes, following material will be helpful.

Pgxc_ctl: Postgres-XC configuration and operation tool.   Reference maulal will be at http://postgres-xc.sourceforge.net/docs/1_2_1/pgxc-ctl.html
pgxc_ctl sample configuration and sample operation scenario will be at
---https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

Regards;
---
Koichi Suzuki

2014/04/08 8:20、Tim Uckun <tim...@gm...<mailto:tim...@gm...>> のメール：

Is there a document someplace which explains how to achieve high availability with PGXC?  For example do you set up two clusters and replicate between them? Do you replicate every data node separately?  Is there a way to "raid" the data so that it exists in multiple nodes so you can safely lose a data node?

I know you can specify that a table be present in all the nodes but I am guessing it won't really get you much if you choose every table to be on every node.

Thanks.
------------------------------------------------------------------------------
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

So far, we don’t have any performance test data where coordinators/datanodes are geographically distributed from GTM.   Any attempt will be helpful for future improvement.

Regards;
---
Koichi Suzuki

2014/04/07 23:40、Attila Berenyi <att...@se...<mailto:att...@se...>> のメール：

Hi,

Thanks for the lightning fast reply.

2. Postgres-XC
The overall structure of the replication looks pretty convincing but I have a few question:
- I read somewhere/saw in a slide deck (unfortunately I cannot find it any more) that it creates a performance bottleneck to have the coordinator and the datanode on the same machine (VMs in my test case). Is it still valid?

No it is not valid.   Instead, for simpler load balancing between coordinator and datanode, we advice to configure both on the same machine.   This can also utilize data localization for better performance.

That is good to know.

- Is it possible to add 'slave' datanodes to the DB cluster, e.g an external web server?

Yes.   pgxc_ctl from contrib module will help much.   You will find the reference at http://postgres-xc.sourceforge.net/docs/1_2_1/pgxc-ctl.html

You will find sample configuration and pgxc_ctl demo scenario from https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

I will definitely check it out.

- Approximate overhead/latency? I guess if the user connects to the local (i.e. the user's office) coordinator and commits some changes, the user still have to wait for the data to be sync'd with all the datanodes in the cluster, right? What if the DB cluser includes coordinators/datanodes from all over the globe?

We’ve not assumed the usecase where coordinators/datanodes are geometrically distributed, not because of the latency but to provide full-featured transaction ACID capability over the cluster.     In the local configuration, this latency is ignorable.

I'll test this anyway; I am kind of curious what will be the performance like.

Cheers,
Attila
------------------------------------------------------------------------------
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Is there a document someplace which explains how to achieve high
availability with PGXC?  For example do you set up two clusters and
replicate between them? Do you replicate every data node separately?  Is
there a way to "raid" the data so that it exists in multiple nodes so you
can safely lose a data node?

I know you can specify that a table be present in all the nodes but I am
guessing it won't really get you much if you choose every table to be on
every node.

Thanks.

On Mon, Apr 7, 2014 at 1:19 AM, Juned Khan <jkh...@gm...> wrote:

> Hi koichi,
>
> Yeah we can have slave of each component but in case of failure i need to
> promote it
> manually right ?
>
> is there any way to make this process automatic ?
>
>
>
We use Corosync/Pacemaker. One should be careful though, it takes time; you
may encounter false positives with premature failover, or one down
component could impact another and also failover. So, I recommend a lot of
testing.

Also, one should be prepared to fall back and have a strategy for that in
place.



-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Hi,

Thanks for the lightning fast reply.
>>
>> 2. Postgres-XC
>> The overall structure of the replication looks pretty convincing but 
>> I have a few question:
>> - I read somewhere/saw in a slide deck (unfortunately I cannot find 
>> it any more) that it creates a performance bottleneck to have the 
>> coordinator and the datanode on the same machine (VMs in my test 
>> case). Is it still valid?
>
> No it is not valid.   Instead, for simpler load balancing between 
> coordinator and datanode, we advice to configure both on the same 
> machine.   This can also utilize data localization for better performance.

That is good to know.

>> - Is it possible to add 'slave' datanodes to the DB cluster, e.g an 
>> external web server?
>
> Yes.   pgxc_ctl from contrib module will help much.   You will find 
> the reference at 
> http://postgres-xc.sourceforge.net/docs/1_2_1/pgxc-ctl.html
>
> You will find sample configuration and pgxc_ctl demo scenario from 
> https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

I will definitely check it out.

>> - Approximate overhead/latency? I guess if the user connects to the 
>> local (i.e. the user's office) coordinator and commits some changes, 
>> the user still have to wait for the data to be sync'd with all the 
>> datanodes in the cluster, right? What if the DB cluser includes 
>> coordinators/datanodes from all over the globe?
>
> We’ve not assumed the usecase where coordinators/datanodes are 
> geometrically distributed, not because of the latency but to provide 
> full-featured transaction ACID capability over the cluster.     In the 
> local configuration, this latency is ignorable.

I'll test this anyway; I am kind of curious what will be the performance 
like.

Cheers,
Attila

Please see my input inline.
---
Koichi Suzuki

2014/04/07 18:46、Attila Berenyi <att...@se...<mailto:att...@se...>> のメール：

Hi,

I have been testing master-master replication solutions recently, and I have identified three possible solutions/software:

The scenario
Multiple masters located in different offices in the UK (phase 1); all around the globe (phase 2). The servers should be in sync all the time.

Possible solutions
1. Bucardo
2. Postgres-XC
3. EnterpriseDB's xDB Replication Server

Test results
1. Bucardo
Easy to use and set up, but as the data authors at the different offices will create and drop tables quite frequently we cannot really cope with the administration of bucardo, i.e. adding table to the herds. If you have a more or less static dataset or you have a DBA at every site it might work for you though.

2. Postgres-XC
The overall structure of the replication looks pretty convincing but I have a few question:
- I read somewhere/saw in a slide deck (unfortunately I cannot find it any more) that it creates a performance bottleneck to have the coordinator and the datanode on the same machine (VMs in my test case). Is it still valid?

No it is not valid.   Instead, for simpler load balancing between coordinator and datanode, we advice to configure both on the same machine.   This can also utilize data localization for better performance.

- Is it possible to add 'slave' datanodes to the DB cluster, e.g an external web server?

Yes.   pgxc_ctl from contrib module will help much.   You will find the reference at http://postgres-xc.sourceforge.net/docs/1_2_1/pgxc-ctl.html

You will find sample configuration and pgxc_ctl demo scenario from https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

- Approximate overhead/latency? I guess if the user connects to the local (i.e. the user's office) coordinator and commits some changes, the user still have to wait for the data to be sync'd with all the datanodes in the cluster, right? What if the DB cluser includes coordinators/datanodes from all over the globe?

We’ve not assumed the usecase where coordinators/datanodes are geometrically distributed, not because of the latency but to provide full-featured transaction ACID capability over the cluster.     In the local configuration, this latency is ignorable.

Thank you.


3. EnterpriseDB's xDB Replication Server
TBD

Thanks in advance.

Cheers,
Attila
------------------------------------------------------------------------------
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees_APR_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hi,

I have been testing master-master replication solutions recently, and I 
have identified three possible solutions/software:

The scenario
Multiple masters located in different offices in the UK (phase 1); all 
around the globe (phase 2). The servers should be in sync all the time.

Possible solutions
1. Bucardo
2. Postgres-XC
3. EnterpriseDB's xDB Replication Server

Test results
1. Bucardo
Easy to use and set up, but as the data authors at the different offices 
will create and drop tables quite frequently we cannot really cope with 
the administration of bucardo, i.e. adding table to the herds. If you 
have a more or less static dataset or you have a DBA at every site it 
might work for you though.

2. Postgres-XC
The overall structure of the replication looks pretty convincing but I 
have a few question:
- I read somewhere/saw in a slide deck (unfortunately I cannot find it 
any more) that it creates a performance bottleneck to have the 
coordinator and the datanode on the same machine (VMs in my test case). 
Is it still valid?
- Is it possible to add 'slave' datanodes to the DB cluster, e.g an 
external web server?
- Approximate overhead/latency? I guess if the user connects to the 
local (i.e. the user's office) coordinator and commits some changes, the 
user still have to wait for the data to be sync'd with all the datanodes 
in the cluster, right? What if the DB cluser includes 
coordinators/datanodes from all over the globe?

3. EnterpriseDB's xDB Replication Server
TBD

Thanks in advance.

Cheers,
Attila

Because monitoring feature is not a part of Postgres-XC yet, you need to promote it manually.   You will fold failover command in pgxc_ctl to help this operation.

The background that automatic failover is not a part of XC is:

We need to monitor many other failures such as server itself, storage, network among other software.   It is more than Postgres-XC can do by itself.

Of course, you can invoke pgxc_ctl command to promote a slave as a part of your resource agent of an automatic failover software such as Pacemaker.

I hope Postgres-XC can have automatic failover system which takes care of XC components only and take other monitoring result from external middleware.

Regards;
---
Koichi Suzuki

2014/04/07 14:19、Juned Khan <jkh...@gm...<mailto:jkh...@gm...>> のメール：

Hi koichi,

Yeah we can have slave of each component but in case of failure i need to promote it
manually right ?

is there any way to make this process automatic ?




On Sat, Apr 5, 2014 at 7:49 AM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote:
Each node can have its own slave.    Example configuration for
pgxc_ctl will be found at
http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

You will also find example script of the demonstration I used at PG Open 2013.

Regards;
---
Koichi Suzuki


2014-04-04 20:12 GMT+09:00 Juned Khan <jkh...@gm...<mailto:jkh...@gm...>>:
> Hi koichi,
>
> so it means if i have 10 clusters and if any of them goes down then whole
> system will be useless
>
> what's the solution of this problem ? How to failover ?
>
>
> On Fri, Apr 4, 2014 at 4:20 PM, 鈴木 幸市 <ko...@in...<mailto:ko...@in...>> wrote:
>>
>> Postgres-XC assumes all the nodes are running normally so that the planner
>> can build a plan regarding clause push-down, table distribution and
>> replication.
>>
>> The error indicates some of such nodes are down.
>>
>> Please restart the failed one and issue ‘select pgxc_pool_reload()’.
>> ---
>> Koichi Suzuki
>>
>> 2014/04/04 19:15、Juned Khan <jkh...@gm...<mailto:jkh...@gm...>> のメール：
>>
>> Hi all,
>>
>> I am using pgxc which has two datanodes (node1 and node2).
>>
>> As of now my application is connected to node1, yesterday due
>> to some problem ssh service just stopped on second server.
>>
>> when i was trying to run any query on node1 then it gave  me
>> "ERROR:  Failed to get pooled connection" .
>>
>> I was just wondering node2 is went down and my application is connected
>> to node1 then why this problem? how does it relates to that error.
>>
>> is it normal ??
>>
>> Please suggest.
>>
>>
>> --
>> Thanks,
>> Juned Khan
>> iNextrix Technologies Pvt Ltd.
>> www.inextrix.com<http://www.inextrix.com/>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...<mailto:Pos...@li...>
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com<http://www.inextrix.com/>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...<mailto:Pos...@li...>
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



--
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com<http://www.inextrix.com/>

Hi koichi,

Yeah we can have slave of each component but in case of failure i need to
promote it
manually right ?

is there any way to make this process automatic ?




On Sat, Apr 5, 2014 at 7:49 AM, Koichi Suzuki <koi...@gm...> wrote:

> Each node can have its own slave.    Example configuration for
> pgxc_ctl will be found at
>
> http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013
>
> You will also find example script of the demonstration I used at PG Open
> 2013.
>
> Regards;
> ---
> Koichi Suzuki
>
>
> 2014-04-04 20:12 GMT+09:00 Juned Khan <jkh...@gm...>:
> > Hi koichi,
> >
> > so it means if i have 10 clusters and if any of them goes down then whole
> > system will be useless
> >
> > what's the solution of this problem ? How to failover ?
> >
> >
> > On Fri, Apr 4, 2014 at 4:20 PM, 鈴木 幸市 <ko...@in...> wrote:
> >>
> >> Postgres-XC assumes all the nodes are running normally so that the
> planner
> >> can build a plan regarding clause push-down, table distribution and
> >> replication.
> >>
> >> The error indicates some of such nodes are down.
> >>
> >> Please restart the failed one and issue ‘select pgxc_pool_reload()’.
> >> ---
> >> Koichi Suzuki
> >>
> >> 2014/04/04 19:15、Juned Khan <jkh...@gm...> のメール：
> >>
> >> Hi all,
> >>
> >> I am using pgxc which has two datanodes (node1 and node2).
> >>
> >> As of now my application is connected to node1, yesterday due
> >> to some problem ssh service just stopped on second server.
> >>
> >> when i was trying to run any query on node1 then it gave  me
> >> "ERROR:  Failed to get pooled connection" .
> >>
> >> I was just wondering node2 is went down and my application is connected
> >> to node1 then why this problem? how does it relates to that error.
> >>
> >> is it normal ??
> >>
> >> Please suggest.
> >>
> >>
> >> --
> >> Thanks,
> >> Juned Khan
> >> iNextrix Technologies Pvt Ltd.
> >> www.inextrix.com
> >>
> >>
> ------------------------------------------------------------------------------
> >> _______________________________________________
> >> Postgres-xc-general mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>
> >>
> >
> >
> >
> > --
> > Thanks,
> > Juned Khan
> > iNextrix Technologies Pvt Ltd.
> > www.inextrix.com
> >
> >
> ------------------------------------------------------------------------------
> >
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>



-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Each node can have its own slave.    Example configuration for
pgxc_ctl will be found at
http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

You will also find example script of the demonstration I used at PG Open 2013.

Regards;
---
Koichi Suzuki


2014-04-04 20:12 GMT+09:00 Juned Khan <jkh...@gm...>:
> Hi koichi,
>
> so it means if i have 10 clusters and if any of them goes down then whole
> system will be useless
>
> what's the solution of this problem ? How to failover ?
>
>
> On Fri, Apr 4, 2014 at 4:20 PM, 鈴木 幸市 <ko...@in...> wrote:
>>
>> Postgres-XC assumes all the nodes are running normally so that the planner
>> can build a plan regarding clause push-down, table distribution and
>> replication.
>>
>> The error indicates some of such nodes are down.
>>
>> Please restart the failed one and issue ‘select pgxc_pool_reload()’.
>> ---
>> Koichi Suzuki
>>
>> 2014/04/04 19:15、Juned Khan <jkh...@gm...> のメール：
>>
>> Hi all,
>>
>> I am using pgxc which has two datanodes (node1 and node2).
>>
>> As of now my application is connected to node1, yesterday due
>> to some problem ssh service just stopped on second server.
>>
>> when i was trying to run any query on node1 then it gave  me
>> "ERROR:  Failed to get pooled connection" .
>>
>> I was just wondering node2 is went down and my application is connected
>> to node1 then why this problem? how does it relates to that error.
>>
>> is it normal ??
>>
>> Please suggest.
>>
>>
>> --
>> Thanks,
>> Juned Khan
>> iNextrix Technologies Pvt Ltd.
>> www.inextrix.com
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

Hi koichi,

so it means if i have 10 clusters and if any of them goes down then whole
system will be useless

what's the solution of this problem ? How to failover ?


On Fri, Apr 4, 2014 at 4:20 PM, 鈴木 幸市 <ko...@in...> wrote:

>  Postgres-XC assumes all the nodes are running normally so that the
> planner can build a plan regarding clause push-down, table distribution and
> replication.
>
>  The error indicates some of such nodes are down.
>
>  Please restart the failed one and issue ‘select pgxc_pool_reload()’.
>  ---
> Koichi Suzuki
>
>  2014/04/04 19:15、Juned Khan <jkh...@gm...> のメール：
>
>   Hi all,
>
>  I am using pgxc which has two datanodes (node1 and node2).
>
>  As of now my application is connected to node1, yesterday due
>  to some problem ssh service just stopped on second server.
>
>  when i was trying to run any query on node1 then it gave  me
> "ERROR:  Failed to get pooled connection" .
>
>  I was just wondering node2 is went down and my application is connected
>  to node1 then why this problem? how does it relates to that error.
>
>  is it normal ??
>
>  Please suggest.
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>


-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Now Postgres-XC 1.0.4 and Postgres-XC 1.1.1 are available.   They come
with various fixes/improvements as seen in each release note.

For 1.1.1,
Release note is available at
http://postgres-xc.sourceforge.net/docs/1_1_1/release-xc-1-1-1.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.1/
Document is available at http://postgres-xc.sourceforge.net/docs/1_1_1/

For 1.0.4
Release note is available at
http://postgres-xc.sourceforge.net/docs/1_0_4/release-xc-1-0-4.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.0/
Document is available at http://postgres-xc.sourceforge.net/docs/1_0_4/

Enjoy;
---
Koichi Suzuki

Postgres-XC assumes all the nodes are running normally so that the planner can build a plan regarding clause push-down, table distribution and replication.

The error indicates some of such nodes are down.

Please restart the failed one and issue ‘select pgxc_pool_reload()’.
---
Koichi Suzuki

2014/04/04 19:15、Juned Khan <jkh...@gm...<mailto:jkh...@gm...>> のメール：

Hi all,

I am using pgxc which has two datanodes (node1 and node2).

As of now my application is connected to node1, yesterday due
to some problem ssh service just stopped on second server.

when i was trying to run any query on node1 then it gave  me
"ERROR:  Failed to get pooled connection" .

I was just wondering node2 is went down and my application is connected
to node1 then why this problem? how does it relates to that error.

is it normal ??

Please suggest.


--
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com<http://www.inextrix.com/>
------------------------------------------------------------------------------
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hi all,

I am using pgxc which has two datanodes (node1 and node2).

As of now my application is connected to node1, yesterday due
to some problem ssh service just stopped on second server.

when i was trying to run any query on node1 then it gave  me
"ERROR:  Failed to get pooled connection" .

I was just wondering node2 is went down and my application is connected
to node1 then why this problem? how does it relates to that error.

is it normal ??

Please suggest.


-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

that's the good news.


On Thu, Apr 3, 2014 at 11:54 AM, Koichi Suzuki <koi...@gm...>wrote:

> Postgres-XC development group is pleased to announce that Postgres-XC
> 1.2.1 is out.
>
> This includes many important features from PostgreSQL 9.3 such as
> materialized views, LATERAL, event triggers and automatic updatable
> views, as well as improved distributed query optimizer.   It also
> comes with many fixes and improvement as found in the release note.
>
> Release note is available at
> http://postgres-xc.sourceforge.net/docs/1_2_beta/release-xc-1-2.html
> Source tarball is available at
> https://sourceforge.net/projects/postgres-xc/files/Version_1.2/
> Document is available at http://postgres-xc.sourceforge.net/docs/1_2_1/
>
> Enjoy.
> ---
> Koichi Suzuki
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Postgres-XC development group is pleased to announce that Postgres-XC
1.2.1 is out.

This includes many important features from PostgreSQL 9.3 such as
materialized views, LATERAL, event triggers and automatic updatable
views, as well as improved distributed query optimizer.   It also
comes with many fixes and improvement as found in the release note.

Release note is available at
http://postgres-xc.sourceforge.net/docs/1_2_beta/release-xc-1-2.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.2/
Document is available at http://postgres-xc.sourceforge.net/docs/1_2_1/

Enjoy.
---
Koichi Suzuki

Before removing a datanode, you should drop the datanode from tables before you drop the datanode.   You can do this with ALTER TALBE … DELETE NODE as seen in http://postgres-xc.sourceforge.net/docs/1_1/sql-altertable.html.   This will extract all the rows from the node and redistribute them (as well as others) to the new set of nodes.

List of the nodes where a table is distributed/replicated in pgxc_class.

Regards;
---
Koichi Suzuki

2014/04/02 15:06、Sergio Sinuco <ser...@da...<mailto:ser...@da...>> のメール：

I use pgxc_ctl.

My configuration is:

Node A: 1 gtm master
Node B: 1 gtm_proxy, 1 coordinator, 1 datanode
Node C: 1 gtm_proxy, 1 coordinator, 1 datanode
Node D: 1 gtm slave

First I stopped coordinator and data node from Node C. Then I executed pgxc_ctl remove datanode.... and pgxc_ctl remove coordinator... were ok. But I got "Failed to get pooled connections" message when I tried an insert or update in Node B. I also ran pgxc_pool_reload and restarted the cluster. If I run "SELECT * FROM pgxc_node" in Node B I only have one coordinator and one data node.

The inserts and update commands was ran in a DISTRIBUTE BY REPLICATION table.



2014-04-02 0:32 GMT-05:00 鈴木 幸市 <ko...@in...<mailto:ko...@in...>>:
Did you change your cluster configuration?   Are you using pgxc_ctl or doing configuration/operation manually?

They help to see what’s going on.

Thank you;
---
Koichi Suzuki

2014/04/02 14:02、Sergio Sinuco <ser...@da...<mailto:ser...@da...>> のメール：

Hi. I dropped a coordinator node and a data node from a cluster, but when i try to make an insert or update I have a "Failed to get pooled connections" message. I executed pgxc_pool_reload() but it didn't work. What can I do? I use pgxc 1.1.

--
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115<tel:%28%2B571%29%207426160%20Ext%20115>
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co<http://www.datatraffic.com.co/>
------------------------------------------------------------------------------
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general




--
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co<http://www.datatraffic.com.co/>

I use pgxc_ctl.

My configuration is:

Node A: 1 gtm master
Node B: 1 gtm_proxy, 1 coordinator, 1 datanode
Node C: 1 gtm_proxy, 1 coordinator, 1 datanode
Node D: 1 gtm slave

First I stopped coordinator and data node from Node C. Then I executed*
pgxc_ctl remove datanode.... *and *pgxc_ctl remove coordinator...* were ok.
But I got "Failed to get pooled connections" message when I tried an insert
or update in Node B. I also ran pgxc_pool_reload and restarted the cluster.
If I run "SELECT * FROM pgxc_node" in Node B I only have one coordinator
and one data node.

The inserts and update commands was ran in a DISTRIBUTE BY REPLICATION
table.



2014-04-02 0:32 GMT-05:00 鈴木 幸市 <ko...@in...>:

>  Did you change your cluster configuration?   Are you using pgxc_ctl or
> doing configuration/operation manually?
>
>  They help to see what’s going on.
>
>  Thank you;
> ---
> Koichi Suzuki
>
>  2014/04/02 14:02、Sergio Sinuco <ser...@da...> のメール：
>
>  Hi. I dropped a coordinator node and a data node from a cluster, but
> when i try to make an insert or update I have a "Failed to get pooled
> connections" message. I executed pgxc_pool_reload() but it didn't work.
> What can I do? I use pgxc 1.1.
>
> --
> Sergio E. Sinuco Leon
> Arquitecto de soluciones
> Datatraffic S.A.S.
> Móvil: (57) 310 884 26 50
> Fijo (+571) 7426160 Ext 115
> Calle 93 # 15-27 Ofc. 502
> Calle 29 # 6 - 94 Ofc. 601
> Bogotá, Colombia.
> www.datatraffic.com.co
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>


-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co

Did you change your cluster configuration?   Are you using pgxc_ctl or doing configuration/operation manually?

They help to see what’s going on.

Thank you;
---
Koichi Suzuki

2014/04/02 14:02、Sergio Sinuco <ser...@da...<mailto:ser...@da...>> のメール：

Hi. I dropped a coordinator node and a data node from a cluster, but when i try to make an insert or update I have a "Failed to get pooled connections" message. I executed pgxc_pool_reload() but it didn't work. What can I do? I use pgxc 1.1.

--
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co<http://www.datatraffic.com.co/>
------------------------------------------------------------------------------
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hi. I dropped a coordinator node and a data node from a cluster, but when i
try to make an insert or update I have a "Failed to get pooled connections"
message. I executed pgxc_pool_reload() but it didn't work. What can I do? I
use pgxc 1.1.

-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co