postgres-xc-general Mailing List for Postgres-XC

Thanks to all for the tips, all up and running now !

PS , on the short version of documentation, it does not mentioned the HOST parameter:

http://postgres-xc.sourceforge.net/docs/1_0/install-short.html

Though I am guessing it all works fine when installed on the same machine

Thanks again

Neil
________________________________________
From: Michael Paquier [mic...@gm...]
Sent: Monday, June 25, 2012 2:09 AM
To: Neil Mooney
Cc: pos...@li...
Subject: Re: [Postgres-xc-general] createdb: database creation failed: ERROR: Failed to get pooled connections

On Mon, Jun 25, 2012 at 10:07 AM, Michael Paquier <mic...@gm...<mailto:mic...@gm...>> wrote:
I may be stepping someone's else answer... But here is some input.

On Fri, Jun 22, 2012 at 9:30 PM, Neil Mooney <nm...@xu...<mailto:nm...@xu...>> wrote:
Can anyone point out where I am going wrong here ?

Here is a complete log of what i've done to setup , error at the end:

Data/Coordinator machines:
==========================

node1:
======

rm -fr /opt/postgres/data*

mkdir -p /opt/postgres/data_coordinator1
mkdir -p /opt/postgres/data_datanode1

chown postgres: /opt/postgres/data_*

su - postgres

initdb -D /opt/postgres/data_coordinator1 --nodename coordinator1
initdb -D /opt/postgres/data_datanode1 --nodename datanode1

#configure /opt/postgres/data_coordinator1/postgres.conf with GTM server
#configure /opt/postgres/data_datanode1/postgres.conf with GTM server


postgres -X -p 15432 -D /opt/postgres/data_datanode1 >>logfile 2>&1 &
postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &

node2:
======

rm -fr /opt/postgres/data*

mkdir -p /opt/postgres/data_coordinator2
mkdir -p /opt/postgres/data_datanode2

chown postgres: /opt/postgres/data_*

su - postgres

initdb -D /opt/postgres/data_coordinator2 --nodename coordinator2
initdb -D /opt/postgres/data_datanode2 --nodename datanode2

#configure /opt/postgres/data_coordinator2/postgres.conf with GTM server
#configure /opt/postgres/data_datanode2/postgres.conf with GTM server

postgres -X -p 15433 -D /opt/postgres/data_datanode2 >>logfile 2>&1 &
postgres -C -D /opt/postgres/data_coordinator2 >>logfile 2>&1 &

each coordinator node:
======================

postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &
You already started your nodes at an earlier step, so this is not necessary.
An error would be returned telling that node is already started like postgres.
GTM machine:
=============

rm -fr /opt/postgres/data*

mkdir -p /opt/postgres/data_gtm

chown postgres: /opt/postgres/data*

su - postgres

initgtm -D /opt/postgres/data_gtm -Z gtm
gtm -D /opt/postgres/data_gtm >logfile 2>&1 &
You should start GTM before starting your 2 Coordinators and your 2 Datanodes.
Postgres-XC nodes need to register on GTM at start-up or they cannot start.


Registering Nodes: (all datanodes ??? has been done on both, output below)
==================

psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)" postgres
psql -c "CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433)" postgres
psql -c "SELECT pgxc_pool_reload()" postgres
select oid,node_name from pgxc_node;

Here you have to launch CREATE NODE and pgxc_pool_reload on each Coordinator. When you start up a node this node knows only about itself, so you need to register all the other Coordinators and Datanodes. Registering other Coordinators on a Coordinator is mandatory or you won't be able to synchronize DDL among Coordinators, this could result in a table being created on a Coordinator and not another.

So, in your case, launch the following queries:
- On Coordinator 1:
CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST = '$node1_ip');
CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST = '$node2_ip');
CREATE NODE coordinator2 WITH (TYPE = 'coordinator', PORT = 5432, HOST = '$node2_ip');
- On Coordinator 2:
CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST = '$node1_ip');
CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST = '$node2_ip');
CREATE NODE coordinator1 WITH (TYPE = 'coordinator', PORT = 5432, HOST = '$node2_ip');
There is a mistake here. On Coordinator 2, coordinator1 should be registered with a host set to $node1_ip, resulting in:
CREATE NODE coordinator1 WITH (TYPE = 'coordinator', PORT = 5432, HOST = '$node1_ip');
--
Michael Paquier
http://michael.otacoo.com

On Mon, Jun 25, 2012 at 10:07 AM, Michael Paquier <mic...@gm...
> wrote:

> I may be stepping someone's else answer... But here is some input.
>
> On Fri, Jun 22, 2012 at 9:30 PM, Neil Mooney <nm...@xu...> wrote:
>
>> Can anyone point out where I am going wrong here ?
>>
>> Here is a complete log of what i've done to setup , error at the end:
>>
>> Data/Coordinator machines:
>> ==========================
>>
>> node1:
>> ======
>>
>> rm -fr /opt/postgres/data*
>>
>> mkdir -p /opt/postgres/data_coordinator1
>> mkdir -p /opt/postgres/data_datanode1
>>
>> chown postgres: /opt/postgres/data_*
>>
>> su - postgres
>>
>> initdb -D /opt/postgres/data_coordinator1 --nodename coordinator1
>> initdb -D /opt/postgres/data_datanode1 --nodename datanode1
>>
>> #configure /opt/postgres/data_coordinator1/postgres.conf with GTM server
>> #configure /opt/postgres/data_datanode1/postgres.conf with GTM server
>>
>>
>> postgres -X -p 15432 -D /opt/postgres/data_datanode1 >>logfile 2>&1 &
>> postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &
>>
>> node2:
>> ======
>>
>> rm -fr /opt/postgres/data*
>>
>> mkdir -p /opt/postgres/data_coordinator2
>> mkdir -p /opt/postgres/data_datanode2
>>
>> chown postgres: /opt/postgres/data_*
>>
>> su - postgres
>>
>> initdb -D /opt/postgres/data_coordinator2 --nodename coordinator2
>> initdb -D /opt/postgres/data_datanode2 --nodename datanode2
>>
>> #configure /opt/postgres/data_coordinator2/postgres.conf with GTM server
>> #configure /opt/postgres/data_datanode2/postgres.conf with GTM server
>>
>> postgres -X -p 15433 -D /opt/postgres/data_datanode2 >>logfile 2>&1 &
>> postgres -C -D /opt/postgres/data_coordinator2 >>logfile 2>&1 &
>>
>> each coordinator node:
>> ======================
>>
>> postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &
>>
> You already started your nodes at an earlier step, so this is not
> necessary.
> An error would be returned telling that node is already started like
> postgres.
>
>> GTM machine:
>> =============
>>
>> rm -fr /opt/postgres/data*
>>
>> mkdir -p /opt/postgres/data_gtm
>>
>> chown postgres: /opt/postgres/data*
>>
>> su - postgres
>>
>> initgtm -D /opt/postgres/data_gtm -Z gtm
>> gtm -D /opt/postgres/data_gtm >logfile 2>&1 &
>>
> You should start GTM *before* starting your 2 Coordinators and your 2
> Datanodes.
> Postgres-XC nodes need to register on GTM at start-up or they cannot start.
>
>
>>
>> Registering Nodes: (all datanodes ??? has been done on both, output below)
>> ==================
>>
>> psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
>> postgres
>> psql -c "CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433)"
>> postgres
>> psql -c "SELECT pgxc_pool_reload()" postgres
>> select oid,node_name from pgxc_node;
>>
>
> Here you have to launch CREATE NODE and pgxc_pool_reload on *each
> Coordinator*. When you start up a node this node knows only about itself,
> so you need to register all the other Coordinators and Datanodes.
> Registering other Coordinators on a Coordinator is mandatory or you won't
> be able to synchronize DDL among Coordinators, this could result in a table
> being created on a Coordinator and not another.
>
> So, in your case, launch the following queries:
> - On Coordinator 1:
> CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST =
> '$node1_ip');
> CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST =
> '$node2_ip');
> CREATE NODE coordinator2 WITH (TYPE = 'coordinator', PORT = 5432, HOST =
> '$node2_ip');
> - On Coordinator 2:
> CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST =
> '$node1_ip');
> CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST =
> '$node2_ip');
> CREATE NODE coordinator1 WITH (TYPE = 'coordinator', PORT = 5432, HOST =
> '$node2_ip');
>
There is a mistake here. On Coordinator 2, coordinator1 should be
registered with a host set to $node1_ip, resulting in:
CREATE NODE coordinator1 WITH (TYPE = 'coordinator', PORT = 5432, HOST =
'$node1_ip');
-- 
Michael Paquier
http://michael.otacoo.com

I may be stepping someone's else answer... But here is some input.

On Fri, Jun 22, 2012 at 9:30 PM, Neil Mooney <nm...@xu...> wrote:

> Can anyone point out where I am going wrong here ?
>
> Here is a complete log of what i've done to setup , error at the end:
>
> Data/Coordinator machines:
> ==========================
>
> node1:
> ======
>
> rm -fr /opt/postgres/data*
>
> mkdir -p /opt/postgres/data_coordinator1
> mkdir -p /opt/postgres/data_datanode1
>
> chown postgres: /opt/postgres/data_*
>
> su - postgres
>
> initdb -D /opt/postgres/data_coordinator1 --nodename coordinator1
> initdb -D /opt/postgres/data_datanode1 --nodename datanode1
>
> #configure /opt/postgres/data_coordinator1/postgres.conf with GTM server
> #configure /opt/postgres/data_datanode1/postgres.conf with GTM server
>
>
> postgres -X -p 15432 -D /opt/postgres/data_datanode1 >>logfile 2>&1 &
> postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &
>
> node2:
> ======
>
> rm -fr /opt/postgres/data*
>
> mkdir -p /opt/postgres/data_coordinator2
> mkdir -p /opt/postgres/data_datanode2
>
> chown postgres: /opt/postgres/data_*
>
> su - postgres
>
> initdb -D /opt/postgres/data_coordinator2 --nodename coordinator2
> initdb -D /opt/postgres/data_datanode2 --nodename datanode2
>
> #configure /opt/postgres/data_coordinator2/postgres.conf with GTM server
> #configure /opt/postgres/data_datanode2/postgres.conf with GTM server
>
> postgres -X -p 15433 -D /opt/postgres/data_datanode2 >>logfile 2>&1 &
> postgres -C -D /opt/postgres/data_coordinator2 >>logfile 2>&1 &
>
> each coordinator node:
> ======================
>
> postgres -C -D /opt/postgres/data_coordinator1 >>logfile 2>&1 &
>
You already started your nodes at an earlier step, so this is not necessary.
An error would be returned telling that node is already started like
postgres.

> GTM machine:
> =============
>
> rm -fr /opt/postgres/data*
>
> mkdir -p /opt/postgres/data_gtm
>
> chown postgres: /opt/postgres/data*
>
> su - postgres
>
> initgtm -D /opt/postgres/data_gtm -Z gtm
> gtm -D /opt/postgres/data_gtm >logfile 2>&1 &
>
You should start GTM *before* starting your 2 Coordinators and your 2
Datanodes.
Postgres-XC nodes need to register on GTM at start-up or they cannot start.


>
> Registering Nodes: (all datanodes ??? has been done on both, output below)
> ==================
>
> psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
> postgres
> psql -c "CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433)"
> postgres
> psql -c "SELECT pgxc_pool_reload()" postgres
> select oid,node_name from pgxc_node;
>

Here you have to launch CREATE NODE and pgxc_pool_reload on *each
Coordinator*. When you start up a node this node knows only about itself,
so you need to register all the other Coordinators and Datanodes.
Registering other Coordinators on a Coordinator is mandatory or you won't
be able to synchronize DDL among Coordinators, this could result in a table
being created on a Coordinator and not another.

So, in your case, launch the following queries:
- On Coordinator 1:
CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST =
'$node1_ip');
CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST =
'$node2_ip');
CREATE NODE coordinator2 WITH (TYPE = 'coordinator', PORT = 5432, HOST =
'$node2_ip');
- On Coordinator 2:
CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST =
'$node1_ip');
CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433, HOST =
'$node2_ip');
CREATE NODE coordinator1 WITH (TYPE = 'coordinator', PORT = 5432, HOST =
'$node2_ip');

In the case of node1, datanode1 and coordinator1 are on the same node. So
you can replace $node1_ip with localhost or 127.0.0.1 when defining
datanode1 for coordinator1.
Same for node node2, datanode2 and coordinator2 are on the same node. So
you can replace $node2_ip with localhost or 127.0.0.1 when defining
datanode2 for coordinator2.



> #output
>
> node1:
> postgres=# select oid,node_name from pgxc_node;
>  oid  |  node_name
> -------+--------------
>  11129 | coordinator1
>  16384 | datanode1
>  16385 | datanode2
> (3 rows)
>
This output is incorrect.
Coordinator 1 is not aware of Coordinator 2. Coordinator 1 also thinks that
datanode2 is on node1 as you did not define an IP to redirect correctly.


>
>
> node2:
> [postgres@las1-app015 ~]$ psql -c "select oid,node_name from pgxc_node;"
> postgres
>  oid  |  node_name
> -------+--------------
>  11129 | coordinator2
>  16384 | datanode1
>  16385 | datanode2
> (3 rows)
>
Same here, Coordinator 1 is not aware of Coordinator2.
Coordinator2 thinks that datanode1 is on the same server, which is not the
case as it looks that coordinator2/datanode2 and coordinator1/datanode1 are
grouped on 2 different nodes.

>
> Creating DB's :
> ===============
>
> [postgres@las1-app014 ~]$ createdb test
> createdb: database creation failed: ERROR:  Failed to get pooled
> connections
>

Here I guess you are connecting to coordinator1.
This error happens because coordinator1 cannot connect to datanode2, as it
is not located on local server but on server node2.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com