Thread: [Postgres-xc-general] coordinator deaths

Hello,

I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
1.0.1 as the back end for a new project.  I would've searched prior posts
for similar ones before posting, but I don't see any way to do that?

I'm using CentOS 6.3, I've used the "short version" installation to set up
a gtm, a single coordinator, and a single data node on my local host.  Here
are the contents of my pgxc_node table:

-bash-4.1$ psql -U postgres
psql (PGXC 1.0.1, based on PG 9.1.5)
Type "help" for help.
postgres=# select * from pgxc_node;
 node_name | node_type | node_port | node_host | nodeis_primary |
nodeis_preferred |  node_id
-----------+-----------+-----------+-----------+----------------+------------------+------------
 coord1    | C         |      5432 | localhost | f              | f
       | 1885696643
 datanode1 | D         |     15432 | localhost | t              | t
       |  888802358


The problem I'm having is that after maybe 60 to 90 seconds or so after
starting, the coordinator dies with a log message like

FATAL:  Can not register Coordinator on GTM

The gtm is definitely running, I can see it in the process list.  The only
changes to the default postgresql.conf's I've made is to specify the port
of 15432 for datanode1, and (as part of an effort to fix this problem)
uncomment the host ('localhost') and port (6666) of the gtm in all 3
postgresql.conf files.

Another thing I did to try to fix this was:

alter node datanode1 with (primary = true, preferred = true);

As you can see, that has affected the pgxc_node contents.  But that didn't
have any impact on this crashing behavior.

I tried starting using the method presented in the short version install to
start everything, and I also tried starting up like this:

gtm_ctl -Z gtm -D data_gtm -l gtm.log start
sleep 1
pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
sleep 1
pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start

..from the /usr/local/pgsql directory.  (I also tried specifying absolute
paths.)

I've tried using kill to clear everything out of the process table, I've
tried making sure the last shutdown was clean for everything, using pg_ctl
and gtm_ctl to shutdown, and then starting up fresh from having shutdown
cleanly.

SELinux is disabled, the local firewall is disabled.

Everything works for about 60 to 90 seconds, then the coordinator dies with
the message above.  Any help would be greatly appreciated,

Shavais

Hi,

You were not registered on the general mailing list, so I had to check the
content of your message manually.
This explains the delay of your message.

By the way, regarding your message, please see comments below.

On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:

> I'm using CentOS 6.3, I've used the "short version" installation to set up
> a gtm, a single coordinator, and a single data node on my local host.  Here
> are the contents of my pgxc_node table:
>
> -bash-4.1$ psql -U postgres
> psql (PGXC 1.0.1, based on PG 9.1.5)
> Type "help" for help.
> postgres=# select * from pgxc_node;
>  node_name | node_type | node_port | node_host | nodeis_primary |
> nodeis_preferred |  node_id
>
> -----------+-----------+-----------+-----------+----------------+------------------+------------
>  coord1    | C         |      5432 | localhost | f              | f
>          | 1885696643
>  datanode1 | D         |     15432 | localhost | t              | t
>          |  888802358
>
I assume that you used the short install document from here:
http://postgres-xc.sourceforge.net/docs/1_0/install-short.html

I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
datanode on the Coordinator.
This looks definitely correct.

The problem I'm having is that after maybe 60 to 90 seconds or so after
> starting, the coordinator dies with a log message like
>
> FATAL:  Can not register Coordinator on GTM
>
This basically means that the Coordinator is trying to register to GTM with
a node name that is already used by another node.
What is the value of pgxc_node_name in each node's postgresql.conf?

The gtm is definitely running, I can see it in the process list.  The only
> changes to the default postgresql.conf's I've made is to specify the port
> of 15432 for datanode1, and (as part of an effort to fix this problem)
> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> postgresql.conf files.
>
For a really basic setting, changing the port number of datanode1 is
enough, as well as specifying the port value with -p at startup.
The value specified by option -p has higher priority than the value of port
in postgresql.conf.


> Another thing I did to try to fix this was:
>
> alter node datanode1 with (primary = true, preferred = true);
>
Primary nodes are the nodes used in priority for replicated writes.
Preferred nodes are the nodes used in priority for replicated reads. So
changing this setting won't solve anything in your case.


>
> As you can see, that has affected the pgxc_node contents.  But that didn't
> have any impact on this crashing behavior.
>
> I tried starting using the method presented in the short version install
> to start everything, and I also tried starting up like this:
>
> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> sleep 1
> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> sleep 1
> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
>
> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> paths.)
>
Absolute and non-absolute paths both work, like postgres.

In your case, I suggest to check for the values of pgxc_node_name in
postgresql.conf. Please be sure that they are set to the same values as
node_name in pgxc_node.
What you can also check is the data of the file called register.node in the
data folder of GTM. The data of this file is binary, but you may be able to
check the data folders of the nodes registered on GTM even by opening the
file manually.

Please also try the following thing if problem persists. This simple set of
commands allows to start a cluster with 1 Coordinator and 1 Datanode:
mkdir $HOME/pgsql/data_coord1
mkdir $HOME/pgsql/data_datanode1
mkdir $HOME/pgsql/data_gtm
initdb -D $HOME/pgsql/data_coord1 --nodename coord1
initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
initgtm -D $HOME/pgsql/data_gtm -Z gtm
gtm -D $HOME/pgsql/data_gtm &
postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
postgres -C -D $HOME/pgsql/data_coord1 &
psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
postgres
psql -c "SELECT pgxc_pool_reload()" postgres
createdb test
psql test

In my case, this works perfectly with XC 1.0.1, even with firewall and
SELinux in my box.

Then why not testing pgbench?
pgbench -i test
pgbench -c 24 -S test

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Hello,

I've been using a computer on which gmail was already logged in.  I
mistakenly assumed it was logged into my account.  Sorry about that.

The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
respectively.

I reinitialized the install, as you suggested (except I just shut
everything down, blew away the data_* folders in /usr/local/pgsql, and
reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
to have fixed the problem.  (I guess probably those data_* folders should
be under /var somewhere..)

I'm not sure what the problem was.  I think I recall having attempted to
issue a sql command to create a coordinator node, the previous time around.
 I notice your steps don't include that step?  I wonder if somehow that
affected things?

The reason I had set the port in datanode1's postgresql.conf was because I
would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
init.d script, (ala service postgresql-xc start / restart / stop) and I
haven't been able to get pg_ctl to pass the port parameter to postgres.
(pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
usage docs).  After putting that setting back in again, I'm using gtm_ctl
and pg_ctl to start and stop it all at this point, and it seems to working
fine.

The pgbench utility wasn't installed, but I finally found it and installed
it.  I notice that it somehow left a bunch of idle connection processes
behind after exiting.  Would those get reused, or is that a problem with
pgbench?  (I restarted postgres-xc and they went away, but.  Just
wondering.)


Shavais

On Tue, Sep 18, 2012 at 4:01 PM, Michael Paquier
<mic...@gm...>wrote:

> Hi,
>
> You were not registered on the general mailing list, so I had to check the
> content of your message manually.
> This explains the delay of your message.
>
> By the way, regarding your message, please see comments below.
>
> On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:
>
>> I'm using CentOS 6.3, I've used the "short version" installation to set
>> up a gtm, a single coordinator, and a single data node on my local host.
>>  Here are the contents of my pgxc_node table:
>>
>> -bash-4.1$ psql -U postgres
>> psql (PGXC 1.0.1, based on PG 9.1.5)
>> Type "help" for help.
>> postgres=# select * from pgxc_node;
>>  node_name | node_type | node_port | node_host | nodeis_primary |
>> nodeis_preferred |  node_id
>>
>> -----------+-----------+-----------+-----------+----------------+------------------+------------
>>  coord1    | C         |      5432 | localhost | f              | f
>>          | 1885696643
>>  datanode1 | D         |     15432 | localhost | t              | t
>>          |  888802358
>>
> I assume that you used the short install document from here:
> http://postgres-xc.sourceforge.net/docs/1_0/install-short.html
>
> I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
> datanode on the Coordinator.
> This looks definitely correct.
>
>  The problem I'm having is that after maybe 60 to 90 seconds or so after
>> starting, the coordinator dies with a log message like
>>
>> FATAL:  Can not register Coordinator on GTM
>>
> This basically means that the Coordinator is trying to register to GTM
> with a node name that is already used by another node.
> What is the value of pgxc_node_name in each node's postgresql.conf?
>
> The gtm is definitely running, I can see it in the process list.  The only
>> changes to the default postgresql.conf's I've made is to specify the port
>> of 15432 for datanode1, and (as part of an effort to fix this problem)
>> uncomment the host ('localhost') and port (6666) of the gtm in all 3
>> postgresql.conf files.
>>
> For a really basic setting, changing the port number of datanode1 is
> enough, as well as specifying the port value with -p at startup.
> The value specified by option -p has higher priority than the value of
> port in postgresql.conf.
>
>
>> Another thing I did to try to fix this was:
>>
>> alter node datanode1 with (primary = true, preferred = true);
>>
> Primary nodes are the nodes used in priority for replicated writes.
> Preferred nodes are the nodes used in priority for replicated reads. So
> changing this setting won't solve anything in your case.
>
>
>>
>> As you can see, that has affected the pgxc_node contents.  But that
>> didn't have any impact on this crashing behavior.
>>
>> I tried starting using the method presented in the short version install
>> to start everything, and I also tried starting up like this:
>>
>> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
>> sleep 1
>> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
>> sleep 1
>> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
>>
>> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
>> paths.)
>>
> Absolute and non-absolute paths both work, like postgres.
>
> In your case, I suggest to check for the values of pgxc_node_name in
> postgresql.conf. Please be sure that they are set to the same values as
> node_name in pgxc_node.
> What you can also check is the data of the file called register.node in
> the data folder of GTM. The data of this file is binary, but you may be
> able to check the data folders of the nodes registered on GTM even by
> opening the file manually.
>
> Please also try the following thing if problem persists. This simple set
> of commands allows to start a cluster with 1 Coordinator and 1 Datanode:
> mkdir $HOME/pgsql/data_coord1
> mkdir $HOME/pgsql/data_datanode1
> mkdir $HOME/pgsql/data_gtm
> initdb -D $HOME/pgsql/data_coord1 --nodename coord1
> initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
> initgtm -D $HOME/pgsql/data_gtm -Z gtm
> gtm -D $HOME/pgsql/data_gtm &
> postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
> postgres -C -D $HOME/pgsql/data_coord1 &
> psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
> postgres
> psql -c "SELECT pgxc_pool_reload()" postgres
> createdb test
> psql test
>
> In my case, this works perfectly with XC 1.0.1, even with firewall and
> SELinux in my box.
>
> Then why not testing pgbench?
> pgbench -i test
> pgbench -c 24 -S test
>
> Regards,
> --
> Michael Paquier
> http://michael.otacoo.com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

Did you do any cleanup for gtm resources?    Sorry I wrote a bit late.   The message suggests that coordinator/datade attempted to register itself to GTM which has alrady been registered.

If you have similar problem, please try to cleanup register.node file at gtm work directory.   This should not bring any problem if you stop gtm (and all other nodes) when you cleanup.

Regards;
---
Koichi Suzuki

On Tue, 18 Sep 2012 18:12:34 -0700
Shavais Zarathustra <sh...@gm...> wrote:

> Hello,
> 
> I've been using a computer on which gmail was already logged in.  I
> mistakenly assumed it was logged into my account.  Sorry about that.
> 
> The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
> data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
> respectively.
> 
> I reinitialized the install, as you suggested (except I just shut
> everything down, blew away the data_* folders in /usr/local/pgsql, and
> reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
> to have fixed the problem.  (I guess probably those data_* folders should
> be under /var somewhere..)
> 
> I'm not sure what the problem was.  I think I recall having attempted to
> issue a sql command to create a coordinator node, the previous time around.
>  I notice your steps don't include that step?  I wonder if somehow that
> affected things?
> 
> The reason I had set the port in datanode1's postgresql.conf was because I
> would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
> init.d script, (ala service postgresql-xc start / restart / stop) and I
> haven't been able to get pg_ctl to pass the port parameter to postgres.
> (pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
> usage docs).  After putting that setting back in again, I'm using gtm_ctl
> and pg_ctl to start and stop it all at this point, and it seems to working
> fine.
> 
> The pgbench utility wasn't installed, but I finally found it and installed
> it.  I notice that it somehow left a bunch of idle connection processes
> behind after exiting.  Would those get reused, or is that a problem with
> pgbench?  (I restarted postgres-xc and they went away, but.  Just
> wondering.)
> 
> 
> Shavais
> 
> On Tue, Sep 18, 2012 at 4:01 PM, Michael Paquier
> <mic...@gm...>wrote:
> 
> > Hi,
> >
> > You were not registered on the general mailing list, so I had to check the
> > content of your message manually.
> > This explains the delay of your message.
> >
> > By the way, regarding your message, please see comments below.
> >
> > On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:
> >
> >> I'm using CentOS 6.3, I've used the "short version" installation to set
> >> up a gtm, a single coordinator, and a single data node on my local host.
> >>  Here are the contents of my pgxc_node table:
> >>
> >> -bash-4.1$ psql -U postgres
> >> psql (PGXC 1.0.1, based on PG 9.1.5)
> >> Type "help" for help.
> >> postgres=# select * from pgxc_node;
> >>  node_name | node_type | node_port | node_host | nodeis_primary |
> >> nodeis_preferred |  node_id
> >>
> >> -----------+-----------+-----------+-----------+----------------+------------------+------------
> >>  coord1    | C         |      5432 | localhost | f              | f
> >>          | 1885696643
> >>  datanode1 | D         |     15432 | localhost | t              | t
> >>          |  888802358
> >>
> > I assume that you used the short install document from here:
> > http://postgres-xc.sourceforge.net/docs/1_0/install-short.html
> >
> > I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
> > datanode on the Coordinator.
> > This looks definitely correct.
> >
> >  The problem I'm having is that after maybe 60 to 90 seconds or so after
> >> starting, the coordinator dies with a log message like
> >>
> >> FATAL:  Can not register Coordinator on GTM
> >>
> > This basically means that the Coordinator is trying to register to GTM
> > with a node name that is already used by another node.
> > What is the value of pgxc_node_name in each node's postgresql.conf?
> >
> > The gtm is definitely running, I can see it in the process list.  The only
> >> changes to the default postgresql.conf's I've made is to specify the port
> >> of 15432 for datanode1, and (as part of an effort to fix this problem)
> >> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> >> postgresql.conf files.
> >>
> > For a really basic setting, changing the port number of datanode1 is
> > enough, as well as specifying the port value with -p at startup.
> > The value specified by option -p has higher priority than the value of
> > port in postgresql.conf.
> >
> >
> >> Another thing I did to try to fix this was:
> >>
> >> alter node datanode1 with (primary = true, preferred = true);
> >>
> > Primary nodes are the nodes used in priority for replicated writes.
> > Preferred nodes are the nodes used in priority for replicated reads. So
> > changing this setting won't solve anything in your case.
> >
> >
> >>
> >> As you can see, that has affected the pgxc_node contents.  But that
> >> didn't have any impact on this crashing behavior.
> >>
> >> I tried starting using the method presented in the short version install
> >> to start everything, and I also tried starting up like this:
> >>
> >> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> >> sleep 1
> >> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> >> sleep 1
> >> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> >>
> >> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> >> paths.)
> >>
> > Absolute and non-absolute paths both work, like postgres.
> >
> > In your case, I suggest to check for the values of pgxc_node_name in
> > postgresql.conf. Please be sure that they are set to the same values as
> > node_name in pgxc_node.
> > What you can also check is the data of the file called register.node in
> > the data folder of GTM. The data of this file is binary, but you may be
> > able to check the data folders of the nodes registered on GTM even by
> > opening the file manually.
> >
> > Please also try the following thing if problem persists. This simple set
> > of commands allows to start a cluster with 1 Coordinator and 1 Datanode:
> > mkdir $HOME/pgsql/data_coord1
> > mkdir $HOME/pgsql/data_datanode1
> > mkdir $HOME/pgsql/data_gtm
> > initdb -D $HOME/pgsql/data_coord1 --nodename coord1
> > initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
> > initgtm -D $HOME/pgsql/data_gtm -Z gtm
> > gtm -D $HOME/pgsql/data_gtm &
> > postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
> > postgres -C -D $HOME/pgsql/data_coord1 &
> > psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
> > postgres
> > psql -c "SELECT pgxc_pool_reload()" postgres
> > createdb test
> > psql test
> >
> > In my case, this works perfectly with XC 1.0.1, even with firewall and
> > SELinux in my box.
> >
> > Then why not testing pgbench?
> > pgbench -i test
> > pgbench -c 24 -S test
> >
> > Regards,
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> > ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
> >

Hi,

Maybe coordinator or datanode is trying to register itself doubly to gtm.

To solve this problem, please try the following:

1. Stop gtm, coordinator and datanode,
2. Visit gtm's work directory (you should have specified by -D option with gtm_ctl)
3. Rename the file register.node to something else
4. Restart everything.

BTW, I've posted bash script pgxclocal which takes care of XC installation, start and stop for local installation just for test.

This is available from https://github.com/koichi-szk/PGXC-Tools.git
Current version is attached.

This will help to configure XC in local environment.   The script consists of two coordinators and two datanodes.   You can rewrite this section as you like.   Also, this comes with  a set of functions which shows what you should do to run your XC cluster.

Good luck;
---
Koichi Suzuki
# I'm rewriting this into general Postgres-XC operating utility called pgxc_ctl.

On Tue, 18 Sep 2012 12:51:57 -0700
Roger Mayes <rog...@gm...> wrote:

> Hello,
> 
> I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
> 1.0.1 as the back end for a new project.  I would've searched prior posts
> for similar ones before posting, but I don't see any way to do that?
> 
> I'm using CentOS 6.3, I've used the "short version" installation to set up
> a gtm, a single coordinator, and a single data node on my local host.  Here
> are the contents of my pgxc_node table:
> 
> -bash-4.1$ psql -U postgres
> psql (PGXC 1.0.1, based on PG 9.1.5)
> Type "help" for help.
> postgres=# select * from pgxc_node;
>  node_name | node_type | node_port | node_host | nodeis_primary |
> nodeis_preferred |  node_id
> -----------+-----------+-----------+-----------+----------------+------------------+------------
>  coord1    | C         |      5432 | localhost | f              | f
>        | 1885696643
>  datanode1 | D         |     15432 | localhost | t              | t
>        |  888802358
> 
> 
> The problem I'm having is that after maybe 60 to 90 seconds or so after
> starting, the coordinator dies with a log message like
> 
> FATAL:  Can not register Coordinator on GTM
> 
> The gtm is definitely running, I can see it in the process list.  The only
> changes to the default postgresql.conf's I've made is to specify the port
> of 15432 for datanode1, and (as part of an effort to fix this problem)
> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> postgresql.conf files.
> 
> Another thing I did to try to fix this was:
> 
> alter node datanode1 with (primary = true, preferred = true);
> 
> As you can see, that has affected the pgxc_node contents.  But that didn't
> have any impact on this crashing behavior.
> 
> I tried starting using the method presented in the short version install to
> start everything, and I also tried starting up like this:
> 
> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> sleep 1
> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> sleep 1
> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> 
> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> paths.)
> 
> I've tried using kill to clear everything out of the process table, I've
> tried making sure the last shutdown was clean for everything, using pg_ctl
> and gtm_ctl to shutdown, and then starting up fresh from having shutdown
> cleanly.
> 
> SELinux is disabled, the local firewall is disabled.
> 
> Everything works for about 60 to 90 seconds, then the coordinator dies with
> the message above.  Any help would be greatly appreciated,
> 
> Shavais

Hello,

This looks like it has a lot of helpful stuff in it, it can dissipate a
fair amount of fogginess by providing an example.

One of the things I'll want to do is create a script in /etc/init.d that I
can use with chkconfig, so that things shutdown and start up cleanly on
server reboot.  It seems like it would be nice to have a configuration file
under /etc/config.d, or in /etc/postgresql-xc or something, that the init.d
script would read in order to know everything it needed to know, like
whether or not a gtm is present, and what coordinators and datanodes are
present and where their data directories are, and what their names and
ports are and such.


Shavais

On Tue, Sep 18, 2012 at 6:28 PM, Koichi Suzuki <ko...@in...>wrote:

> Hi,
>
> Maybe coordinator or datanode is trying to register itself doubly to gtm.
>
> To solve this problem, please try the following:
>
> 1. Stop gtm, coordinator and datanode,
> 2. Visit gtm's work directory (you should have specified by -D option with
> gtm_ctl)
> 3. Rename the file register.node to something else
> 4. Restart everything.
>
> BTW, I've posted bash script pgxclocal which takes care of XC
> installation, start and stop for local installation just for test.
>
> This is available from https://github.com/koichi-szk/PGXC-Tools.git
> Current version is attached.
>
> This will help to configure XC in local environment.   The script consists
> of two coordinators and two datanodes.   You can rewrite this section as
> you like.   Also, this comes with  a set of functions which shows what you
> should do to run your XC cluster.
>
> Good luck;
> ---
> Koichi Suzuki
> # I'm rewriting this into general Postgres-XC operating utility called
> pgxc_ctl.
>
> On Tue, 18 Sep 2012 12:51:57 -0700
> Roger Mayes <rog...@gm...> wrote:
>
> > Hello,
> >
> > I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
> > 1.0.1 as the back end for a new project.  I would've searched prior posts
> > for similar ones before posting, but I don't see any way to do that?
> >
> > I'm using CentOS 6.3, I've used the "short version" installation to set
> up
> > a gtm, a single coordinator, and a single data node on my local host.
>  Here
> > are the contents of my pgxc_node table:
> >
> > -bash-4.1$ psql -U postgres
> > psql (PGXC 1.0.1, based on PG 9.1.5)
> > Type "help" for help.
> > postgres=# select * from pgxc_node;
> >  node_name | node_type | node_port | node_host | nodeis_primary |
> > nodeis_preferred |  node_id
> >
> -----------+-----------+-----------+-----------+----------------+------------------+------------
> >  coord1    | C         |      5432 | localhost | f              | f
> >        | 1885696643
> >  datanode1 | D         |     15432 | localhost | t              | t
> >        |  888802358
> >
> >
> > The problem I'm having is that after maybe 60 to 90 seconds or so after
> > starting, the coordinator dies with a log message like
> >
> > FATAL:  Can not register Coordinator on GTM
> >
> > The gtm is definitely running, I can see it in the process list.  The
> only
> > changes to the default postgresql.conf's I've made is to specify the port
> > of 15432 for datanode1, and (as part of an effort to fix this problem)
> > uncomment the host ('localhost') and port (6666) of the gtm in all 3
> > postgresql.conf files.
> >
> > Another thing I did to try to fix this was:
> >
> > alter node datanode1 with (primary = true, preferred = true);
> >
> > As you can see, that has affected the pgxc_node contents.  But that
> didn't
> > have any impact on this crashing behavior.
> >
> > I tried starting using the method presented in the short version install
> to
> > start everything, and I also tried starting up like this:
> >
> > gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> > sleep 1
> > pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> > sleep 1
> > pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> >
> > ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> > paths.)
> >
> > I've tried using kill to clear everything out of the process table, I've
> > tried making sure the last shutdown was clean for everything, using
> pg_ctl
> > and gtm_ctl to shutdown, and then starting up fresh from having shutdown
> > cleanly.
> >
> > SELinux is disabled, the local firewall is disabled.
> >
> > Everything works for about 60 to 90 seconds, then the coordinator dies
> with
> > the message above.  Any help would be greatly appreciated,
> >
> > Shavais
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

On Wed, Sep 19, 2012 at 10:12 AM, Shavais Zarathustra <sh...@gm...>wrote:

> Hello,
>
> I've been using a computer on which gmail was already logged in.  I
> mistakenly assumed it was logged into my account.  Sorry about that.
>
Don't worry :)

>
> The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
> data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
> respectively.
>
OK, nice.

>
> I reinitialized the install, as you suggested (except I just shut
> everything down, blew away the data_* folders in /usr/local/pgsql, and
> reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
> to have fixed the problem.  (I guess probably those data_* folders should
> be under /var somewhere..)
>
> I'm not sure what the problem was.  I think I recall having attempted to
> issue a sql command to create a coordinator node, the previous time around.
>  I notice your steps don't include that step?  I wonder if somehow that
> affected things?
>
When you initialized a node with initdb, --nodename is a mandatory option.
This option value is used to set up pgxc_node_name automatically and
initialize the catalog pgxc_node with the data of the node itself.
You need to create a Coordinator with CREATE NODE on a Coordinator only if
there is at least 2 Coordinators in your cluster.


> The reason I had set the port in datanode1's postgresql.conf was because I
> would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
> init.d script, (ala service postgresql-xc start / restart / stop) and I
> haven't been able to get pg_ctl to pass the port parameter to postgres.
> (pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
> usage docs).  After putting that setting back in again, I'm using gtm_ctl
> and pg_ctl to start and stop it all at this point, and it seems to working
> fine.
>
Nice to hear that it is working. I am sure you missed something at
initialization.
When you want to add a port number with pg_ctl, be sure to use quotes with
-o, like:
-o "-p $PORT"

The XC documentation is not perfect, if you could point to the page and the
paragraph where you had doubts I can try to have a look and correct the doc
if necessary.


> The pgbench utility wasn't installed, but I finally found it and installed
> it.  I notice that it somehow left a bunch of idle connection processes
> behind after exiting.  Would those get reused, or is that a problem with
> pgbench?  (I restarted postgres-xc and they went away, but.  Just
> wondering.)
>
The connection pooling of XC is made to maintain connections between nodes
as it consumes a bunch of ressources to create connections on the fly for
each transaction.
You can however drop those connections with CLEAN CONNECTION, command
specific to XC:
http://postgres-xc.sourceforge.net/docs/1_0/sql-cleanconnection.html
-- 
Michael Paquier
http://michael.otacoo.com

When you want to add a port number with pg_ctl, be sure to use quotes with
> -o, like:
> -o "-p $PORT"
>
> The XC documentation is not perfect, if you could point to the page and
> the paragraph where you had doubts I can try to have a look and correct the
> doc if necessary.
>

After looking at the docs of alternatives to XC that I've looked at, I
appreciate the XC docs a lot.

The pg_ctl --help screen says  "-o OPTIONS".  I feel a bit dumb for not
considering that the options probably need to be in quotes.


>
>
>> The pgbench utility wasn't installed, but I finally found it and
>> installed it.  I notice that it somehow left a bunch of idle connection
>> processes behind after exiting.  Would those get reused, or is that a
>> problem with pgbench?  (I restarted postgres-xc and they went away, but.
>>  Just wondering.)
>>
> The connection pooling of XC is made to maintain connections between nodes
> as it consumes a bunch of ressources to create connections on the fly for
> each transaction.
> You can however drop those connections with CLEAN CONNECTION, command
> specific to XC:
> http://postgres-xc.sourceforge.net/docs/1_0/sql-cleanconnection.html
>
>
Nice..