postgres-xc-general Mailing List for Postgres-XC

Hello,

This looks like it has a lot of helpful stuff in it, it can dissipate a
fair amount of fogginess by providing an example.

One of the things I'll want to do is create a script in /etc/init.d that I
can use with chkconfig, so that things shutdown and start up cleanly on
server reboot.  It seems like it would be nice to have a configuration file
under /etc/config.d, or in /etc/postgresql-xc or something, that the init.d
script would read in order to know everything it needed to know, like
whether or not a gtm is present, and what coordinators and datanodes are
present and where their data directories are, and what their names and
ports are and such.


Shavais

On Tue, Sep 18, 2012 at 6:28 PM, Koichi Suzuki <ko...@in...>wrote:

> Hi,
>
> Maybe coordinator or datanode is trying to register itself doubly to gtm.
>
> To solve this problem, please try the following:
>
> 1. Stop gtm, coordinator and datanode,
> 2. Visit gtm's work directory (you should have specified by -D option with
> gtm_ctl)
> 3. Rename the file register.node to something else
> 4. Restart everything.
>
> BTW, I've posted bash script pgxclocal which takes care of XC
> installation, start and stop for local installation just for test.
>
> This is available from https://github.com/koichi-szk/PGXC-Tools.git
> Current version is attached.
>
> This will help to configure XC in local environment.   The script consists
> of two coordinators and two datanodes.   You can rewrite this section as
> you like.   Also, this comes with  a set of functions which shows what you
> should do to run your XC cluster.
>
> Good luck;
> ---
> Koichi Suzuki
> # I'm rewriting this into general Postgres-XC operating utility called
> pgxc_ctl.
>
> On Tue, 18 Sep 2012 12:51:57 -0700
> Roger Mayes <rog...@gm...> wrote:
>
> > Hello,
> >
> > I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
> > 1.0.1 as the back end for a new project.  I would've searched prior posts
> > for similar ones before posting, but I don't see any way to do that?
> >
> > I'm using CentOS 6.3, I've used the "short version" installation to set
> up
> > a gtm, a single coordinator, and a single data node on my local host.
>  Here
> > are the contents of my pgxc_node table:
> >
> > -bash-4.1$ psql -U postgres
> > psql (PGXC 1.0.1, based on PG 9.1.5)
> > Type "help" for help.
> > postgres=# select * from pgxc_node;
> >  node_name | node_type | node_port | node_host | nodeis_primary |
> > nodeis_preferred |  node_id
> >
> -----------+-----------+-----------+-----------+----------------+------------------+------------
> >  coord1    | C         |      5432 | localhost | f              | f
> >        | 1885696643
> >  datanode1 | D         |     15432 | localhost | t              | t
> >        |  888802358
> >
> >
> > The problem I'm having is that after maybe 60 to 90 seconds or so after
> > starting, the coordinator dies with a log message like
> >
> > FATAL:  Can not register Coordinator on GTM
> >
> > The gtm is definitely running, I can see it in the process list.  The
> only
> > changes to the default postgresql.conf's I've made is to specify the port
> > of 15432 for datanode1, and (as part of an effort to fix this problem)
> > uncomment the host ('localhost') and port (6666) of the gtm in all 3
> > postgresql.conf files.
> >
> > Another thing I did to try to fix this was:
> >
> > alter node datanode1 with (primary = true, preferred = true);
> >
> > As you can see, that has affected the pgxc_node contents.  But that
> didn't
> > have any impact on this crashing behavior.
> >
> > I tried starting using the method presented in the short version install
> to
> > start everything, and I also tried starting up like this:
> >
> > gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> > sleep 1
> > pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> > sleep 1
> > pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> >
> > ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> > paths.)
> >
> > I've tried using kill to clear everything out of the process table, I've
> > tried making sure the last shutdown was clean for everything, using
> pg_ctl
> > and gtm_ctl to shutdown, and then starting up fresh from having shutdown
> > cleanly.
> >
> > SELinux is disabled, the local firewall is disabled.
> >
> > Everything works for about 60 to 90 seconds, then the coordinator dies
> with
> > the message above.  Any help would be greatly appreciated,
> >
> > Shavais
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

When you want to add a port number with pg_ctl, be sure to use quotes with
> -o, like:
> -o "-p $PORT"
>
> The XC documentation is not perfect, if you could point to the page and
> the paragraph where you had doubts I can try to have a look and correct the
> doc if necessary.
>

After looking at the docs of alternatives to XC that I've looked at, I
appreciate the XC docs a lot.

The pg_ctl --help screen says  "-o OPTIONS".  I feel a bit dumb for not
considering that the options probably need to be in quotes.


>
>
>> The pgbench utility wasn't installed, but I finally found it and
>> installed it.  I notice that it somehow left a bunch of idle connection
>> processes behind after exiting.  Would those get reused, or is that a
>> problem with pgbench?  (I restarted postgres-xc and they went away, but.
>>  Just wondering.)
>>
> The connection pooling of XC is made to maintain connections between nodes
> as it consumes a bunch of ressources to create connections on the fly for
> each transaction.
> You can however drop those connections with CLEAN CONNECTION, command
> specific to XC:
> http://postgres-xc.sourceforge.net/docs/1_0/sql-cleanconnection.html
>
>
Nice..

On Wed, Sep 19, 2012 at 10:12 AM, Shavais Zarathustra <sh...@gm...>wrote:

> Hello,
>
> I've been using a computer on which gmail was already logged in.  I
> mistakenly assumed it was logged into my account.  Sorry about that.
>
Don't worry :)

>
> The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
> data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
> respectively.
>
OK, nice.

>
> I reinitialized the install, as you suggested (except I just shut
> everything down, blew away the data_* folders in /usr/local/pgsql, and
> reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
> to have fixed the problem.  (I guess probably those data_* folders should
> be under /var somewhere..)
>
> I'm not sure what the problem was.  I think I recall having attempted to
> issue a sql command to create a coordinator node, the previous time around.
>  I notice your steps don't include that step?  I wonder if somehow that
> affected things?
>
When you initialized a node with initdb, --nodename is a mandatory option.
This option value is used to set up pgxc_node_name automatically and
initialize the catalog pgxc_node with the data of the node itself.
You need to create a Coordinator with CREATE NODE on a Coordinator only if
there is at least 2 Coordinators in your cluster.


> The reason I had set the port in datanode1's postgresql.conf was because I
> would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
> init.d script, (ala service postgresql-xc start / restart / stop) and I
> haven't been able to get pg_ctl to pass the port parameter to postgres.
> (pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
> usage docs).  After putting that setting back in again, I'm using gtm_ctl
> and pg_ctl to start and stop it all at this point, and it seems to working
> fine.
>
Nice to hear that it is working. I am sure you missed something at
initialization.
When you want to add a port number with pg_ctl, be sure to use quotes with
-o, like:
-o "-p $PORT"

The XC documentation is not perfect, if you could point to the page and the
paragraph where you had doubts I can try to have a look and correct the doc
if necessary.


> The pgbench utility wasn't installed, but I finally found it and installed
> it.  I notice that it somehow left a bunch of idle connection processes
> behind after exiting.  Would those get reused, or is that a problem with
> pgbench?  (I restarted postgres-xc and they went away, but.  Just
> wondering.)
>
The connection pooling of XC is made to maintain connections between nodes
as it consumes a bunch of ressources to create connections on the fly for
each transaction.
You can however drop those connections with CLEAN CONNECTION, command
specific to XC:
http://postgres-xc.sourceforge.net/docs/1_0/sql-cleanconnection.html
-- 
Michael Paquier
http://michael.otacoo.com

Did you do any cleanup for gtm resources?    Sorry I wrote a bit late.   The message suggests that coordinator/datade attempted to register itself to GTM which has alrady been registered.

If you have similar problem, please try to cleanup register.node file at gtm work directory.   This should not bring any problem if you stop gtm (and all other nodes) when you cleanup.

Regards;
---
Koichi Suzuki

On Tue, 18 Sep 2012 18:12:34 -0700
Shavais Zarathustra <sh...@gm...> wrote:

> Hello,
> 
> I've been using a computer on which gmail was already logged in.  I
> mistakenly assumed it was logged into my account.  Sorry about that.
> 
> The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
> data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
> respectively.
> 
> I reinitialized the install, as you suggested (except I just shut
> everything down, blew away the data_* folders in /usr/local/pgsql, and
> reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
> to have fixed the problem.  (I guess probably those data_* folders should
> be under /var somewhere..)
> 
> I'm not sure what the problem was.  I think I recall having attempted to
> issue a sql command to create a coordinator node, the previous time around.
>  I notice your steps don't include that step?  I wonder if somehow that
> affected things?
> 
> The reason I had set the port in datanode1's postgresql.conf was because I
> would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
> init.d script, (ala service postgresql-xc start / restart / stop) and I
> haven't been able to get pg_ctl to pass the port parameter to postgres.
> (pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
> usage docs).  After putting that setting back in again, I'm using gtm_ctl
> and pg_ctl to start and stop it all at this point, and it seems to working
> fine.
> 
> The pgbench utility wasn't installed, but I finally found it and installed
> it.  I notice that it somehow left a bunch of idle connection processes
> behind after exiting.  Would those get reused, or is that a problem with
> pgbench?  (I restarted postgres-xc and they went away, but.  Just
> wondering.)
> 
> 
> Shavais
> 
> On Tue, Sep 18, 2012 at 4:01 PM, Michael Paquier
> <mic...@gm...>wrote:
> 
> > Hi,
> >
> > You were not registered on the general mailing list, so I had to check the
> > content of your message manually.
> > This explains the delay of your message.
> >
> > By the way, regarding your message, please see comments below.
> >
> > On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:
> >
> >> I'm using CentOS 6.3, I've used the "short version" installation to set
> >> up a gtm, a single coordinator, and a single data node on my local host.
> >>  Here are the contents of my pgxc_node table:
> >>
> >> -bash-4.1$ psql -U postgres
> >> psql (PGXC 1.0.1, based on PG 9.1.5)
> >> Type "help" for help.
> >> postgres=# select * from pgxc_node;
> >>  node_name | node_type | node_port | node_host | nodeis_primary |
> >> nodeis_preferred |  node_id
> >>
> >> -----------+-----------+-----------+-----------+----------------+------------------+------------
> >>  coord1    | C         |      5432 | localhost | f              | f
> >>          | 1885696643
> >>  datanode1 | D         |     15432 | localhost | t              | t
> >>          |  888802358
> >>
> > I assume that you used the short install document from here:
> > http://postgres-xc.sourceforge.net/docs/1_0/install-short.html
> >
> > I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
> > datanode on the Coordinator.
> > This looks definitely correct.
> >
> >  The problem I'm having is that after maybe 60 to 90 seconds or so after
> >> starting, the coordinator dies with a log message like
> >>
> >> FATAL:  Can not register Coordinator on GTM
> >>
> > This basically means that the Coordinator is trying to register to GTM
> > with a node name that is already used by another node.
> > What is the value of pgxc_node_name in each node's postgresql.conf?
> >
> > The gtm is definitely running, I can see it in the process list.  The only
> >> changes to the default postgresql.conf's I've made is to specify the port
> >> of 15432 for datanode1, and (as part of an effort to fix this problem)
> >> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> >> postgresql.conf files.
> >>
> > For a really basic setting, changing the port number of datanode1 is
> > enough, as well as specifying the port value with -p at startup.
> > The value specified by option -p has higher priority than the value of
> > port in postgresql.conf.
> >
> >
> >> Another thing I did to try to fix this was:
> >>
> >> alter node datanode1 with (primary = true, preferred = true);
> >>
> > Primary nodes are the nodes used in priority for replicated writes.
> > Preferred nodes are the nodes used in priority for replicated reads. So
> > changing this setting won't solve anything in your case.
> >
> >
> >>
> >> As you can see, that has affected the pgxc_node contents.  But that
> >> didn't have any impact on this crashing behavior.
> >>
> >> I tried starting using the method presented in the short version install
> >> to start everything, and I also tried starting up like this:
> >>
> >> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> >> sleep 1
> >> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> >> sleep 1
> >> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> >>
> >> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> >> paths.)
> >>
> > Absolute and non-absolute paths both work, like postgres.
> >
> > In your case, I suggest to check for the values of pgxc_node_name in
> > postgresql.conf. Please be sure that they are set to the same values as
> > node_name in pgxc_node.
> > What you can also check is the data of the file called register.node in
> > the data folder of GTM. The data of this file is binary, but you may be
> > able to check the data folders of the nodes registered on GTM even by
> > opening the file manually.
> >
> > Please also try the following thing if problem persists. This simple set
> > of commands allows to start a cluster with 1 Coordinator and 1 Datanode:
> > mkdir $HOME/pgsql/data_coord1
> > mkdir $HOME/pgsql/data_datanode1
> > mkdir $HOME/pgsql/data_gtm
> > initdb -D $HOME/pgsql/data_coord1 --nodename coord1
> > initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
> > initgtm -D $HOME/pgsql/data_gtm -Z gtm
> > gtm -D $HOME/pgsql/data_gtm &
> > postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
> > postgres -C -D $HOME/pgsql/data_coord1 &
> > psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
> > postgres
> > psql -c "SELECT pgxc_pool_reload()" postgres
> > createdb test
> > psql test
> >
> > In my case, this works perfectly with XC 1.0.1, even with firewall and
> > SELinux in my box.
> >
> > Then why not testing pgbench?
> > pgbench -i test
> > pgbench -c 24 -S test
> >
> > Regards,
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> > ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
> >

Hi,

Maybe coordinator or datanode is trying to register itself doubly to gtm.

To solve this problem, please try the following:

1. Stop gtm, coordinator and datanode,
2. Visit gtm's work directory (you should have specified by -D option with gtm_ctl)
3. Rename the file register.node to something else
4. Restart everything.

BTW, I've posted bash script pgxclocal which takes care of XC installation, start and stop for local installation just for test.

This is available from https://github.com/koichi-szk/PGXC-Tools.git
Current version is attached.

This will help to configure XC in local environment.   The script consists of two coordinators and two datanodes.   You can rewrite this section as you like.   Also, this comes with  a set of functions which shows what you should do to run your XC cluster.

Good luck;
---
Koichi Suzuki
# I'm rewriting this into general Postgres-XC operating utility called pgxc_ctl.

On Tue, 18 Sep 2012 12:51:57 -0700
Roger Mayes <rog...@gm...> wrote:

> Hello,
> 
> I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
> 1.0.1 as the back end for a new project.  I would've searched prior posts
> for similar ones before posting, but I don't see any way to do that?
> 
> I'm using CentOS 6.3, I've used the "short version" installation to set up
> a gtm, a single coordinator, and a single data node on my local host.  Here
> are the contents of my pgxc_node table:
> 
> -bash-4.1$ psql -U postgres
> psql (PGXC 1.0.1, based on PG 9.1.5)
> Type "help" for help.
> postgres=# select * from pgxc_node;
>  node_name | node_type | node_port | node_host | nodeis_primary |
> nodeis_preferred |  node_id
> -----------+-----------+-----------+-----------+----------------+------------------+------------
>  coord1    | C         |      5432 | localhost | f              | f
>        | 1885696643
>  datanode1 | D         |     15432 | localhost | t              | t
>        |  888802358
> 
> 
> The problem I'm having is that after maybe 60 to 90 seconds or so after
> starting, the coordinator dies with a log message like
> 
> FATAL:  Can not register Coordinator on GTM
> 
> The gtm is definitely running, I can see it in the process list.  The only
> changes to the default postgresql.conf's I've made is to specify the port
> of 15432 for datanode1, and (as part of an effort to fix this problem)
> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> postgresql.conf files.
> 
> Another thing I did to try to fix this was:
> 
> alter node datanode1 with (primary = true, preferred = true);
> 
> As you can see, that has affected the pgxc_node contents.  But that didn't
> have any impact on this crashing behavior.
> 
> I tried starting using the method presented in the short version install to
> start everything, and I also tried starting up like this:
> 
> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> sleep 1
> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> sleep 1
> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
> 
> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> paths.)
> 
> I've tried using kill to clear everything out of the process table, I've
> tried making sure the last shutdown was clean for everything, using pg_ctl
> and gtm_ctl to shutdown, and then starting up fresh from having shutdown
> cleanly.
> 
> SELinux is disabled, the local firewall is disabled.
> 
> Everything works for about 60 to 90 seconds, then the coordinator dies with
> the message above.  Any help would be greatly appreciated,
> 
> Shavais

Hello,

I've been using a computer on which gmail was already logged in.  I
mistakenly assumed it was logged into my account.  Sorry about that.

The pgxc_node_name setting was set to 'coord1' and 'datanode1' in
data_coord1/postgresql.conf and data_datanode1/postgersql.conf,
respectively.

I reinitialized the install, as you suggested (except I just shut
everything down, blew away the data_* folders in /usr/local/pgsql, and
reused those instead of using $HOME/pgsql/data_*), and, so far, that seems
to have fixed the problem.  (I guess probably those data_* folders should
be under /var somewhere..)

I'm not sure what the problem was.  I think I recall having attempted to
issue a sql command to create a coordinator node, the previous time around.
 I notice your steps don't include that step?  I wonder if somehow that
affected things?

The reason I had set the port in datanode1's postgresql.conf was because I
would like to use gtm_ctl and pg_ctl to start up and shutdown, using an
init.d script, (ala service postgresql-xc start / restart / stop) and I
haven't been able to get pg_ctl to pass the port parameter to postgres.
(pg_ctl ... -o -p 15432 doesn't work; apparently I'm misinterpreting the
usage docs).  After putting that setting back in again, I'm using gtm_ctl
and pg_ctl to start and stop it all at this point, and it seems to working
fine.

The pgbench utility wasn't installed, but I finally found it and installed
it.  I notice that it somehow left a bunch of idle connection processes
behind after exiting.  Would those get reused, or is that a problem with
pgbench?  (I restarted postgres-xc and they went away, but.  Just
wondering.)


Shavais

On Tue, Sep 18, 2012 at 4:01 PM, Michael Paquier
<mic...@gm...>wrote:

> Hi,
>
> You were not registered on the general mailing list, so I had to check the
> content of your message manually.
> This explains the delay of your message.
>
> By the way, regarding your message, please see comments below.
>
> On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:
>
>> I'm using CentOS 6.3, I've used the "short version" installation to set
>> up a gtm, a single coordinator, and a single data node on my local host.
>>  Here are the contents of my pgxc_node table:
>>
>> -bash-4.1$ psql -U postgres
>> psql (PGXC 1.0.1, based on PG 9.1.5)
>> Type "help" for help.
>> postgres=# select * from pgxc_node;
>>  node_name | node_type | node_port | node_host | nodeis_primary |
>> nodeis_preferred |  node_id
>>
>> -----------+-----------+-----------+-----------+----------------+------------------+------------
>>  coord1    | C         |      5432 | localhost | f              | f
>>          | 1885696643
>>  datanode1 | D         |     15432 | localhost | t              | t
>>          |  888802358
>>
> I assume that you used the short install document from here:
> http://postgres-xc.sourceforge.net/docs/1_0/install-short.html
>
> I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
> datanode on the Coordinator.
> This looks definitely correct.
>
>  The problem I'm having is that after maybe 60 to 90 seconds or so after
>> starting, the coordinator dies with a log message like
>>
>> FATAL:  Can not register Coordinator on GTM
>>
> This basically means that the Coordinator is trying to register to GTM
> with a node name that is already used by another node.
> What is the value of pgxc_node_name in each node's postgresql.conf?
>
> The gtm is definitely running, I can see it in the process list.  The only
>> changes to the default postgresql.conf's I've made is to specify the port
>> of 15432 for datanode1, and (as part of an effort to fix this problem)
>> uncomment the host ('localhost') and port (6666) of the gtm in all 3
>> postgresql.conf files.
>>
> For a really basic setting, changing the port number of datanode1 is
> enough, as well as specifying the port value with -p at startup.
> The value specified by option -p has higher priority than the value of
> port in postgresql.conf.
>
>
>> Another thing I did to try to fix this was:
>>
>> alter node datanode1 with (primary = true, preferred = true);
>>
> Primary nodes are the nodes used in priority for replicated writes.
> Preferred nodes are the nodes used in priority for replicated reads. So
> changing this setting won't solve anything in your case.
>
>
>>
>> As you can see, that has affected the pgxc_node contents.  But that
>> didn't have any impact on this crashing behavior.
>>
>> I tried starting using the method presented in the short version install
>> to start everything, and I also tried starting up like this:
>>
>> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
>> sleep 1
>> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
>> sleep 1
>> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
>>
>> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
>> paths.)
>>
> Absolute and non-absolute paths both work, like postgres.
>
> In your case, I suggest to check for the values of pgxc_node_name in
> postgresql.conf. Please be sure that they are set to the same values as
> node_name in pgxc_node.
> What you can also check is the data of the file called register.node in
> the data folder of GTM. The data of this file is binary, but you may be
> able to check the data folders of the nodes registered on GTM even by
> opening the file manually.
>
> Please also try the following thing if problem persists. This simple set
> of commands allows to start a cluster with 1 Coordinator and 1 Datanode:
> mkdir $HOME/pgsql/data_coord1
> mkdir $HOME/pgsql/data_datanode1
> mkdir $HOME/pgsql/data_gtm
> initdb -D $HOME/pgsql/data_coord1 --nodename coord1
> initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
> initgtm -D $HOME/pgsql/data_gtm -Z gtm
> gtm -D $HOME/pgsql/data_gtm &
> postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
> postgres -C -D $HOME/pgsql/data_coord1 &
> psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
> postgres
> psql -c "SELECT pgxc_pool_reload()" postgres
> createdb test
> psql test
>
> In my case, this works perfectly with XC 1.0.1, even with firewall and
> SELinux in my box.
>
> Then why not testing pgbench?
> pgbench -i test
> pgbench -c 24 -S test
>
> Regards,
> --
> Michael Paquier
> http://michael.otacoo.com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

Hi all,

Just to let you know about the existence of a thread about Postgres-XC on
reddit, as a result of the presentation(s) at PGOpen in Chicago.
http://www.reddit.com/r/programming/comments/102qtr/postgresxc_a_writescalable_synchronous/
-- 
Michael Paquier
http://michael.otacoo.com

Hi,

You were not registered on the general mailing list, so I had to check the
content of your message manually.
This explains the delay of your message.

By the way, regarding your message, please see comments below.

On Wed, Sep 19, 2012 at 4:51 AM, Roger Mayes <rog...@gm...>wrote:

> I'm using CentOS 6.3, I've used the "short version" installation to set up
> a gtm, a single coordinator, and a single data node on my local host.  Here
> are the contents of my pgxc_node table:
>
> -bash-4.1$ psql -U postgres
> psql (PGXC 1.0.1, based on PG 9.1.5)
> Type "help" for help.
> postgres=# select * from pgxc_node;
>  node_name | node_type | node_port | node_host | nodeis_primary |
> nodeis_preferred |  node_id
>
> -----------+-----------+-----------+-----------+----------------+------------------+------------
>  coord1    | C         |      5432 | localhost | f              | f
>          | 1885696643
>  datanode1 | D         |     15432 | localhost | t              | t
>          |  888802358
>
I assume that you used the short install document from here:
http://postgres-xc.sourceforge.net/docs/1_0/install-short.html

I also imagine that you ran "SELECT pgxc_pool_reload();" after setting the
datanode on the Coordinator.
This looks definitely correct.

The problem I'm having is that after maybe 60 to 90 seconds or so after
> starting, the coordinator dies with a log message like
>
> FATAL:  Can not register Coordinator on GTM
>
This basically means that the Coordinator is trying to register to GTM with
a node name that is already used by another node.
What is the value of pgxc_node_name in each node's postgresql.conf?

The gtm is definitely running, I can see it in the process list.  The only
> changes to the default postgresql.conf's I've made is to specify the port
> of 15432 for datanode1, and (as part of an effort to fix this problem)
> uncomment the host ('localhost') and port (6666) of the gtm in all 3
> postgresql.conf files.
>
For a really basic setting, changing the port number of datanode1 is
enough, as well as specifying the port value with -p at startup.
The value specified by option -p has higher priority than the value of port
in postgresql.conf.


> Another thing I did to try to fix this was:
>
> alter node datanode1 with (primary = true, preferred = true);
>
Primary nodes are the nodes used in priority for replicated writes.
Preferred nodes are the nodes used in priority for replicated reads. So
changing this setting won't solve anything in your case.


>
> As you can see, that has affected the pgxc_node contents.  But that didn't
> have any impact on this crashing behavior.
>
> I tried starting using the method presented in the short version install
> to start everything, and I also tried starting up like this:
>
> gtm_ctl -Z gtm -D data_gtm -l gtm.log start
> sleep 1
> pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
> sleep 1
> pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start
>
> ..from the /usr/local/pgsql directory.  (I also tried specifying absolute
> paths.)
>
Absolute and non-absolute paths both work, like postgres.

In your case, I suggest to check for the values of pgxc_node_name in
postgresql.conf. Please be sure that they are set to the same values as
node_name in pgxc_node.
What you can also check is the data of the file called register.node in the
data folder of GTM. The data of this file is binary, but you may be able to
check the data folders of the nodes registered on GTM even by opening the
file manually.

Please also try the following thing if problem persists. This simple set of
commands allows to start a cluster with 1 Coordinator and 1 Datanode:
mkdir $HOME/pgsql/data_coord1
mkdir $HOME/pgsql/data_datanode1
mkdir $HOME/pgsql/data_gtm
initdb -D $HOME/pgsql/data_coord1 --nodename coord1
initdb -D $HOME/pgsql/data_datanode1 --nodename datanode1
initgtm -D $HOME/pgsql/data_gtm -Z gtm
gtm -D $HOME/pgsql/data_gtm &
postgres -X -p 15432 -D $HOME/pgsql/data_datanode1 &
postgres -C -D $HOME/pgsql/data_coord1 &
psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)"
postgres
psql -c "SELECT pgxc_pool_reload()" postgres
createdb test
psql test

In my case, this works perfectly with XC 1.0.1, even with firewall and
SELinux in my box.

Then why not testing pgbench?
pgbench -i test
pgbench -c 24 -S test

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Hi,

Just a notice for people sending emails to postgres-XC mailing lists...
You need to register first to the mailing lists before sending a message to
it.
If you are not registered, ML administrators need to authorize manually the
content of the message.
This is particularly useful to prevent the pollution of spams here. It also
takes time to approve the messages.

You can register to the following mailing lists from the following links:
- pos...@li...,
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
- pos...@li...,
https://lists.sourceforge.net/lists/listinfo/postgres-xc-bugs
- pos...@li...,
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Thanks in advance,
-- 
Michael Paquier
http://michael.otacoo.com

Hello,

I'm new to pg-xc, I'm just working on getting off the ground using pg-xc
1.0.1 as the back end for a new project.  I would've searched prior posts
for similar ones before posting, but I don't see any way to do that?

I'm using CentOS 6.3, I've used the "short version" installation to set up
a gtm, a single coordinator, and a single data node on my local host.  Here
are the contents of my pgxc_node table:

-bash-4.1$ psql -U postgres
psql (PGXC 1.0.1, based on PG 9.1.5)
Type "help" for help.
postgres=# select * from pgxc_node;
 node_name | node_type | node_port | node_host | nodeis_primary |
nodeis_preferred |  node_id
-----------+-----------+-----------+-----------+----------------+------------------+------------
 coord1    | C         |      5432 | localhost | f              | f
       | 1885696643
 datanode1 | D         |     15432 | localhost | t              | t
       |  888802358


The problem I'm having is that after maybe 60 to 90 seconds or so after
starting, the coordinator dies with a log message like

FATAL:  Can not register Coordinator on GTM

The gtm is definitely running, I can see it in the process list.  The only
changes to the default postgresql.conf's I've made is to specify the port
of 15432 for datanode1, and (as part of an effort to fix this problem)
uncomment the host ('localhost') and port (6666) of the gtm in all 3
postgresql.conf files.

Another thing I did to try to fix this was:

alter node datanode1 with (primary = true, preferred = true);

As you can see, that has affected the pgxc_node contents.  But that didn't
have any impact on this crashing behavior.

I tried starting using the method presented in the short version install to
start everything, and I also tried starting up like this:

gtm_ctl -Z gtm -D data_gtm -l gtm.log start
sleep 1
pg_ctl -Z coordinator -D data_coord1 -l coord1.log start
sleep 1
pg_ctl -Z datanode -D data_datanode1 -l datanode1.log start

..from the /usr/local/pgsql directory.  (I also tried specifying absolute
paths.)

I've tried using kill to clear everything out of the process table, I've
tried making sure the last shutdown was clean for everything, using pg_ctl
and gtm_ctl to shutdown, and then starting up fresh from having shutdown
cleanly.

SELinux is disabled, the local firewall is disabled.

Everything works for about 60 to 90 seconds, then the coordinator dies with
the message above.  Any help would be greatly appreciated,

Shavais

On Tue, Sep 18, 2012 at 5:11 PM, Paulo Pires <pj...@ub...> wrote:

>
> On 18/09/12 00:43, Michael Paquier wrote:
>
>
>
> On Tue, Sep 18, 2012 at 7:40 AM, Ashutosh Bapat <
> ash...@en...> wrote:
>
>>
>>
>>  On Tue, Sep 18, 2012 at 2:05 AM, Paulo Pires <pj...@ub...>wrote:
>>
>>> OK, I've tried this with plain PostgreSQL and it works. I've executed
>>> the query by hand and the ID of the user returns properly.
>>> In postgres-xc the RETURNING clause is not yet supported.
>>>
>>> Any idea on when could this get to postgres-xc or at least could anyone
>>> please explain why it has not been yet included (any technical issues?)?
>>>
>>>
>> The RETURNING clause support will be available in the next release of XC.
>> We are working on it.
>>
> The patch for support of RETURNING will be committed in something like 2
> weeks on the current development branch, so you may also consider  testing
> your application not on a stable release but on Postgres-XC master branch.
>
> Count me in for testing. Btw, when is the new release expected to happen?
> Is it still May 2013?
>
Yes, the next stable release is planned for this period. Might be a bit
earlier or a bit later.
A beta phase will be done before that though.
-- 
Michael Paquier
http://michael.otacoo.com

On 18/09/12 00:43, Michael Paquier wrote:
>
>
> On Tue, Sep 18, 2012 at 7:40 AM, Ashutosh Bapat
> <ash...@en...
> <mailto:ash...@en...>> wrote:
>
>
>
>     On Tue, Sep 18, 2012 at 2:05 AM, Paulo Pires <pj...@ub...
>     <mailto:pj...@ub...>> wrote:
>
>         OK, I've tried this with plain PostgreSQL and it works. I've
>         executed
>         the query by hand and the ID of the user returns properly.
>         In postgres-xc the RETURNING clause is not yet supported.
>
>         Any idea on when could this get to postgres-xc or at least
>         could anyone
>         please explain why it has not been yet included (any technical
>         issues?)?
>
>
>     The RETURNING clause support will be available in the next release
>     of XC. We are working on it.
>
> The patch for support of RETURNING will be committed in something like
> 2 weeks on the current development branch, so you may also consider 
> testing your application not on a stable release but on Postgres-XC
> master branch.
Count me in for testing. Btw, when is the new release expected to
happen? Is it still May 2013?

> -- 
> Michael Paquier
> http://michael.otacoo.com

-- 
Paulo Pires

On Tue, Sep 18, 2012 at 7:40 AM, Ashutosh Bapat <
ash...@en...> wrote:

>
>
> On Tue, Sep 18, 2012 at 2:05 AM, Paulo Pires <pj...@ub...> wrote:
>
>> OK, I've tried this with plain PostgreSQL and it works. I've executed
>> the query by hand and the ID of the user returns properly.
>> In postgres-xc the RETURNING clause is not yet supported.
>>
>> Any idea on when could this get to postgres-xc or at least could anyone
>> please explain why it has not been yet included (any technical issues?)?
>>
>>
> The RETURNING clause support will be available in the next release of XC.
> We are working on it.
>
The patch for support of RETURNING will be committed in something like 2
weeks on the current development branch, so you may also consider  testing
your application not on a stable release but on Postgres-XC master branch.
-- 
Michael Paquier
http://michael.otacoo.com

On Tue, Sep 18, 2012 at 2:05 AM, Paulo Pires <pj...@ub...> wrote:

> OK, I've tried this with plain PostgreSQL and it works. I've executed
> the query by hand and the ID of the user returns properly.
> In postgres-xc the RETURNING clause is not yet supported.
>
> Any idea on when could this get to postgres-xc or at least could anyone
> please explain why it has not been yet included (any technical issues?)?
>
>
The RETURNING clause support will be available in the next release of XC.
We are working on it.


> Cheers,
> PP
>
> On 9/17/12 10:47 PM, Paulo Pires wrote:
> > Dear all,
> >
> > I'm testing postgres-xc with an existing Java EE solution of mine and
> > right now I'm facing the following issue when reading from an entity
> > I've just persisted:
> >
> > org.postgresql.util.PSQLException: ERROR: RETURNING clause not yet
> supported
> >
> > Here's what I'm doing:
> > 1) I have a JDBC Connection Pool configured in my application server
> > pointing to one of my coordinators;
> > 2) In my application, I'm using Hibernate (Java ORM implementation) to
> > persist my dat-model;
> > 3) I insert a new entity to my database;
> > 4) I get the ID of the object, that usually is available after inserting
> > the new entity (I don't force RETURNING clause, but I believe the ORM
> > may be doing it or something similar, if any);
> > 5) I get the aforementioned exception.
> >
> > Is this a known issue?
> >
> > Cheers,
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >
> >
> >
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>
> --
> Paulo Pires
> Ubiwhere
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

OK, I've tried this with plain PostgreSQL and it works. I've executed
the query by hand and the ID of the user returns properly.
In postgres-xc the RETURNING clause is not yet supported.

Any idea on when could this get to postgres-xc or at least could anyone
please explain why it has not been yet included (any technical issues?)?

Cheers,
PP

On 9/17/12 10:47 PM, Paulo Pires wrote:
> Dear all,
> 
> I'm testing postgres-xc with an existing Java EE solution of mine and
> right now I'm facing the following issue when reading from an entity
> I've just persisted:
> 
> org.postgresql.util.PSQLException: ERROR: RETURNING clause not yet supported
> 
> Here's what I'm doing:
> 1) I have a JDBC Connection Pool configured in my application server
> pointing to one of my coordinators;
> 2) In my application, I'm using Hibernate (Java ORM implementation) to
> persist my dat-model;
> 3) I insert a new entity to my database;
> 4) I get the ID of the object, that usually is available after inserting
> the new entity (I don't force RETURNING clause, but I believe the ORM
> may be doing it or something similar, if any);
> 5) I get the aforementioned exception.
> 
> Is this a known issue?
> 
> Cheers,
> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> 
> 
> 
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> 

-- 
Paulo Pires
Ubiwhere

Dear all,

I'm testing postgres-xc with an existing Java EE solution of mine and
right now I'm facing the following issue when reading from an entity
I've just persisted:

org.postgresql.util.PSQLException: ERROR: RETURNING clause not yet supported

Here's what I'm doing:
1) I have a JDBC Connection Pool configured in my application server
pointing to one of my coordinators;
2) In my application, I'm using Hibernate (Java ORM implementation) to
persist my dat-model;
3) I insert a new entity to my database;
4) I get the ID of the object, that usually is available after inserting
the new entity (I don't force RETURNING clause, but I believe the ORM
may be doing it or something similar, if any);
5) I get the aforementioned exception.

Is this a known issue?

Cheers,

-- 
Paulo Pires
Ubiwhere

If you are in the New York City area and want to learn more about
Postgres-XC, I will be giving a talk this Thursday, Sept. 20th at 6:30
pm. More details can be found here:

http://www.unigroup.org/unigroup-next.html

Hope to see you there!


-- 
Mason Sharp

StormDB - http://www.stormdb.com
The Database Cloud
Postgres-XC Support and Services

Year DNS round robin may work, although connection point will be fixed
once cached.

Regards;
----------
Koichi Suzuki


2012/9/12 Mason Sharp <ma...@st...>:
> On Tue, Sep 11, 2012 at 1:52 PM, Ashutosh Bapat
> <ash...@en...> wrote:
>> Hi Paulo,
>> Although Postgres-XC provides facility to run many coordinators and thus
>> provide multiple access points, it does not have load balancer in built. You
>> can use third party load balancers with multiple coordinators.
>
> In addition to load balancers like AVS and HAProxy, you could also
> consider using round-robin DNS with a short TTL value.
>
> If you do try out AVS or HAProxy, I would love to hear about your
> experience with them.
>
>>
>>
>> On Tue, Sep 11, 2012 at 8:13 PM, Paulo Pires <pj...@ub...> wrote:
>>>
>>> Hi all,
>>>
>>> First of all I'd like to congratulate postgres-xc team for the amazing
>>> effort into getting version 1.0 released. Thanks guys!
>>>
>>> Now, I'd like to know if there's something like GTM-Proxy, but for
>>> Coordinators, of course.
>>> The thing is that I'm using Java JPA for accessing my database, and even
>>> that I can configure one persistence-unit per database I need to access,
>>> I'd have to implement something on my side that would select which
>>> Coordinator to connect to. That would be counterproductive!
>>>
>>> If postgres is or were to have a proxy facility that would select
>>> Coordinator based on its availability and load, it would be awesome.
>>>
>>> Can anyone clarify me?
>>>
>>> Cheers,
>>>
>>> --
>>> Paulo Pires
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>
>
> --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
> Postgres-XC Support and Services
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

On 11/09/12 19:04, Mason Sharp wrote:
> On Tue, Sep 11, 2012 at 1:52 PM, Ashutosh Bapat
> <ash...@en...> wrote:
>> Hi Paulo,
>> Although Postgres-XC provides facility to run many coordinators and thus
>> provide multiple access points, it does not have load balancer in built. You
>> can use third party load balancers with multiple coordinators.
> In addition to load balancers like AVS and HAProxy, you could also
> consider using round-robin DNS with a short TTL value.

I have hands-on experience with HAProxy and if I don't find better
tools, I'll give it a try.
Never heard about AVS. Could you please provide more info? Also, I'm not
fond of round-robin DNS.
>
> If you do try out AVS or HAProxy, I would love to hear about your
> experience with them.

If I do, I'll write about it and let you guys know.

>
>>
>> On Tue, Sep 11, 2012 at 8:13 PM, Paulo Pires <pj...@ub...> wrote:
>>> Hi all,
>>>
>>> First of all I'd like to congratulate postgres-xc team for the amazing
>>> effort into getting version 1.0 released. Thanks guys!
>>>
>>> Now, I'd like to know if there's something like GTM-Proxy, but for
>>> Coordinators, of course.
>>> The thing is that I'm using Java JPA for accessing my database, and even
>>> that I can configure one persistence-unit per database I need to access,
>>> I'd have to implement something on my side that would select which
>>> Coordinator to connect to. That would be counterproductive!
>>>
>>> If postgres is or were to have a proxy facility that would select
>>> Coordinator based on its availability and load, it would be awesome.
>>>
>>> Can anyone clarify me?
>>>
>>> Cheers,
>>>
>>> --
>>> Paulo Pires
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>

-- 
Paulo Pires

On 11/09/12 18:52, Ashutosh Bapat wrote:
> Hi Paulo,
> Although Postgres-XC provides facility to run many coordinators and
> thus provide multiple access points, it does not have load balancer in
> built. You can use third party load balancers with multiple coordinators.

Hi Ashutosh,

Could you please point me to a couple 3rd-party load-balancers that are
OSS and you've had experience with, please?

Thanks,
PP
>
> On Tue, Sep 11, 2012 at 8:13 PM, Paulo Pires <pj...@ub...
> <mailto:pj...@ub...>> wrote:
>
>     Hi all,
>
>     First of all I'd like to congratulate postgres-xc team for the amazing
>     effort into getting version 1.0 released. Thanks guys!
>
>     Now, I'd like to know if there's something like GTM-Proxy, but for
>     Coordinators, of course.
>     The thing is that I'm using Java JPA for accessing my database,
>     and even
>     that I can configure one persistence-unit per database I need to
>     access,
>     I'd have to implement something on my side that would select which
>     Coordinator to connect to. That would be counterproductive!
>
>     If postgres is or were to have a proxy facility that would select
>     Coordinator based on its availability and load, it would be awesome.
>
>     Can anyone clarify me?
>
>     Cheers,
>
>     --
>     Paulo Pires
>
>
>     ------------------------------------------------------------------------------
>     Live Security Virtual Conference
>     Exclusive live event will cover all the ways today's security and
>     threat landscape has changed and how IT managers can respond.
>     Discussions
>     will include endpoint security, mobile security and the latest in
>     malware
>     threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>     _______________________________________________
>     Postgres-xc-general mailing list
>     Pos...@li...
>     <mailto:Pos...@li...>
>     https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
>
> -- 
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>

-- 
Paulo Pires

On Tue, Sep 11, 2012 at 1:52 PM, Ashutosh Bapat
<ash...@en...> wrote:
> Hi Paulo,
> Although Postgres-XC provides facility to run many coordinators and thus
> provide multiple access points, it does not have load balancer in built. You
> can use third party load balancers with multiple coordinators.

In addition to load balancers like AVS and HAProxy, you could also
consider using round-robin DNS with a short TTL value.

If you do try out AVS or HAProxy, I would love to hear about your
experience with them.

>
>
> On Tue, Sep 11, 2012 at 8:13 PM, Paulo Pires <pj...@ub...> wrote:
>>
>> Hi all,
>>
>> First of all I'd like to congratulate postgres-xc team for the amazing
>> effort into getting version 1.0 released. Thanks guys!
>>
>> Now, I'd like to know if there's something like GTM-Proxy, but for
>> Coordinators, of course.
>> The thing is that I'm using Java JPA for accessing my database, and even
>> that I can configure one persistence-unit per database I need to access,
>> I'd have to implement something on my side that would select which
>> Coordinator to connect to. That would be counterproductive!
>>
>> If postgres is or were to have a proxy facility that would select
>> Coordinator based on its availability and load, it would be awesome.
>>
>> Can anyone clarify me?
>>
>> Cheers,
>>
>> --
>> Paulo Pires
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
Mason Sharp

StormDB - http://www.stormdb.com
The Database Cloud
Postgres-XC Support and Services

Hi Paulo,
Although Postgres-XC provides facility to run many coordinators and thus
provide multiple access points, it does not have load balancer in built.
You can use third party load balancers with multiple coordinators.

On Tue, Sep 11, 2012 at 8:13 PM, Paulo Pires <pj...@ub...> wrote:

> Hi all,
>
> First of all I'd like to congratulate postgres-xc team for the amazing
> effort into getting version 1.0 released. Thanks guys!
>
> Now, I'd like to know if there's something like GTM-Proxy, but for
> Coordinators, of course.
> The thing is that I'm using Java JPA for accessing my database, and even
> that I can configure one persistence-unit per database I need to access,
> I'd have to implement something on my side that would select which
> Coordinator to connect to. That would be counterproductive!
>
> If postgres is or were to have a proxy facility that would select
> Coordinator based on its availability and load, it would be awesome.
>
> Can anyone clarify me?
>
> Cheers,
>
> --
> Paulo Pires
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi all,

First of all I'd like to congratulate postgres-xc team for the amazing
effort into getting version 1.0 released. Thanks guys!

Now, I'd like to know if there's something like GTM-Proxy, but for
Coordinators, of course.
The thing is that I'm using Java JPA for accessing my database, and even
that I can configure one persistence-unit per database I need to access,
I'd have to implement something on my side that would select which
Coordinator to connect to. That would be counterproductive!

If postgres is or were to have a proxy facility that would select
Coordinator based on its availability and load, it would be awesome.

Can anyone clarify me?

Cheers,

-- 
Paulo Pires

On Thu, Sep 6, 2012 at 12:32 PM, Koichi Suzuki
<koi...@gm...> wrote:
> Design discussion has not been made extensively.  I just gave my idea
> to the developer personaly.   I'm planning to make more discussion
> before posting to this ML.    It is not just simple and will take a
> bit more (a week or so).

OK, thanks, I understand. In my previous email I was initially just
focusing on the node registration part, and not the rebalancing part.

>
> Regards;
> ----------
> Koichi Suzuki
>
>
> 2012/9/6 Mason Sharp <ma...@st...>:
>> On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote:
>>> Thanks for the question.
>>>
>>> With XC-1.0.x, we cannot add a node at runtime.   This feature is
>>> included in the current development and will be available with the
>>> next major release.
>>
>> I may have missed emails discussing the design.  Can you please
>> summarize the design?
>>
>> We could have GTM manage node membership. For example, a new candidate
>> data node registers itself as available to GTM (we may want some kind
>> of GTM authentication instead of merely connecting to the ip:port).
>> ADD NODE is executed on one single coordinator, which validates
>> against GTM, updates the status, adds it to the cluster and notifies
>> the other Coordinators. There would no longer be any need for the
>> manual DDL that is done now.
>>
>> This could be expanded in the future to help support multiple
>> read-only replicas for read intensive workloads, where they are
>> identified by distribution segment and registered in a similar
>> fashion. (Instead of just node id, we use segment id, where a node can
>> be associated with a segment, either as the primary or a read-only
>> slave.)
>>
>>>
>>> The safest way is to backup the whole database with pg_dump,
>>> reinitialize all the nodes including the additional ones, configure
>>> the cluster (CREATE NODE and ALTER NODE) and then restore the data
>>> with pg_restore.   You may need to create each table in advance so
>>> that it can be distributed/replicated among all the nodes.
>>>
>>> New feature of node addition will include:
>>> 1) Create new coordinators and datanodes on new servers at runtime,
>>> which copies all the necessary catalog such as role, database, etc.,
>>> from existing components,
>>> 2) Add them to the cluster,
>>> 3) Reballance tables with new components.
>>>
>>> It's very helpful if you submit your requirement for this feature.
>>>
>>> Regards;
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>> 2012/9/6 "Marco Güttler" <mar...@gm...>:
>>>> Hey all,
>>>>
>>>> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition?
>>>> If not so, what is an alternative way to add a coordinator and datanode?
>>>>
>>>> Thanks and regards
>>>> Marco
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Live Security Virtual Conference
>>>> Exclusive live event will cover all the ways today's security and
>>>> threat landscape has changed and how IT managers can respond. Discussions
>>>> will include endpoint security, mobile security and the latest in malware
>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>> _______________________________________________
>>>> Postgres-xc-general mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>>
>> --
>> Mason Sharp
>>
>> StormDB - http://www.stormdb.com
>> The Database Cloud



-- 
Mason Sharp

StormDB - http://www.stormdb.com
The Database Cloud

Design discussion has not been made extensively.  I just gave my idea
to the developer personaly.   I'm planning to make more discussion
before posting to this ML.    It is not just simple and will take a
bit more (a week or so).

Regards;
----------
Koichi Suzuki


2012/9/6 Mason Sharp <ma...@st...>:
> On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote:
>> Thanks for the question.
>>
>> With XC-1.0.x, we cannot add a node at runtime.   This feature is
>> included in the current development and will be available with the
>> next major release.
>
> I may have missed emails discussing the design.  Can you please
> summarize the design?
>
> We could have GTM manage node membership. For example, a new candidate
> data node registers itself as available to GTM (we may want some kind
> of GTM authentication instead of merely connecting to the ip:port).
> ADD NODE is executed on one single coordinator, which validates
> against GTM, updates the status, adds it to the cluster and notifies
> the other Coordinators. There would no longer be any need for the
> manual DDL that is done now.
>
> This could be expanded in the future to help support multiple
> read-only replicas for read intensive workloads, where they are
> identified by distribution segment and registered in a similar
> fashion. (Instead of just node id, we use segment id, where a node can
> be associated with a segment, either as the primary or a read-only
> slave.)
>
>>
>> The safest way is to backup the whole database with pg_dump,
>> reinitialize all the nodes including the additional ones, configure
>> the cluster (CREATE NODE and ALTER NODE) and then restore the data
>> with pg_restore.   You may need to create each table in advance so
>> that it can be distributed/replicated among all the nodes.
>>
>> New feature of node addition will include:
>> 1) Create new coordinators and datanodes on new servers at runtime,
>> which copies all the necessary catalog such as role, database, etc.,
>> from existing components,
>> 2) Add them to the cluster,
>> 3) Reballance tables with new components.
>>
>> It's very helpful if you submit your requirement for this feature.
>>
>> Regards;
>> ----------
>> Koichi Suzuki
>>
>>
>> 2012/9/6 "Marco Güttler" <mar...@gm...>:
>>> Hey all,
>>>
>>> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition?
>>> If not so, what is an alternative way to add a coordinator and datanode?
>>>
>>> Thanks and regards
>>> Marco
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
> --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud