postgres-xc-general Mailing List for Postgres-XC (Page 47)

Hi All,

I am trying to install recently build rpm of pgxc but facing dependency
issue.
and not find any rpm who provide this

perl(Win32) is needed by pgxc-1.0.1-0.noarch

  perl(ucs2utf.pl) is needed by pgxc-1.0.1-0.noarch


please help


-- 
ViVek Raghuwanshi
Mobile -+91-09595950504
Skype - vivek_raghuwanshi
IRC vivekraghuwanshi
http://vivekraghuwanshi.wordpress.com/
http://in.linkedin.com/in/vivekraghuwanshi



-- 
ViVek Raghuwanshi
Mobile -+91-09595950504
Skype - vivek_raghuwanshi
IRC vivekraghuwanshi
http://vivekraghuwanshi.wordpress.com/
http://in.linkedin.com/in/vivekraghuwanshi

I have not looked into it very in detail.   So this is just a guess.

XC's datanode cluster contains three more system catalogues,
pgxc_class, pgxc_node and pgxc_nodegroup and also contains many more
build-in functions which have to be hard-coded.

Even though other part of the database cluster has no serious
difference, the above could be very serious.

It looks a good idea to allow XC to start with only one node
configuration, and then add nodes as needed.

Other than that, you can use pg_dump and pg_restore to move all your
data from PostgreSQL to Postgres-XC.

Regards;
----------
Koichi Suzuki

2013/1/29 Filip Rembiałkowski <fil...@gm...>:
> Hi
>
> 1. Is it possible to add postgres-xc on top of existing PostgreSQL database
> cluster?
>
> I mean something like this:
>
>  - setup coord1 and gtm1
>  - shutdown existing postgres
>  - replace postgres binaries with pg-xc binaries
>  - bring up the postgres as dn1
>
> ( After I have this, I would like to add new data nodes and
> replicate/distribute existing data with postgres-xc )
>
> 2. (maybe simpler) question - is node cluster format compatible with
> postgres cluster format?
>
> 3. question - is this reversible - can shutdown existing data node, replace
> binaries with regular postgres and bring it up?
>
>
> Thanks,
> Filip
>
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

Hi

1. Is it possible to add postgres-xc on top of existing PostgreSQL database
cluster?

I mean something like this:

 - setup coord1 and gtm1
 - shutdown existing postgres
 - replace postgres binaries with pg-xc binaries
 - bring up the postgres as dn1

( After I have this, I would like to add new data nodes and
replicate/distribute existing data with postgres-xc )

2. (maybe simpler) question - is node cluster format compatible with
postgres cluster format?

3. question - is this reversible - can shutdown existing data node, replace
binaries with regular postgres and bring it up?


Thanks,
Filip

On Sat, Jan 26, 2013 at 3:07 AM, Steven Vaughan <
ste...@da...> wrote:

>
>   I have a 3 node PGXC 1.0.1 cluster going.  As part of an initial test, I
> issued a query like : "select * from veryLargeTable limit 1" at a
> coordinator's psql prompt.  The query, as shipped to the datanodes, was :
> "select literalColumnList from only veryLargeTable where true" {without any
> limit clause}.  Clearly, this isn't the best or fastest query to hand down
> to the datanodes.  In this specific trivial case, a single query to a
> single data node would have been optimal, but even just passing on the
> limit to all datanodes would be a truckload faster.  Sure, there are cases
> where the coordinator might not be able to easily determine an upper limit
> on how many rows it needs from each datanode to fulfill the initially
> requested limit, but many cases (like my trivial one) exist where it would
> be straightforward to determine in the coordinator's optimizer.
>
>
Is the veryLargeTable a distributed table or replicated table. For
distributed table, yes, 1.0 coordinator does not ship LIMIT clause. For
replicated table it should ship LIMIT clause.

But, as Michael said LIMIT clause is being shipped to datanodes in 1.1.


> -steve
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Thank you Mason for the reply.

I have pgxc_ctl tool at github.   Please try to get pgxc_ctl bash
script from https://github.com/koichi-szk/PGXC-Tools

In this script, you will find what you should do to configure XC
cluster, as well as slave of each component, what you should do to
run/start whole XC cluster.

2013/1/27 Mason Sharp <ma...@st...>:
>
>
> On Sat, Jan 26, 2013 at 3:29 PM, Gintautas Sulskus <gi...@gm...> wrote:
>>
>> Hello guys,
>>
>> I am trying to set up postgres xc cluster distributed on two separate
>> machines. To my understanding this is the correct approach:
>>
>> host1
>> GTM:6666
>> coord1:5432
>> node1:15432
>> node2:15433
>>
>> host2
>> coord2:5432 (identical to coord1 + GTM host1)
>> node3:15432 (identical to node1 + GTM host1)
>>
>> To ALL coord and nodeX nodes I have added relevant hosts to pg_hba.conf as
>> trusted and accept_connections='*' to postgresql.conf.
>>
>> Also i've added ALL coord and nodeX nodes to coordinator's pgxc_node
>> table.
>>
>> But whenever i try to create a new database i get "Failed to get pooled
>> connections". Is it something i have missed?
>
>
> Perhaps we could improve the error message. Did you execute the SQL command
> CREATE NODE, too?
>
>>
>>
>> Also, could you clarify:
>> 1. If one of these nodes crashes cluster will work uninterrupted?
>
>
> No, high availability has to be done outside of the core. You can set up
> data node replicas and use something like Corosync/Pacemaker, and use a GTM
> Standby.

Depends upon your transaction.   If a failed node is not involved in a
transaction, it can continue to run.   However, as Mason suggested,
you should configure slave of each component so that failed node can
be failed over by a slave.   Good thing is transactions not involved
by a failed not can continue to run.

XC HA feature is covered by the material:
http://postgresxc.wikia.com/wiki/File:Prague_Presentation_20121024.pdf

>> 2. is it possible to set up the system that cluster replicates data to a
>> newly added node  automatically?
>>
>
> Not in the current system, you will have to redistribute tables, but ALTER
> TABLE makes this easier.

Right.   It depends on data distribution policy of each cluster.   I
hope next major release will come with node addition/removal on the
fly.   Even in this case, it is not reasonable to determine all the
table distribution/replication should be extended to this new node.
DBA can determine which set of nodes each table should be
distributed/replicated.

Regards and Good Luck;
---
Koichi Suzuki

>
>
>>
>> Thank you very much!
>>
>> Best Regards,
>> Gintas
>>
>>
>> ------------------------------------------------------------------------------
>> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>> MVPs and experts. ON SALE this month only -- learn more at:
>> http://p.sf.net/sfu/learnnow-d2d
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>
>
> --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
> Postgres-XC Support and Services
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

On Sat, Jan 26, 2013 at 3:29 PM, Gintautas Sulskus <gi...@gm...> wrote:

> Hello guys,
>
> I am trying to set up postgres xc cluster distributed on two separate
> machines. To my understanding this is the correct approach:
>
> host1
> GTM:6666
> coord1:5432
> node1:15432
> node2:15433
>
> host2
> coord2:5432 (identical to coord1 + GTM host1)
> node3:15432 (identical to node1 + GTM host1)
>
> To ALL coord and nodeX nodes I have added relevant hosts to pg_hba.conf as
> trusted and accept_connections='*' to postgresql.conf.
>
> Also i've added ALL coord and nodeX nodes to coordinator's pgxc_node table.
>
> But whenever i try to create a new database i get "Failed to get pooled
> connections". Is it something i have missed?
>

Perhaps we could improve the error message. Did you execute the SQL command
CREATE NODE, too?


>
> Also, could you clarify:
> 1. If one of these nodes crashes cluster will work uninterrupted?
>

No, high availability has to be done outside of the core. You can set up
data node replicas and use something like Corosync/Pacemaker, and use a GTM
Standby.



> 2. is it possible to set up the system that cluster replicates data to a
> newly added node  automatically?
>
>
Not in the current system, you will have to redistribute tables, but ALTER
TABLE makes this easier.



> Thank you very much!
>
> Best Regards,
> Gintas
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Mason Sharp

StormDB - http://www.stormdb.com
The Database Cloud
Postgres-XC Support and Services

Hello guys,

I am trying to set up postgres xc cluster distributed on two separate
machines. To my understanding this is the correct approach:

host1
GTM:6666
coord1:5432
node1:15432
node2:15433

host2
coord2:5432 (identical to coord1 + GTM host1)
node3:15432 (identical to node1 + GTM host1)

To ALL coord and nodeX nodes I have added relevant hosts to pg_hba.conf as
trusted and accept_connections='*' to postgresql.conf.

Also i've added ALL coord and nodeX nodes to coordinator's pgxc_node table.

But whenever i try to create a new database i get "Failed to get pooled
connections". Is it something i have missed?


Also, could you clarify:
1. If one of these nodes crashes cluster will work uninterrupted?
2. is it possible to set up the system that cluster replicates data to a
newly added node  automatically?

Thank you very much!

Best Regards,
Gintas

On Sat, Jan 26, 2013 at 6:37 AM, Steven Vaughan <
ste...@da...> wrote:

>
>   I have a 3 node PGXC 1.0.1 cluster going.  As part of an initial test, I
> issued a query like : "select * from veryLargeTable limit 1" at a
> coordinator's psql prompt.  The query, as shipped to the datanodes, was :
> "select literalColumnList from only veryLargeTable where true" {without any
> limit clause}.  Clearly, this isn't the best or fastest query to hand down
> to the datanodes.  In this specific trivial case, a single query to a
> single data node would have been optimal, but even just passing on the
> limit to all datanodes would be a truckload faster.  Sure, there are cases
> where the coordinator might not be able to easily determine an upper limit
> on how many rows it needs from each datanode to fulfill the initially
> requested limit, but many cases (like my trivial one) exist where it would
> be straightforward to determine in the coordinator's optimizer.
>
Sadly, LIMIT clause is not pushed down to remote nodes in 1.0 releases :(
This has been fixed in current development branch however, so you should
wait for the next major release to come out, or test with the current
master branch.
-- 
Michael Paquier
http://michael.otacoo.com

  I have a 3 node PGXC 1.0.1 cluster going.  As part of an initial test, I
issued a query like : "select * from veryLargeTable limit 1" at a
coordinator's psql prompt.  The query, as shipped to the datanodes, was :
"select literalColumnList from only veryLargeTable where true" {without any
limit clause}.  Clearly, this isn't the best or fastest query to hand down
to the datanodes.  In this specific trivial case, a single query to a
single data node would have been optimal, but even just passing on the
limit to all datanodes would be a truckload faster.  Sure, there are cases
where the coordinator might not be able to easily determine an upper limit
on how many rows it needs from each datanode to fulfill the initially
requested limit, but many cases (like my trivial one) exist where it would
be straightforward to determine in the coordinator's optimizer.

-steve

On Fri, Jan 25, 2013 at 3:20 AM, Steven Vaughan <
ste...@da...> wrote:

>
>   Howdy, I was watching the GTM's log on a completely idle cluster (in
> fact it's just now being built - there are absolutely zero clients
> performing zero reads or writes) ... and I see a global txn id go by about
> every 30 seconds :
>
> For example :
>
> 1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Assigning new
> transaction ID = 10158
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
> 1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Sending transaction
> ids from 10158 to 10159
> LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
> 1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Assigning new
> transaction ID = 10159
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
> 1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Sending transaction
> ids from 10159 to 10160
> LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
> 1:139937565083392:2013-01-24 18:15:11.838 UTC -LOG:  Assigning new
> transaction ID = 10160
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
>
> ...is this a normal heartbeat / synchronization signal or is there
> something unexpected going on?  This cluster consists of precisely one GTM
> (on machine 1) and one GTM_Proxy, Coordinator, and Datanode (all on machine
> 2).
>
You have also automatic transactions in your cluster with autovacuum
workers.
So this is normal.
-- 
Michael Paquier
http://michael.otacoo.com

  Howdy, I was watching the GTM's log on a completely idle cluster (in fact
it's just now being built - there are absolutely zero clients performing
zero reads or writes) ... and I see a global txn id go by about every 30
seconds :

For example :

1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Assigning new
transaction ID = 10158
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Sending transaction
ids from 10158 to 10159
LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Assigning new
transaction ID = 10159
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Sending transaction
ids from 10159 to 10160
LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
1:139937565083392:2013-01-24 18:15:11.838 UTC -LOG:  Assigning new
transaction ID = 10160
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581

...is this a normal heartbeat / synchronization signal or is there
something unexpected going on?  This cluster consists of precisely one GTM
(on machine 1) and one GTM_Proxy, Coordinator, and Datanode (all on machine
2).

Thanks in advance,
-steve

I'm afraid initgtm failed.  Please try to cleanup everything in gtm
directory, run initgtm, check it is successful, edit gtm.conf and
start gtm.

To avoid errors, I'm doing as follows:

$ rm -f $gtmdir
$ mkdir -p $gtmdir
$ initgtm -Z gtm -D $gtmdir
$ (... edit gtm.conf ...)
$ (... start gtm ...)

Regards;
----------
Koichi Suzuki


2013/1/23 Navin C <nav...@gm...>:
>
>
> I tried starting the GTM again :
>>
>> [postgres@centos6 data_gtm]$  /usr/local/pgsql/bin/initgtm -Z gtm -D
>> /usr/local/
>> pgsql/data_gtm
>> The files belonging to this GTM system will be owned by user "postgres".
>> This user must also own the server process.
>>
>> initgtm: directory "/usr/local/pgsql/data_gtm" exists but is not empty
>> If you want to create a new GTM system, either remove or empty
>> the directory "/usr/local/pgsql/data_gtm" or run initgtm
>> with an argument other than "/usr/local/pgsql/data_gtm".
>> [postgres@centos6 data_gtm]$
>
>
> This is what the gtm.log file says:
>
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Starting GTM server
>> at (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
>> LOCATION:  main, main.c:552
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Restoring last GXID
>> to 10000
>> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Started to run as
>> GTM-Active.
>> LOCATION:  main, main.c:624
>> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  GTM shutting down.
>> LOCATION:  ServerLoop, main.c:785
>> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  Saving transaction
>> info - next_gxid: 10000
>> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
>> Received signal 2
>> 1:140344341612288:2013-01-23 16:01:18.754 EST -LOG:  Starting GTM server
>> at (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
>> LOCATION:  main, main.c:552
>> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Restoring last GXID
>> to 10000
>> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
>> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Started to run as
>> GTM-Active.
>> LOCATION:  main, main.c:624
>> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  GTM shutting down.
>> LOCATION:  ServerLoop, main.c:785
>> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  Saving transaction
>> info - next_gxid: 10000
>> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
>> Received signal 2
>
> ~
>
> Do we need to set a passwordless ssh before installing postgres-xc.
> This is what I have done till now:
> 1. Configure,  the postgres-xc on all the three servers
> 2. added superuser postgres and gave permission to user postgres on the
> /data_xx directories on all the servers
> 3. Started the postgres-xc instance on every server and components using
> intidb
> 4. [postgres@centos6 bin]$ /usr/local/pgsql/bin/postgres -X -p 15432 -D
> /usr/local/pgsql/data_Datanode1
> LOG:  database system was shut down at 2013-01-22 18:14:22 EST
> LOG:  autovacuum launcher started
> LOG:  database system is ready to accept connections
>
>
>
>
> On Wed, Jan 23, 2013 at 10:10 AM, Ashutosh Bapat
> <ash...@en...> wrote:
>>
>> On Wed, Jan 23, 2013 at 9:58 AM, Navin C <nav...@gm...>
>> wrote:
>> > Hello All,
>> >
>> > First of all, I am really happy to see PostgreSQL come up with a Cluster
>> > based MPP solution.
>> > I am excited to use Postgres-xc on my cluster.
>> > I am facing some issues with the installation of Postgres-xc
>> >
>> > I am using Centos 6.3.
>> > First of all I wanted to know whether we need to setup passwordless ssh
>> > on
>> > all the servers of the cluster before starting to install Postgres-xc.
>> >
>> > I followed the steps in the documentation section Short version of
>> > installation
>> > ./configure
>> > gmake
>> > su
>> > gmake install
>> > adduser postgres
>> > mkdir /usr/local/pgsql/data_coord1
>> > mkdir /usr/local/pgsql/data_Datanode1
>> > mkdir /usr/local/pgsql/data_Datanode2
>> > mkdir /usr/local/pgsql/data_gtm
>> > chown postgres /usr/local/pgsql/data_coord1
>> > chown postgres /usr/local/pgsql/data_Datanode1
>> > chown postgres /usr/local/pgsql/data_Datanode2
>> > chown postgres /usr/local/pgsql/data_gtm
>> > su - postgres
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename
>> > coord1
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode1
>> > --nodename
>> > Datanode1
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode2
>> > --nodename
>> > Datanode2
>> > /usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm
>> > /usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 &
>> > /usr/local/pgsql/bin/postgres -X -p 15432 -D
>> > /usr/local/pgsql/data_Datanode1
>> >>logfile 2>
>> > /usr/local/pgsql/bin/postgres -X -p 15433 -D
>> > /usr/local/pgsql/data_Datanode2
>> >>logfile 2>
>> > /usr/local/pgsql/bin/postgres -C -D /usr/local/pgsql/data_coord1
>> > >logfile
>> > 2>&1 &
>> > /usr/local/pgsql/bin/psql -c "CREATE NODE Datanode1 WITH (TYPE =
>> > ’Datanode’,
>> > PORT = 15432)"
>> >
>> > When I try the last command as above It says the gtm is not connected,
>> > Xid
>> > is invalid.
>> > I also couldn't fine and gtm.pid file in data_gtm folder
>> >
>>
>> It looks like the GTM didn't start. Can you please check in the log
>> files what was the reason.
>>
>> > Can someone please help me with this. I need more detailed steps on each
>> > Co-ordinator, Datanode and gtm
>> > I have 3 servers 1 for GTM and 2 for Datanodes and Co-ordinators each.
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>> > MVPs and experts. ON SALE this month only -- learn more at:
>> > http://p.sf.net/sfu/learnnow-d2d
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>> >
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>
>
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

On Wed, 23 Jan 2013 07:28:27 -0600
Steven Vaughan <ste...@da...> wrote:

>   Thanks, pgxc_ctl is interesting - it's like a Swiss Army chainsaw!  The
> deploy/bootstrap (via ssh) feature is a neat addition (though it would be
> nice to be able to configure keys per host in the pgxc_ctl config file
> rather than relying on the ssh config to make 'ssh user@host' work without
> arguments). 

To deliver such key to the remote node, users have to type their password at least once.
I have simple bash script to help this for my personal use.   You have to type your password
just for a couple of times. 

Yes, I can add this to pgxc_ctl but I think this kind of capability should be taken care of by
system adminitration, not by XC.

---%<---- set_autologin ----------------%<---------------
#!/bin/bash
#
# set_autologin username host
#
function usage
{
        echo "Usage --"
        echo "   set_autologin username host"
}

if [ $# -lt 2 ]
then
        usage
        exit 1
fi

if [ $1 == '--help' ]
then
        usage
        exit 0
fi

cat > /tmp/$USER << EOF
#!/bin/bash
cd 
if [ -d .ssh ]; then 
        cd .
else
        mkdir .ssh
        chmod 0700 .ssh
fi
cd .ssh
touch authorized_keys2
chmod 0600 authorized_keys2
cat ~/id_dsa.pub >> authorized_keys2
rm ~/id_dsa.pub
EOF
chmod +x /tmp/$USER
cd
if [ -d .ssh ];
then
        cd .
else
        mkdir .ssh
        chmod 0700 .ssh
fi
if [ -f .ssh/id_dsa.pub ];
then
        cd .
else
        ssh-keygen -t dsa << EOF




EOF
fi
scp /tmp/$USER $1@$2:
scp .ssh/id_dsa.pub $1@$2:
ssh $1@$2 /home/$1/$USER
ssh $1@$2 rm /home/$1/$USER
rm /tmp/$USER
--->%--------------------------------------->%---------------

 One nice side effect is that it intrinsically produces the
> same binary version across the cluster (since it uses the same source
> tarball)  The bash script also answers my next question which would have
> been whether or not each coordinator needs to be sent the same CREATE NODE
> statements (clearly they do).  Are there any plans to make
> pgxc_node a replicated table at the coordinator level or add flags/syntax
> to CREATE NODE to synchronize mutations across the cluster?

This is interesting but have some fundamental issues.   To propagate CREATE NODE to all the nodes, we should know where to propagate to, which is defined by another CREATE NODE.    So far, it's safer to maintain current behaviour.

Regards;
---
Koichi Suzuki

> 
> Cheers,
> -steve
> 
> 
> On Tue, Jan 22, 2013 at 6:30 PM, Koichi Suzuki <koi...@gm...>wrote:
> 
> > Please visit https://github.com/koichi-szk/PGXC-Tools/tree/master/pgxc_ctl
> >
> > You will find pgxc_ctl and manual.txt.   (Sorry files in this page are
> > still WIP).
> >
> > Pgxc_ctl helps operation/houskeeping/configuration of XC cluster.
> > It's bash script so you can find what you should do manually.
> > Manual.txt describes the usage of the script.
> >
> > Good luck.
> > ----------
> > Koichi Suzuki
> >
> >
> > 2013/1/23 Steven Vaughan <ste...@da...>:
> > >
> > >   Howdy all, I was wondering if all node entries (manually managed by
> > CREATE
> > > NODE... ) were manually created or if there was some automation in the
> > > background that kept the pgxc metadata table up to date?  (For instance,
> > > when a datanode connected to the gtm?)
> > >
> > > Thanks in advance,
> > > -steve
> > >
> > >
> > ------------------------------------------------------------------------------
> > > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > > MVPs and experts. ON SALE this month only -- learn more at:
> > > http://p.sf.net/sfu/learnnow-d2d
> > > _______________________________________________
> > > Postgres-xc-general mailing list
> > > Pos...@li...
> > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> > >
> >

  Thanks, pgxc_ctl is interesting - it's like a Swiss Army chainsaw!  The
deploy/bootstrap (via ssh) feature is a neat addition (though it would be
nice to be able to configure keys per host in the pgxc_ctl config file
rather than relying on the ssh config to make 'ssh user@host' work without
arguments).  One nice side effect is that it intrinsically produces the
same binary version across the cluster (since it uses the same source
tarball)  The bash script also answers my next question which would have
been whether or not each coordinator needs to be sent the same CREATE NODE
statements (clearly they do).  Are there any plans to make
pgxc_node a replicated table at the coordinator level or add flags/syntax
to CREATE NODE to synchronize mutations across the cluster?

Cheers,
-steve


On Tue, Jan 22, 2013 at 6:30 PM, Koichi Suzuki <koi...@gm...>wrote:

> Please visit https://github.com/koichi-szk/PGXC-Tools/tree/master/pgxc_ctl
>
> You will find pgxc_ctl and manual.txt.   (Sorry files in this page are
> still WIP).
>
> Pgxc_ctl helps operation/houskeeping/configuration of XC cluster.
> It's bash script so you can find what you should do manually.
> Manual.txt describes the usage of the script.
>
> Good luck.
> ----------
> Koichi Suzuki
>
>
> 2013/1/23 Steven Vaughan <ste...@da...>:
> >
> >   Howdy all, I was wondering if all node entries (manually managed by
> CREATE
> > NODE... ) were manually created or if there was some automation in the
> > background that kept the pgxc metadata table up to date?  (For instance,
> > when a datanode connected to the gtm?)
> >
> > Thanks in advance,
> > -steve
> >
> >
> ------------------------------------------------------------------------------
> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > MVPs and experts. ON SALE this month only -- learn more at:
> > http://p.sf.net/sfu/learnnow-d2d
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>

I tried starting the GTM again :

> [postgres@centos6 data_gtm]$  /usr/local/pgsql/bin/initgtm -Z gtm -D
> /usr/local/
>             pgsql/data_gtm
> The files belonging to this GTM system will be owned by user "postgres".
> This user must also own the server process.
>
> initgtm: directory "/usr/local/pgsql/data_gtm" exists but is not empty
> If you want to create a new GTM system, either remove or empty
> the directory "/usr/local/pgsql/data_gtm" or run initgtm
> with an argument other than "/usr/local/pgsql/data_gtm".
> [postgres@centos6 data_gtm]$


This is what the gtm.log file says:

1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Starting GTM server at
> (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
> LOCATION:  main, main.c:552
> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Restoring last GXID
> to 10000
> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Started to run as
> GTM-Active.
> LOCATION:  main, main.c:624
> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  GTM shutting down.
> LOCATION:  ServerLoop, main.c:785
> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  Saving transaction
> info - next_gxid: 10000
> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
> Received signal 2
> 1:140344341612288:2013-01-23 16:01:18.754 EST -LOG:  Starting GTM server
> at (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
> LOCATION:  main, main.c:552
> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Restoring last GXID
> to 10000
> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Started to run as
> GTM-Active.
> LOCATION:  main, main.c:624
> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  GTM shutting down.
> LOCATION:  ServerLoop, main.c:785
> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  Saving transaction
> info - next_gxid: 10000
> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
> Received signal 2

~

Do we need to set a passwordless ssh before installing postgres-xc.
This is what I have done till now:
1. Configure,  the postgres-xc on all the three servers
2. added superuser postgres and gave permission to user postgres on the
/data_xx directories on all the servers
3. Started the postgres-xc instance on every server and components using
intidb
4. [postgres@centos6 bin]$ /usr/local/pgsql/bin/postgres -X -p 15432 -D
/usr/local/pgsql/data_Datanode1
LOG:  database system was shut down at 2013-01-22 18:14:22 EST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections




On Wed, Jan 23, 2013 at 10:10 AM, Ashutosh Bapat <
ash...@en...> wrote:

> On Wed, Jan 23, 2013 at 9:58 AM, Navin C <nav...@gm...>
> wrote:
> > Hello All,
> >
> > First of all, I am really happy to see PostgreSQL come up with a Cluster
> > based MPP solution.
> > I am excited to use Postgres-xc on my cluster.
> > I am facing some issues with the installation of Postgres-xc
> >
> > I am using Centos 6.3.
> > First of all I wanted to know whether we need to setup passwordless ssh
> on
> > all the servers of the cluster before starting to install Postgres-xc.
> >
> > I followed the steps in the documentation section Short version of
> > installation
> > ./configure
> > gmake
> > su
> > gmake install
> > adduser postgres
> > mkdir /usr/local/pgsql/data_coord1
> > mkdir /usr/local/pgsql/data_Datanode1
> > mkdir /usr/local/pgsql/data_Datanode2
> > mkdir /usr/local/pgsql/data_gtm
> > chown postgres /usr/local/pgsql/data_coord1
> > chown postgres /usr/local/pgsql/data_Datanode1
> > chown postgres /usr/local/pgsql/data_Datanode2
> > chown postgres /usr/local/pgsql/data_gtm
> > su - postgres
> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename
> > coord1
> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode1 --nodename
> > Datanode1
> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode2 --nodename
> > Datanode2
> > /usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm
> > /usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 &
> > /usr/local/pgsql/bin/postgres -X -p 15432 -D
> /usr/local/pgsql/data_Datanode1
> >>logfile 2>
> > /usr/local/pgsql/bin/postgres -X -p 15433 -D
> /usr/local/pgsql/data_Datanode2
> >>logfile 2>
> > /usr/local/pgsql/bin/postgres -C -D /usr/local/pgsql/data_coord1 >logfile
> > 2>&1 &
> > /usr/local/pgsql/bin/psql -c "CREATE NODE Datanode1 WITH (TYPE =
> ’Datanode’,
> > PORT = 15432)"
> >
> > When I try the last command as above It says the gtm is not connected,
> Xid
> > is invalid.
> > I also couldn't fine and gtm.pid file in data_gtm folder
> >
>
> It looks like the GTM didn't start. Can you please check in the log
> files what was the reason.
>
> > Can someone please help me with this. I need more detailed steps on each
> > Co-ordinator, Datanode and gtm
> > I have 3 servers 1 for GTM and 2 for Datanodes and Co-ordinators each.
> >
> >
> ------------------------------------------------------------------------------
> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > MVPs and experts. ON SALE this month only -- learn more at:
> > http://p.sf.net/sfu/learnnow-d2d
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>

On Wed, Jan 23, 2013 at 9:58 AM, Navin C <nav...@gm...> wrote:
> Hello All,
>
> First of all, I am really happy to see PostgreSQL come up with a Cluster
> based MPP solution.
> I am excited to use Postgres-xc on my cluster.
> I am facing some issues with the installation of Postgres-xc
>
> I am using Centos 6.3.
> First of all I wanted to know whether we need to setup passwordless ssh on
> all the servers of the cluster before starting to install Postgres-xc.
>
> I followed the steps in the documentation section Short version of
> installation
> ./configure
> gmake
> su
> gmake install
> adduser postgres
> mkdir /usr/local/pgsql/data_coord1
> mkdir /usr/local/pgsql/data_Datanode1
> mkdir /usr/local/pgsql/data_Datanode2
> mkdir /usr/local/pgsql/data_gtm
> chown postgres /usr/local/pgsql/data_coord1
> chown postgres /usr/local/pgsql/data_Datanode1
> chown postgres /usr/local/pgsql/data_Datanode2
> chown postgres /usr/local/pgsql/data_gtm
> su - postgres
> /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename
> coord1
> /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode1 --nodename
> Datanode1
> /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode2 --nodename
> Datanode2
> /usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm
> /usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 &
> /usr/local/pgsql/bin/postgres -X -p 15432 -D /usr/local/pgsql/data_Datanode1
>>logfile 2>
> /usr/local/pgsql/bin/postgres -X -p 15433 -D /usr/local/pgsql/data_Datanode2
>>logfile 2>
> /usr/local/pgsql/bin/postgres -C -D /usr/local/pgsql/data_coord1 >logfile
> 2>&1 &
> /usr/local/pgsql/bin/psql -c "CREATE NODE Datanode1 WITH (TYPE = ’Datanode’,
> PORT = 15432)"
>
> When I try the last command as above It says the gtm is not connected, Xid
> is invalid.
> I also couldn't fine and gtm.pid file in data_gtm folder
>

It looks like the GTM didn't start. Can you please check in the log
files what was the reason.

> Can someone please help me with this. I need more detailed steps on each
> Co-ordinator, Datanode and gtm
> I have 3 servers 1 for GTM and 2 for Datanodes and Co-ordinators each.
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hello All,

First of all, I am really happy to see PostgreSQL come up with a Cluster
based MPP solution.
I am excited to use Postgres-xc on my cluster.
I am facing some issues with the installation of Postgres-xc

I am using Centos 6.3.
First of all I wanted to know whether we need to setup *passwordless
ssh*on all the servers of the cluster before starting to install
Postgres-xc.

I followed the steps in the documentation section Short version of
installation
./configure
gmake
su
gmake install
adduser postgres
mkdir /usr/local/pgsql/data_coord1
mkdir /usr/local/pgsql/data_Datanode1
mkdir /usr/local/pgsql/data_Datanode2
mkdir /usr/local/pgsql/data_gtm
chown postgres /usr/local/pgsql/data_coord1
chown postgres /usr/local/pgsql/data_Datanode1
chown postgres /usr/local/pgsql/data_Datanode2
chown postgres /usr/local/pgsql/data_gtm
su - postgres
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename
coord1
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode1 --nodename
Datanode1
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode2 --nodename
Datanode2
/usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm
/usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 &
/usr/local/pgsql/bin/postgres -X -p 15432 -D
/usr/local/pgsql/data_Datanode1 >logfile 2>
/usr/local/pgsql/bin/postgres -X -p 15433 -D
/usr/local/pgsql/data_Datanode2 >logfile 2>
/usr/local/pgsql/bin/postgres -C -D /usr/local/pgsql/data_coord1 >logfile
2>&1 &
/usr/local/pgsql/bin/psql -c "CREATE NODE Datanode1 WITH (TYPE =
’Datanode’, PORT = 15432)"

When I try the last command as above It says the gtm is not connected, Xid
is invalid.
I also couldn't fine and gtm.pid file in data_gtm folder

Can someone please help me with this. I need more detailed steps on each
Co-ordinator, Datanode and gtm
I have 3 servers 1 for GTM and 2 for Datanodes and Co-ordinators each.

Please visit https://github.com/koichi-szk/PGXC-Tools/tree/master/pgxc_ctl

You will find pgxc_ctl and manual.txt.   (Sorry files in this page are
still WIP).

Pgxc_ctl helps operation/houskeeping/configuration of XC cluster.
It's bash script so you can find what you should do manually.
Manual.txt describes the usage of the script.

Good luck.
----------
Koichi Suzuki


2013/1/23 Steven Vaughan <ste...@da...>:
>
>   Howdy all, I was wondering if all node entries (manually managed by CREATE
> NODE... ) were manually created or if there was some automation in the
> background that kept the pgxc metadata table up to date?  (For instance,
> when a datanode connected to the gtm?)
>
> Thanks in advance,
> -steve
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

On Wed, Jan 23, 2013 at 7:03 AM, Steven Vaughan <
ste...@da...> wrote:

>
>   Howdy all, I was wondering if all node entries (manually managed by
> CREATE NODE... ) were manually created or if there was some automation in
> the background that kept the pgxc metadata table up to date?  (For
> instance, when a datanode connected to the gtm?)
>
You need to do that manually or with an external manager tool.
-- 
Michael Paquier
http://michael.otacoo.com

  Howdy all, I was wondering if all node entries (manually managed by
CREATE NODE... ) were manually created or if there was some automation in
the background that kept the pgxc metadata table up to date?  (For
instance, when a datanode connected to the gtm?)

Thanks in advance,
-steve

Ashutosh, Michael - thank you both for explanations, I really appreciate it.

Very good it is not the same thing as in pgpool :-)

regards,
Filip



On Mon, Jan 21, 2013 at 10:25 PM, Ashutosh Bapat
<ash...@en...> wrote:
> On Tue, Jan 22, 2013 at 3:41 AM, Filip Rembiałkowski
> <fil...@gm...> wrote:
>> Hi PG-XC authors.
>>
>> I am reading the PGCon2012 Tutorial presentation
>> [ http://postgresxc.wikia.com/wiki/File:20120515_PGXC_Tutorial_global_1.pdf ]
>>
>> Thank you for this document - it really covers many things.
>>
>> On page 44 you write about data distribution, and using distributed
>> tables versus replicated tables.
>> You use the term "Statement based replication" - and this alerted me instantly.
>>
>
> statement based replication simply means that data is replicated using
> SQL statements as opposed to WALs etc. These statements are generated
> at the coordinator, while planning the query. Postgres-XC is
> intelligent enough to understand which parts of original DML can be
> evaluated at the datanodes and which are not and it generates the
> statements by feeding values for the parts that need evaluation at
> coordinator (like volatile functions). So the problems like pgpool do
> not occur in Postgres-XC.
>
>> What _exactly_ did you mean by this?
>>
>> As you probably know, pgpool project uses exactly same term, and for
>> them it means sending same statement to many parallel nodes.
>> This approach is proven to be flawed by design - at least when you use
>> any volatile functions when writing, it makes replicas inconsistent.
>>
>> Thanks,
>> Filip
>>
>> ------------------------------------------------------------------------------
>> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>> MVPs and experts. SALE $99.99 this month only -- learn more at:
>> http://p.sf.net/sfu/learnmore_122412
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company

On Tue, Jan 22, 2013 at 3:41 AM, Filip Rembiałkowski
<fil...@gm...> wrote:
> Hi PG-XC authors.
>
> I am reading the PGCon2012 Tutorial presentation
> [ http://postgresxc.wikia.com/wiki/File:20120515_PGXC_Tutorial_global_1.pdf ]
>
> Thank you for this document - it really covers many things.
>
> On page 44 you write about data distribution, and using distributed
> tables versus replicated tables.
> You use the term "Statement based replication" - and this alerted me instantly.
>

statement based replication simply means that data is replicated using
SQL statements as opposed to WALs etc. These statements are generated
at the coordinator, while planning the query. Postgres-XC is
intelligent enough to understand which parts of original DML can be
evaluated at the datanodes and which are not and it generates the
statements by feeding values for the parts that need evaluation at
coordinator (like volatile functions). So the problems like pgpool do
not occur in Postgres-XC.

> What _exactly_ did you mean by this?
>
> As you probably know, pgpool project uses exactly same term, and for
> them it means sending same statement to many parallel nodes.
> This approach is proven to be flawed by design - at least when you use
> any volatile functions when writing, it makes replicas inconsistent.
>
> Thanks,
> Filip
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122412
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

On Tue, Jan 22, 2013 at 7:11 AM, Filip Rembiałkowski <
fil...@gm...> wrote:

> Hi PG-XC authors.
>
> I am reading the PGCon2012 Tutorial presentation
> [
> http://postgresxc.wikia.com/wiki/File:20120515_PGXC_Tutorial_global_1.pdf]
>
> Thank you for this document - it really covers many things.
>
> On page 44 you write about data distribution, and using distributed
> tables versus replicated tables.
> You use the term "Statement based replication" - and this alerted me
> instantly.
>
> What _exactly_ did you mean by this?
>
A query string is generated on Coordinator and used to communicate with the
remote nodes.
For replicated tables, the same query is used on all the nodes if SQL is a
DML, on one node if it is a SELECT.

As you probably know, pgpool project uses exactly same term, and for
> them it means sending same statement to many parallel nodes.
> This approach is proven to be flawed by design - at least when you use
> any volatile functions when writing, it makes replicas inconsistent.
>
The term might be the same, but XC is integrated directly with the Postgres
core, while pgpool is a different instance running on top of Postgres
servers. In consequence, the query planner in XC can use the concept of
query clause shippability. This means that some part of the query can be
shipped to remote nodes (meaning that the function or clause is managed on
remote Datanodes) or non-shippable (in this case clause is compiled on
Coordinator). A volatile function is always considered as non-shippable as
it plays by definition with the database data, so in the case of XC its
value is computed on Coordinator once and what is sent to Datanodes is:
1) A query of the type "SELECT * FROM tab WHERE val = $1" (It might have
been something similar to "SELECT * from tab WHERE val = nextval('seq')" on
Coordinator, $1 is replaced by the value computed on Coordinator)
2) The value computed.

Make a try with an XC with a typical example of this type:
CREATE TABLE tab (a serial) DISTRIBUTE BY REPLICATION;
INSERT INTO tab VALUES (DEFAULT);
Then have a look at the logs of the Datanodes, you will notice that what
has been sent down is not the query received by Coordinator, but something
like:
Received query from Coordinator 'INSERT INTO tab VALUES ($1)'
Received values: $1 = '1'

Or similar.
So no need to worry about data inconsistencies in those cases.
-- 
Michael Paquier
http://michael.otacoo.com

Hi PG-XC authors.

I am reading the PGCon2012 Tutorial presentation
[ http://postgresxc.wikia.com/wiki/File:20120515_PGXC_Tutorial_global_1.pdf ]

Thank you for this document - it really covers many things.

On page 44 you write about data distribution, and using distributed
tables versus replicated tables.
You use the term "Statement based replication" - and this alerted me instantly.

What _exactly_ did you mean by this?

As you probably know, pgpool project uses exactly same term, and for
them it means sending same statement to many parallel nodes.
This approach is proven to be flawed by design - at least when you use
any volatile functions when writing, it makes replicas inconsistent.

Thanks,
Filip

Hello,

Please visit http://postgresxc.wikia.com/wiki/Postgres-XC_Wiki.   I
added a couple of link and files uploaded in this page.

I will find other useful materials and will upload them.

Good luck.
----------
Koichi Suzuki


2013/1/21 Ashutosh Bapat <ash...@en...>:
> Hi Raju,
> There is a quite substantial (considering the age of this product)
> material for Postgres-XC available at various places.
>
> Take a look at http://postgres-xc.sourceforge.net/, that's project homepage.
>
> The product documentation can be found at
>
> http://postgres-xc.sourceforge.net/docs/
>
> You will find more material at
> http://sourceforge.net/projects/postgres-xc/files/?source=navbar
>
> Hope that's helpful.
>
> On Sat, Jan 19, 2013 at 4:13 AM, Raju Angani <an...@gm...> wrote:
>> Hi Ashutosh,
>>
>> This morning I came accross postgres-xc.
>> I'm very much interested in learning about postgres-xc, could you please
>> send me some pointers(pdf, details notes)
>> or even better if you could point me to tutorial(implementation,
>> maintenance, troubleshooting).
>>
>> I wanted to know the internals, I'm familiar with oracel RAC, and MSSQL
>> HADR
>>
>> Appreciate your help!
>>
>> Thank you
>> Raju
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122412
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general