postgres-xc-general Mailing List for Postgres-XC

On Fri, Jan 25, 2013 at 3:20 AM, Steven Vaughan <
ste...@da...> wrote:

>
>   Howdy, I was watching the GTM's log on a completely idle cluster (in
> fact it's just now being built - there are absolutely zero clients
> performing zero reads or writes) ... and I see a global txn id go by about
> every 30 seconds :
>
> For example :
>
> 1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Assigning new
> transaction ID = 10158
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
> 1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Sending transaction
> ids from 10158 to 10159
> LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
> 1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Assigning new
> transaction ID = 10159
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
> 1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Sending transaction
> ids from 10159 to 10160
> LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
> 1:139937565083392:2013-01-24 18:15:11.838 UTC -LOG:  Assigning new
> transaction ID = 10160
> LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
>
> ...is this a normal heartbeat / synchronization signal or is there
> something unexpected going on?  This cluster consists of precisely one GTM
> (on machine 1) and one GTM_Proxy, Coordinator, and Datanode (all on machine
> 2).
>
You have also automatic transactions in your cluster with autovacuum
workers.
So this is normal.
-- 
Michael Paquier
http://michael.otacoo.com

  Howdy, I was watching the GTM's log on a completely idle cluster (in fact
it's just now being built - there are absolutely zero clients performing
zero reads or writes) ... and I see a global txn id go by about every 30
seconds :

For example :

1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Assigning new
transaction ID = 10158
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
1:139937565083392:2013-01-24 18:14:41.802 UTC -LOG:  Sending transaction
ids from 10158 to 10159
LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Assigning new
transaction ID = 10159
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581
1:139937483585280:2013-01-24 18:15:10.863 UTC -LOG:  Sending transaction
ids from 10159 to 10160
LOCATION:  ProcessBeginTransactionGetGXIDCommandMulti, gtm_txn.c:1465
1:139937565083392:2013-01-24 18:15:11.838 UTC -LOG:  Assigning new
transaction ID = 10160
LOCATION:  GTM_GetGlobalTransactionIdMulti, gtm_txn.c:581

...is this a normal heartbeat / synchronization signal or is there
something unexpected going on?  This cluster consists of precisely one GTM
(on machine 1) and one GTM_Proxy, Coordinator, and Datanode (all on machine
2).

Thanks in advance,
-steve

I'm afraid initgtm failed.  Please try to cleanup everything in gtm
directory, run initgtm, check it is successful, edit gtm.conf and
start gtm.

To avoid errors, I'm doing as follows:

$ rm -f $gtmdir
$ mkdir -p $gtmdir
$ initgtm -Z gtm -D $gtmdir
$ (... edit gtm.conf ...)
$ (... start gtm ...)

Regards;
----------
Koichi Suzuki


2013/1/23 Navin C <nav...@gm...>:
>
>
> I tried starting the GTM again :
>>
>> [postgres@centos6 data_gtm]$  /usr/local/pgsql/bin/initgtm -Z gtm -D
>> /usr/local/
>> pgsql/data_gtm
>> The files belonging to this GTM system will be owned by user "postgres".
>> This user must also own the server process.
>>
>> initgtm: directory "/usr/local/pgsql/data_gtm" exists but is not empty
>> If you want to create a new GTM system, either remove or empty
>> the directory "/usr/local/pgsql/data_gtm" or run initgtm
>> with an argument other than "/usr/local/pgsql/data_gtm".
>> [postgres@centos6 data_gtm]$
>
>
> This is what the gtm.log file says:
>
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Starting GTM server
>> at (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
>> LOCATION:  main, main.c:552
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Restoring last GXID
>> to 10000
>> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
>> 1:140344638105344:2013-01-22 17:33:12.788 EST -LOG:  Started to run as
>> GTM-Active.
>> LOCATION:  main, main.c:624
>> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  GTM shutting down.
>> LOCATION:  ServerLoop, main.c:785
>> 1:140344638105344:2013-01-22 17:34:15.613 EST -LOG:  Saving transaction
>> info - next_gxid: 10000
>> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
>> Received signal 2
>> 1:140344341612288:2013-01-23 16:01:18.754 EST -LOG:  Starting GTM server
>> at (*:6666) -- control file /usr/local/pgsql/data_gtm/gtm.control
>> LOCATION:  main, main.c:552
>> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Restoring last GXID
>> to 10000
>> LOCATION:  GTM_RestoreTxnInfo, gtm_txn.c:2598
>> 1:140344341612288:2013-01-23 16:01:18.789 EST -LOG:  Started to run as
>> GTM-Active.
>> LOCATION:  main, main.c:624
>> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  GTM shutting down.
>> LOCATION:  ServerLoop, main.c:785
>> 1:140344341612288:2013-01-23 16:01:22.035 EST -LOG:  Saving transaction
>> info - next_gxid: 10000
>> LOCATION:  GTM_SaveTxnInfo, gtm_txn.c:2614
>> Received signal 2
>
> ~
>
> Do we need to set a passwordless ssh before installing postgres-xc.
> This is what I have done till now:
> 1. Configure,  the postgres-xc on all the three servers
> 2. added superuser postgres and gave permission to user postgres on the
> /data_xx directories on all the servers
> 3. Started the postgres-xc instance on every server and components using
> intidb
> 4. [postgres@centos6 bin]$ /usr/local/pgsql/bin/postgres -X -p 15432 -D
> /usr/local/pgsql/data_Datanode1
> LOG:  database system was shut down at 2013-01-22 18:14:22 EST
> LOG:  autovacuum launcher started
> LOG:  database system is ready to accept connections
>
>
>
>
> On Wed, Jan 23, 2013 at 10:10 AM, Ashutosh Bapat
> <ash...@en...> wrote:
>>
>> On Wed, Jan 23, 2013 at 9:58 AM, Navin C <nav...@gm...>
>> wrote:
>> > Hello All,
>> >
>> > First of all, I am really happy to see PostgreSQL come up with a Cluster
>> > based MPP solution.
>> > I am excited to use Postgres-xc on my cluster.
>> > I am facing some issues with the installation of Postgres-xc
>> >
>> > I am using Centos 6.3.
>> > First of all I wanted to know whether we need to setup passwordless ssh
>> > on
>> > all the servers of the cluster before starting to install Postgres-xc.
>> >
>> > I followed the steps in the documentation section Short version of
>> > installation
>> > ./configure
>> > gmake
>> > su
>> > gmake install
>> > adduser postgres
>> > mkdir /usr/local/pgsql/data_coord1
>> > mkdir /usr/local/pgsql/data_Datanode1
>> > mkdir /usr/local/pgsql/data_Datanode2
>> > mkdir /usr/local/pgsql/data_gtm
>> > chown postgres /usr/local/pgsql/data_coord1
>> > chown postgres /usr/local/pgsql/data_Datanode1
>> > chown postgres /usr/local/pgsql/data_Datanode2
>> > chown postgres /usr/local/pgsql/data_gtm
>> > su - postgres
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename
>> > coord1
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode1
>> > --nodename
>> > Datanode1
>> > /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_Datanode2
>> > --nodename
>> > Datanode2
>> > /usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm
>> > /usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 &
>> > /usr/local/pgsql/bin/postgres -X -p 15432 -D
>> > /usr/local/pgsql/data_Datanode1
>> >>logfile 2>
>> > /usr/local/pgsql/bin/postgres -X -p 15433 -D
>> > /usr/local/pgsql/data_Datanode2
>> >>logfile 2>
>> > /usr/local/pgsql/bin/postgres -C -D /usr/local/pgsql/data_coord1
>> > >logfile
>> > 2>&1 &
>> > /usr/local/pgsql/bin/psql -c "CREATE NODE Datanode1 WITH (TYPE =
>> > ’Datanode’,
>> > PORT = 15432)"
>> >
>> > When I try the last command as above It says the gtm is not connected,
>> > Xid
>> > is invalid.
>> > I also couldn't fine and gtm.pid file in data_gtm folder
>> >
>>
>> It looks like the GTM didn't start. Can you please check in the log
>> files what was the reason.
>>
>> > Can someone please help me with this. I need more detailed steps on each
>> > Co-ordinator, Datanode and gtm
>> > I have 3 servers 1 for GTM and 2 for Datanodes and Co-ordinators each.
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>> > MVPs and experts. ON SALE this month only -- learn more at:
>> > http://p.sf.net/sfu/learnnow-d2d
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>> >
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>
>
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

On Wed, 23 Jan 2013 07:28:27 -0600
Steven Vaughan <ste...@da...> wrote:

>   Thanks, pgxc_ctl is interesting - it's like a Swiss Army chainsaw!  The
> deploy/bootstrap (via ssh) feature is a neat addition (though it would be
> nice to be able to configure keys per host in the pgxc_ctl config file
> rather than relying on the ssh config to make 'ssh user@host' work without
> arguments). 

To deliver such key to the remote node, users have to type their password at least once.
I have simple bash script to help this for my personal use.   You have to type your password
just for a couple of times. 

Yes, I can add this to pgxc_ctl but I think this kind of capability should be taken care of by
system adminitration, not by XC.

---%<---- set_autologin ----------------%<---------------
#!/bin/bash
#
# set_autologin username host
#
function usage
{
        echo "Usage --"
        echo "   set_autologin username host"
}

if [ $# -lt 2 ]
then
        usage
        exit 1
fi

if [ $1 == '--help' ]
then
        usage
        exit 0
fi

cat > /tmp/$USER << EOF
#!/bin/bash
cd 
if [ -d .ssh ]; then 
        cd .
else
        mkdir .ssh
        chmod 0700 .ssh
fi
cd .ssh
touch authorized_keys2
chmod 0600 authorized_keys2
cat ~/id_dsa.pub >> authorized_keys2
rm ~/id_dsa.pub
EOF
chmod +x /tmp/$USER
cd
if [ -d .ssh ];
then
        cd .
else
        mkdir .ssh
        chmod 0700 .ssh
fi
if [ -f .ssh/id_dsa.pub ];
then
        cd .
else
        ssh-keygen -t dsa << EOF




EOF
fi
scp /tmp/$USER $1@$2:
scp .ssh/id_dsa.pub $1@$2:
ssh $1@$2 /home/$1/$USER
ssh $1@$2 rm /home/$1/$USER
rm /tmp/$USER
--->%--------------------------------------->%---------------

 One nice side effect is that it intrinsically produces the
> same binary version across the cluster (since it uses the same source
> tarball)  The bash script also answers my next question which would have
> been whether or not each coordinator needs to be sent the same CREATE NODE
> statements (clearly they do).  Are there any plans to make
> pgxc_node a replicated table at the coordinator level or add flags/syntax
> to CREATE NODE to synchronize mutations across the cluster?

This is interesting but have some fundamental issues.   To propagate CREATE NODE to all the nodes, we should know where to propagate to, which is defined by another CREATE NODE.    So far, it's safer to maintain current behaviour.

Regards;
---
Koichi Suzuki

> 
> Cheers,
> -steve
> 
> 
> On Tue, Jan 22, 2013 at 6:30 PM, Koichi Suzuki <koi...@gm...>wrote:
> 
> > Please visit https://github.com/koichi-szk/PGXC-Tools/tree/master/pgxc_ctl
> >
> > You will find pgxc_ctl and manual.txt.   (Sorry files in this page are
> > still WIP).
> >
> > Pgxc_ctl helps operation/houskeeping/configuration of XC cluster.
> > It's bash script so you can find what you should do manually.
> > Manual.txt describes the usage of the script.
> >
> > Good luck.
> > ----------
> > Koichi Suzuki
> >
> >
> > 2013/1/23 Steven Vaughan <ste...@da...>:
> > >
> > >   Howdy all, I was wondering if all node entries (manually managed by
> > CREATE
> > > NODE... ) were manually created or if there was some automation in the
> > > background that kept the pgxc metadata table up to date?  (For instance,
> > > when a datanode connected to the gtm?)
> > >
> > > Thanks in advance,
> > > -steve
> > >
> > >
> > ------------------------------------------------------------------------------
> > > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > > MVPs and experts. ON SALE this month only -- learn more at:
> > > http://p.sf.net/sfu/learnnow-d2d
> > > _______________________________________________
> > > Postgres-xc-general mailing list
> > > Pos...@li...
> > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> > >
> >