postgres-xc-general Mailing List for Postgres-XC

On Thu, Feb 27, 2014 at 10:16 PM, Juned Khan <jkh...@gm...> wrote:

> Hi Mason,
>
> What alternatives i have for my requirements ?
>
>
Hi Juned,

In the PostgreSQL community, there are a couple of open source multi-master
solutions, Bucardo and RubyRep, but they currently only allow for two
database nodes. I have heard of hacking together a Slony-based solution to
with triggers calling custom functions to roll your own solution.

There are other solutions in the greater PostgreSQL ecosystem. The case you
are describing is really the case that the company I work for addresses
with their flagship product, TED, but it is closed source. I believe
EnterpriseDB also offers a proprietary solution.

Coincidentally, we may provide a special custom version of XC for a
customer with some distributed requirements that are similar to your own.
 If this code gets open sourced, I can let you know, but that would be a
ways out in the future.


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Sorry for the inconvenience.   Please remove src/interfaces/ecpg/preproc/preproc.c before the build.   If your environment has appropriate bison, this will solve the problem.

Best;
---
Koichi Suzuki

2014/02/28 0:57、Lin Wang <lin...@gm...<mailto:lin...@gm...>> のメール：

hi all,
    when I compile the pgxc1.2 source code and execute the command "make", something wrong occures, as beblow:
make[5]: Leaving directory `/home/wln/pgxc/postgres-xc-1.2.0_beta/src/common'
gcc -DPGXC -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -pthread  -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4 -DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE   -c -o parser.o parser.c
parser.c:25:21: error：preproc.h：no such file or folder
parser.c:30: error：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘lookahead_yylval’
parser.c:31: error：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘lookahead_yylloc’

in the file "src/interfaces/ecpg/preproc/parser.c", I can get it:
#include "postgres_fe.h"

#include "extern.h"
#include "preproc.h"

but I can't find the  file "preproc.h" in the folder of source code. at the same time, I find the file of  "src/interfaces/ecpg/preproc/parser.c" in pgxc1.1 has also include the head file of   "preproc.h", and I also don't find the file  "preproc.h" in source code of pgxc1.1, but it is ok to run command "make" on source code of pgxc1.1.

I expect someone  can explain it.

Thanks,
waln

------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hi Mason,

What alternatives i have for my requirements ?


On Mon, Feb 24, 2014 at 8:53 PM, Mason Sharp <ms...@tr...>wrote:

>
>>>>>
>>>>> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...>wrote:
>>>>>
>>>>>> ,
>>>>>> Let me explain you my exact requirement. I have attached one image
>>>>>> for that.
>>>>>>
>>>>>> As per that image i have multiple states which has GTM, datanodes and
>>>>>> coordinators.
>>>>>> In future i will add more states. so how data will be replicated with
>>>>>> the datanodes of different stat.
>>>>>>
>>>>>> can i configure central GTM server which can handle all this stuff.,
>>>>>> or how do i configure each
>>>>>> GTM to communicate with other GTMs so all DBs can be replicated
>>>>>>
>>>>>> I just want to know is that possible using postgre-xc or not ? if yes
>>>>>> then please provide me relevant link and documentation for that.
>>>>>>
>>>>>
>>>
> Hi Juned,
>
> Your architecture seems to be a geographically distributed one, correct? I
> am not sure Postgres-XC is the best solution, but it depends on your
> requirements.  Is it acceptable if there is a network failure that some
> write transactions fail?  What about even reads if GTM is inaccessible? Are
> only some of your tables replicated? For those, is the write latency over a
> WAN acceptable? Again, it might all be fine, it just depends on your
> particular requirements.
>
>
> --
> Mason Sharp
>
> TransLattice - http://www.translattice.com
> Distributed and Clustered Database Solutions
>
>
>


-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

hi all,
    when I compile the pgxc1.2 source code and execute the command "make",
something wrong occures, as beblow:
make[5]: Leaving directory
`/home/wln/pgxc/postgres-xc-1.2.0_beta/src/common'
gcc -DPGXC -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -pthread  -D_REENTRANT
-D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE
-c -o parser.o parser.c
parser.c:25:21: error：preproc.h：no such file or folder
parser.c:30: error：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before
‘lookahead_yylval’
parser.c:31: error：expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before
‘lookahead_yylloc’

in the file "src/interfaces/ecpg/preproc/parser.c", I can get it:
#include "postgres_fe.h"

#include "extern.h"
#include "preproc.h"

but I can't find the  file "preproc.h" in the folder of source code. at the
same time, I find the file of  "src/interfaces/ecpg/preproc/parser.c" in
pgxc1.1 has also include the head file of   "preproc.h", and I also don't
find the file  "preproc.h" in source code of pgxc1.1, but it is ok to run
command "make" on source code of pgxc1.1.

I expect someone  can explain it.

Thanks,
waln

On Tue, Feb 25, 2014 at 11:20 PM, Nirmal Sharma <sha...@gm...>wrote:

> Hi All,
>
> Thanks for your reply.
>
> So now i have 2 questions:
>
> 1.  I have this machine  (Mem: 32 GB and Cores:16) and  have these 2
> option -> (a) Install postgress on the same and start using it. (b) Create
> a 3 node PG-XC cluster on the same machine and start using it.
> So can you tell me which option is better ? Does it make sense to create a
> cluster on the same machine or is it always better to use separate physical
> machines for each node to create an PG-XC cluster?
>

You should never deploy all your nodes on the same machine, that will just
deteriorate the performance.


>
> 2. What is the better configuration for PG-XC cluster among these ->
>     (a) For 3 Node cluster, having 1 coordinator and 3 data node is better
> than having 1 separate coordinator for each data node (i.e 3 Cord, 3 data
> node)?
>
>
It depends upon the work load. If the work load is such that the
coordinators are loaded and datanodes not, then you need to add more
coordinators. An example is large number of connections. So, you will need
to monitor your cluster, to see whether the coordinators are loaded.


>
> Please reply.
>
> Regards.
>
>
>
> On Mon, Feb 24, 2014 at 10:31 PM, Ashutosh Bapat <
> ash...@en...> wrote:
>
>>
>>
>>
>> On Tue, Feb 25, 2014 at 4:47 AM, Nirmal Sharma <sha...@gm...>wrote:
>>
>>> *This is my postgres box config:*
>>>   Mem             : 32GB  Cores           : 16  model name      :
>>> Intel(R) Xeon(R) CPU           E5630  @ 2.53GHz  stepping        : 2  cpu
>>> MHz         : 2527.184  cache size      : 12288 KB
>>> *This is my pg-xc (1 cordinator and 3 data node cluster) config:*
>>>
>>> (All the 4 physical boxes are same with below mentioned config)
>>>  Mem             : 24GB  Cores:            8  Model name:    stepping
>>> : 5  cpu MHz         : 1994.961  cache size      : 4096 KB
>>>
>>>
>> For an apple to apple comparison, it's better to have Postgres and each
>> of the coordinator/datanode of XC deployed on same kind of configuration.
>> From the description, it looks like single Postgres is getting better
>> hardware. So, it's not surprising to see XC not getting at-par performance
>> when compared to PG.
>>
>>>
>>>
>>>  In Pg-xc, only my dimension tables are replicated and facts are hash
>>> key distributed. Data and other things are same in pg and pg-xc.
>>> My pg-xc is giving me only marginal better performance than pg and thats
>>> the reason i am wondering whether to use pg-xc or not?
>>>
>>>
>> This distribution looks good. Make sure that you are using the right keys
>> for distributed tables. Please check my presentation at
>> http://www.youtube.com/watch?v=T1OPKvyj87Q. That might give some more
>> hints as to how to choose right distribution strategy.
>>
>>
>>> Regards
>>> ~Nirmal
>>>
>>>
>>> On Mon, Feb 24, 2014 at 3:09 PM, Mason Sharp <ms...@tr...>wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am using both postgress and postgress-XC and both are working fine
>>>>> but in terms of performance, they both are same(with same amount of data in
>>>>> both).
>>>>> I was hoping for better performance in pg-xc but it is just marginally
>>>>> better than PG and not much. So why then to use pg-xc?
>>>>>
>>>>> Can somebody tell me in what cases do we use pg-xc rather than PG?
>>>>>
>>>>>
>>>> It all depends on your workload and configuration. How many nodes and
>>>> physical servers are you using, and what is your workload and schema like?
>>>> Hopefully you are not replicating every table.
>>>>
>>>> There are cases like TPC-C where you will experience a noticeable
>>>> improvement.  There are other cases like TPC-H where performance will be
>>>> worse than plain  PostgreSQL.
>>>>
>>>>
>>>> --
>>>> Mason Sharp
>>>>
>>>> TransLattice - http://www.translattice.com
>>>> Distributed and Clustered Database Solutions
>>>>
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Flow-based real-time traffic analytics software. Cisco certified tool.
>>> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
>>> Customize your own dashboards, set traffic alerts and generate reports.
>>> Network behavioral analysis & security monitoring. All-in-one tool.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EnterpriseDB Corporation
>> The Postgres Database Company
>>
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Hi. I tried postgres xc 1.1 with these patches and postgis patch. Every
worked perfect. These patches are compatible. However I only was able to
install 1.5 and 2.0 versions of postgis. I had the following error with
postgis 2.1.1 version:

ERROR:  unsafe use of pseudo-type "internal"
DETAIL:  Transition type can not be "internal".

But I think that this trouble is for another thread.

I will use postgres xc 1.1 with postgis 2.0 in a production enviroment. I
will let you know my results. Thanks for help.



2014-02-23 21:05 GMT-05:00 Masataka Saito <pg...@gm...>:

> On 22 February 2014 01:38, Sergio Sinuco
> <ser...@da...> wrote:
> > I forgot to say that when the cluster start after crash, sequences of
> tables
> > advance two thousand units. For example, before kill gtm process,
> sequence
> > was in 4. After restart,  sequence's value of first insertion was in
> 2005.
>
> It's a feature.
> GTM doesn't write sequence values into the disk at each assignment for
> performance. GTM ensures the sequence value will unique but doesn't
> ensure it will continuous. It could skip 2000 at maximum.
>
> >
> >
> > 2014-02-21 11:32 GMT-05:00 Sergio Sinuco <
> ser...@da...>:
> >
> >> Hi. Thanks for reply. Effectively I applied the patches to 1.1 version
> and
> >> It worked. However I had to remove register.node files in GTM and GTM
> Proxy
> >> too because if I didn't do it then the cluster didn't work.
> >>
> >> Now I will try 1.1 version with these patches and postgis patch in order
> >> to find if they are compatible. I will let you know my results.
> >>
> >> I will try 1.2 version too.
>
> The patch Mr.Suzuki suggested is not for the issue I told.
>
> Regards.
>
> >>
> >> Thanks.
> >>
> >>
> >> 2014-02-19 21:15 GMT-05:00 Masataka Saito <pg...@gm...>:
> >>
> >>> I'm afraid you got this issue. Can you check logs in GTM and GTM proxy?
> >>> https://sourceforge.net/p/postgres-xc/bugs/474/
> >>>
> >>> If it prove right, please take the following steps.
> >>> 1. Stop all component
> >>> 2. Remove register.node in GTM and GTM Proxy
> >>> 3. Start GTM
> >>> 4. Start GTM proxy
> >>> 5. Start coordinators and datanodes
> >>>
> >>> Following materials will be help you to configure.
> >>> * Postgres Open 2013 (Especially SPOF Analysis in Slide deck)
> >>>
> >>>
> http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=Presentation_Materials
> >>>
> >>> And the configuration topics described in these materials are
> >>> deprecated yet, but very helpful to know what pgxc_ctl does.
> >>> * PGDay.EU, Oct 24th, 2012, Prague, Czeck
> >>>
> >>>
> http://postgres-xc.sourceforge.net/misc-docs/Prague_Presentation_20121024.pdf
> >>> * Presentation at PGConfChina 2012, Jun16th, 2012, Beijing, China (I
> >>> believe this is the best material for geeks)
> >>>
> >>>
> http://postgres-xc.sourceforge.net/misc-docs/PGConfChina_20120610_Distribution.pdf
> >>>
> >>> Regards.
> >>>
> >>> On 20 February 2014 02:03, Sergio Sinuco
> >>> <ser...@da...> wrote:
> >>> > Hello everybody.
> >>> >
> >>> > I just begin to try postgrexc 1.1. I have followed configuration
> >>> > tutorial
> >>> > for real servers in postgresxc.wikia.com
> >>> >
> >>> > http://postgresxc.wikia.com/wiki/Real_Server_configuration
> >>> >
> >>> > I setup three nodes. The first one with a GTM. The second and third
> >>> > with a
> >>> > GTM Proxy, a datanode and a coordinator in each one.
> >>> >
> >>> > Everything is OK, until I kill the GTM process in first node. After
> >>> > restart
> >>> > everything in all nodes I can't insert new data in tables. When I try
> >>> > it,
> >>> > following message is showed:
> >>> >
> >>> > ERROR:
> >>> > STATEMENT:  INSERT INTO public.test(name) VALUES ('CARLOS'::text)
> >>> >
> >>> > Then, I can't connect to coordinators and following message is
> showed:
> >>> >
> >>> > ERROR:  GTM error, could not obtain snapshot XID = 0
> >>> > WARNING:  can not connect to GTM: Connection refused
> >>> >
> >>> > I know that there is a way to configure Postgres-XC in order to have
> >>> > high
> >>> > availability (GTM Stand By and other stuff), but the question is,
> when
> >>> > GTM
> >>> > process and even its contingency crash there is no way to get a right
> >>> > behaviour of the cluster?
> >>> >
> >>> > Thanks.
> >>> >
> >>> > --
> >>> > Sergio E. Sinuco Leon
> >>> > Arquitecto de soluciones
> >>> > Datatraffic S.A.S.
> >>> > Móvil: (57) 310 884 26 50
> >>> > Fijo (+571) 7426160 Ext 115
> >>> > Calle 93 # 15-27 Ofc. 502
> >>> > Calle 29 # 6 - 94 Ofc. 601
> >>> > Bogotá, Colombia.
> >>> > www.datatraffic.com.co
> >>> >
> >>> >
> >>> >
> ------------------------------------------------------------------------------
> >>> > Managing the Performance of Cloud-Based Applications
> >>> > Take advantage of what the Cloud has to offer - Avoid Common
> Pitfalls.
> >>> > Read the Whitepaper.
> >>> >
> >>> >
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> >>> > _______________________________________________
> >>> > Postgres-xc-general mailing list
> >>> > Pos...@li...
> >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>> >
> >>
> >>
> >>
> >>
> >> --
> >> Sergio E. Sinuco Leon
> >> Arquitecto de soluciones
> >> Datatraffic S.A.S.
> >> Móvil: (57) 310 884 26 50
> >> Fijo (+571) 7426160 Ext 115
> >> Calle 93 # 15-27 Ofc. 502
> >> Calle 29 # 6 - 94 Ofc. 601
> >> Bogotá, Colombia.
> >> www.datatraffic.com.co
> >
> >
> >
> >
> > --
> > Sergio E. Sinuco Leon
> > Arquitecto de soluciones
> > Datatraffic S.A.S.
> > Móvil: (57) 310 884 26 50
> > Fijo (+571) 7426160 Ext 115
> > Calle 93 # 15-27 Ofc. 502
> > Calle 29 # 6 - 94 Ofc. 601
> > Bogotá, Colombia.
> > www.datatraffic.com.co
>



-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co

Hi All,

Thanks for your reply.

So now i have 2 questions:

1.  I have this machine  (Mem: 32 GB and Cores:16) and  have these 2 option
-> (a) Install postgress on the same and start using it. (b) Create a 3
node PG-XC cluster on the same machine and start using it.
So can you tell me which option is better ? Does it make sense to create a
cluster on the same machine or is it always better to use separate physical
machines for each node to create an PG-XC cluster?

2. What is the better configuration for PG-XC cluster among these ->
    (a) For 3 Node cluster, having 1 coordinator and 3 data node is better
than having 1 separate coordinator for each data node (i.e 3 Cord, 3 data
node)?


Please reply.

Regards.



On Mon, Feb 24, 2014 at 10:31 PM, Ashutosh Bapat <
ash...@en...> wrote:

>
>
>
> On Tue, Feb 25, 2014 at 4:47 AM, Nirmal Sharma <sha...@gm...>wrote:
>
>> *This is my postgres box config:*
>>   Mem             : 32GB  Cores           : 16  model name      :
>> Intel(R) Xeon(R) CPU           E5630  @ 2.53GHz  stepping        : 2  cpu
>> MHz         : 2527.184  cache size      : 12288 KB
>> *This is my pg-xc (1 cordinator and 3 data node cluster) config:*
>>
>> (All the 4 physical boxes are same with below mentioned config)
>>  Mem             : 24GB  Cores:            8  Model name:    stepping
>> : 5  cpu MHz         : 1994.961  cache size      : 4096 KB
>>
>>
> For an apple to apple comparison, it's better to have Postgres and each of
> the coordinator/datanode of XC deployed on same kind of configuration. From
> the description, it looks like single Postgres is getting better hardware.
> So, it's not surprising to see XC not getting at-par performance when
> compared to PG.
>
>>
>>
>>  In Pg-xc, only my dimension tables are replicated and facts are hash
>> key distributed. Data and other things are same in pg and pg-xc.
>> My pg-xc is giving me only marginal better performance than pg and thats
>> the reason i am wondering whether to use pg-xc or not?
>>
>>
> This distribution looks good. Make sure that you are using the right keys
> for distributed tables. Please check my presentation at
> http://www.youtube.com/watch?v=T1OPKvyj87Q. That might give some more
> hints as to how to choose right distribution strategy.
>
>
>> Regards
>> ~Nirmal
>>
>>
>> On Mon, Feb 24, 2014 at 3:09 PM, Mason Sharp <ms...@tr...>wrote:
>>
>>>
>>>
>>>
>>> On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...>wrote:
>>>
>>>> Hi,
>>>>
>>>> I am using both postgress and postgress-XC and both are working fine
>>>> but in terms of performance, they both are same(with same amount of data in
>>>> both).
>>>> I was hoping for better performance in pg-xc but it is just marginally
>>>> better than PG and not much. So why then to use pg-xc?
>>>>
>>>> Can somebody tell me in what cases do we use pg-xc rather than PG?
>>>>
>>>>
>>> It all depends on your workload and configuration. How many nodes and
>>> physical servers are you using, and what is your workload and schema like?
>>> Hopefully you are not replicating every table.
>>>
>>> There are cases like TPC-C where you will experience a noticeable
>>> improvement.  There are other cases like TPC-H where performance will be
>>> worse than plain  PostgreSQL.
>>>
>>>
>>> --
>>> Mason Sharp
>>>
>>> TransLattice - http://www.translattice.com
>>> Distributed and Clustered Database Solutions
>>>
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Flow-based real-time traffic analytics software. Cisco certified tool.
>> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
>> Customize your own dashboards, set traffic alerts and generate reports.
>> Network behavioral analysis & security monitoring. All-in-one tool.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>

On Tue, Feb 25, 2014 at 4:47 AM, Nirmal Sharma <sha...@gm...>wrote:

> *This is my postgres box config:*
>   Mem             : 32GB  Cores           : 16  model name      :
> Intel(R) Xeon(R) CPU           E5630  @ 2.53GHz  stepping        : 2  cpu
> MHz         : 2527.184  cache size      : 12288 KB
> *This is my pg-xc (1 cordinator and 3 data node cluster) config:*
>
> (All the 4 physical boxes are same with below mentioned config)
>  Mem             : 24GB  Cores:            8  Model name:    stepping
> : 5  cpu MHz         : 1994.961  cache size      : 4096 KB
>
>
For an apple to apple comparison, it's better to have Postgres and each of
the coordinator/datanode of XC deployed on same kind of configuration. From
the description, it looks like single Postgres is getting better hardware.
So, it's not surprising to see XC not getting at-par performance when
compared to PG.

>
>
>  In Pg-xc, only my dimension tables are replicated and facts are hash key
> distributed. Data and other things are same in pg and pg-xc.
> My pg-xc is giving me only marginal better performance than pg and thats
> the reason i am wondering whether to use pg-xc or not?
>
>
This distribution looks good. Make sure that you are using the right keys
for distributed tables. Please check my presentation at
http://www.youtube.com/watch?v=T1OPKvyj87Q. That might give some more hints
as to how to choose right distribution strategy.


> Regards
> ~Nirmal
>
>
> On Mon, Feb 24, 2014 at 3:09 PM, Mason Sharp <ms...@tr...>wrote:
>
>>
>>
>>
>> On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...>wrote:
>>
>>> Hi,
>>>
>>> I am using both postgress and postgress-XC and both are working fine but
>>> in terms of performance, they both are same(with same amount of data in
>>> both).
>>> I was hoping for better performance in pg-xc but it is just marginally
>>> better than PG and not much. So why then to use pg-xc?
>>>
>>> Can somebody tell me in what cases do we use pg-xc rather than PG?
>>>
>>>
>> It all depends on your workload and configuration. How many nodes and
>> physical servers are you using, and what is your workload and schema like?
>> Hopefully you are not replicating every table.
>>
>> There are cases like TPC-C where you will experience a noticeable
>> improvement.  There are other cases like TPC-H where performance will be
>> worse than plain  PostgreSQL.
>>
>>
>> --
>> Mason Sharp
>>
>> TransLattice - http://www.translattice.com
>> Distributed and Clustered Database Solutions
>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On Mon, Feb 24, 2014 at 3:26 PM, Nirmal Sharma <sha...@gm...>wrote:

> Also, can you please explain what do you mean by TPC-C and TPC-H?
>

These are different types of benchmarks. See http://www.tpc.org


> We are actually using PG for our reporting application that read the data
> from database by running complex queries (query uses aggregate and wndows
> functions) and we are using connection pools for users to connect and view
> data from application. Also, users can changes the data ( update/insert)
> from the application.
> So do you think that our application is TPC-H?
>

I am guessing that not all queries are slow, just some queries. You may
have to run EXPLAIN to examine what is going on.  This type of query
handling is an area that could be improved in Postgres-XC. If you can get
by without window functions, you could take a look at the Stado project. I
recommend this branch: https://code.launchpad.net/~sgdg/stado/stado



-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Also, can you please explain what do you mean by TPC-C and TPC-H?
We are actually using PG for our reporting application that read the data
from database by running complex queries (query uses aggregate and wndows
functions) and we are using connection pools for users to connect and view
data from application. Also, users can changes the data ( update/insert)
from the application.
So do you think that our application is TPC-H?

Regards



On Mon, Feb 24, 2014 at 3:17 PM, Nirmal Sharma <sha...@gm...>wrote:

> *This is my postgres box config:*
>   Mem             : 32GB  Cores           : 16  model name      :
> Intel(R) Xeon(R) CPU           E5630  @ 2.53GHz  stepping        : 2  cpu
> MHz         : 2527.184  cache size      : 12288 KB
> *This is my pg-xc (1 cordinator and 3 data node cluster) config:*
>
> (All the 4 physical boxes are same with below mentioned config)
>  Mem             : 24GB  Cores:            8  Model name:    stepping
> : 5  cpu MHz         : 1994.961  cache size      : 4096 KB
>
>
>
>  In Pg-xc, only my dimension tables are replicated and facts are hash key
> distributed. Data and other things are same in pg and pg-xc.
> My pg-xc is giving me only marginal better performance than pg and thats
> the reason i am wondering whether to use pg-xc or not?
>
> Regards
> ~Nirmal
>
>
> On Mon, Feb 24, 2014 at 3:09 PM, Mason Sharp <ms...@tr...>wrote:
>
>>
>>
>>
>> On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...>wrote:
>>
>>> Hi,
>>>
>>> I am using both postgress and postgress-XC and both are working fine but
>>> in terms of performance, they both are same(with same amount of data in
>>> both).
>>> I was hoping for better performance in pg-xc but it is just marginally
>>> better than PG and not much. So why then to use pg-xc?
>>>
>>> Can somebody tell me in what cases do we use pg-xc rather than PG?
>>>
>>>
>> It all depends on your workload and configuration. How many nodes and
>> physical servers are you using, and what is your workload and schema like?
>> Hopefully you are not replicating every table.
>>
>> There are cases like TPC-C where you will experience a noticeable
>> improvement.  There are other cases like TPC-H where performance will be
>> worse than plain  PostgreSQL.
>>
>>
>> --
>> Mason Sharp
>>
>> TransLattice - http://www.translattice.com
>> Distributed and Clustered Database Solutions
>>
>>
>>
>

*This is my postgres box config:*
  Mem             : 32GB  Cores           : 16  model name      : Intel(R)
Xeon(R) CPU           E5630  @ 2.53GHz  stepping        : 2  cpu
MHz         : 2527.184  cache size      : 12288 KB
*This is my pg-xc (1 cordinator and 3 data node cluster) config:*

(All the 4 physical boxes are same with below mentioned config)
 Mem             : 24GB  Cores:            8  Model name:    stepping
: 5  cpu MHz         : 1994.961  cache size      : 4096 KB



 In Pg-xc, only my dimension tables are replicated and facts are hash key
distributed. Data and other things are same in pg and pg-xc.
My pg-xc is giving me only marginal better performance than pg and thats
the reason i am wondering whether to use pg-xc or not?

Regards
~Nirmal


On Mon, Feb 24, 2014 at 3:09 PM, Mason Sharp <ms...@tr...>wrote:

>
>
>
> On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...>wrote:
>
>> Hi,
>>
>> I am using both postgress and postgress-XC and both are working fine but
>> in terms of performance, they both are same(with same amount of data in
>> both).
>> I was hoping for better performance in pg-xc but it is just marginally
>> better than PG and not much. So why then to use pg-xc?
>>
>> Can somebody tell me in what cases do we use pg-xc rather than PG?
>>
>>
> It all depends on your workload and configuration. How many nodes and
> physical servers are you using, and what is your workload and schema like?
> Hopefully you are not replicating every table.
>
> There are cases like TPC-C where you will experience a noticeable
> improvement.  There are other cases like TPC-H where performance will be
> worse than plain  PostgreSQL.
>
>
> --
> Mason Sharp
>
> TransLattice - http://www.translattice.com
> Distributed and Clustered Database Solutions
>
>
>

On Mon, Feb 24, 2014 at 2:11 PM, Nirmal Sharma <sha...@gm...>wrote:

> Hi,
>
> I am using both postgress and postgress-XC and both are working fine but
> in terms of performance, they both are same(with same amount of data in
> both).
> I was hoping for better performance in pg-xc but it is just marginally
> better than PG and not much. So why then to use pg-xc?
>
> Can somebody tell me in what cases do we use pg-xc rather than PG?
>
>
It all depends on your workload and configuration. How many nodes and
physical servers are you using, and what is your workload and schema like?
Hopefully you are not replicating every table.

There are cases like TPC-C where you will experience a noticeable
improvement.  There are other cases like TPC-H where performance will be
worse than plain  PostgreSQL.


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Hi,

I am using both postgress and postgress-XC and both are working fine but in
terms of performance, they both are same(with same amount of data in both).
I was hoping for better performance in pg-xc but it is just marginally
better than PG and not much. So why then to use pg-xc?

Can somebody tell me in what cases do we use pg-xc rather than PG?

Regards.

>
>
>>>>
>>>> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...>wrote:
>>>>
>>>>> ,
>>>>> Let me explain you my exact requirement. I have attached one image for
>>>>> that.
>>>>>
>>>>> As per that image i have multiple states which has GTM, datanodes and
>>>>> coordinators.
>>>>> In future i will add more states. so how data will be replicated with
>>>>> the datanodes of different stat.
>>>>>
>>>>> can i configure central GTM server which can handle all this stuff.,
>>>>> or how do i configure each
>>>>> GTM to communicate with other GTMs so all DBs can be replicated
>>>>>
>>>>> I just want to know is that possible using postgre-xc or not ? if yes
>>>>> then please provide me relevant link and documentation for that.
>>>>>
>>>>
>>
Hi Juned,

Your architecture seems to be a geographically distributed one, correct? I
am not sure Postgres-XC is the best solution, but it depends on your
requirements.  Is it acceptable if there is a network failure that some
write transactions fail?  What about even reads if GTM is inaccessible? Are
only some of your tables replicated? For those, is the write latency over a
WAN acceptable? Again, it might all be fine, it just depends on your
particular requirements.


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Hi Abbas,

I should try this solution in my local system first.
If it does as you explain then it would be the perfect solution for me.

Thank you very much ;)


On Mon, Feb 24, 2014 at 1:20 PM, Abbas Butt <abb...@en...>wrote:

>
>
>
> On Mon, Feb 24, 2014 at 10:52 AM, Juned Khan <jkh...@gm...> wrote:
>
>> Hi Abbas,
>>
>> Thanks for your response.
>>
>> The problem with using single GTM is if GTM itself will go down then we
>> will loose everything.
>>
>
> GTM Slave can be used for this purpose.
>
>
>>
>> The application in which we want to use postgre-xc is billing kind of
>> application.
>> We setup this appllication in multiple states which will use same
>> database cluster.
>>
>
> Please consider distributing customers based on hash or modulo of
> customer_id. This will make sure that the customers get distributed evenly
> in the cluster. Ideally you should distribute customers based on the state
> where they live, but due to current foreign key limitations of XC you might
> not be able to do it.
>
>
>
>>
>>
>>
>> On Sun, Feb 23, 2014 at 11:24 PM, Abbas Butt <abb...@en...
>> > wrote:
>>
>>> Dear Juned,
>>> A Postgres-XC cluster needs one GTM. You do not need a GTM at each
>>> state. If you need to have data replicated across all the nodes all you
>>> need to do is to distribute all tables of the schema by replication using
>>> the syntax
>>> create table tab(a int, b int) distribute by replication;
>>> Having said that, you need to consider the following points
>>> a) It seems that the nodes of the cluster would be geographically apart,
>>> if yes, please keep in mind that Postgres-XC does not cater for network
>>> failures. If the link to one of the cluster nodes is down while inserting
>>> data, the insert would fail. A select would however pass depending on which
>>> coordinator the application is connected to.
>>> b) Normally you would not replicate all the data to all the nodes, you
>>> should use a mix of replicated and distributed tables. We can guide better
>>> if you can explain a little bit what application you are trying to build
>>> with Postgres-XC as backend.
>>> Regards
>>>
>>>
>>>
>>> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...> wrote:
>>>
>>>> Hi All,
>>>>
>>>> PFA
>>>>
>>>> Let me explain you my exact requirement. I have attached one image for
>>>> that.
>>>>
>>>> As per that image i have multiple states which has GTM, datanodes and
>>>> coordinators.
>>>> In future i will add more states. so how data will be replicated with
>>>> the datanodes of different stat.
>>>>
>>>> can i configure central GTM server which can handle all this stuff., or
>>>> how do i configure each
>>>> GTM to communicate with other GTMs so all DBs can be replicated
>>>>
>>>> I just want to know is that possible using postgre-xc or not ? if yes
>>>> then please provide me relevant link and documentation for that.
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Juned Khan
>>>> iNextrix Technologies Pvt Ltd.
>>>> www.inextrix.com
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Managing the Performance of Cloud-Based Applications
>>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>>> Read the Whitepaper.
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> Postgres-xc-general mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> *Abbas*
>>>  Architect
>>>
>>> Ph: 92.334.5100153
>>>  Skype ID: gabbasb
>>> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>>>
>>> *Follow us on Twitter*
>>> @EnterpriseDB
>>>
>>> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>>>
>>
>>
>>
>> --
>> Thanks,
>> Juned Khan
>> iNextrix Technologies Pvt Ltd.
>> www.inextrix.com
>>
>
>
>
> --
> --
> *Abbas*
>  Architect
>
> Ph: 92.334.5100153
>  Skype ID: gabbasb
> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>
> *Follow us on Twitter*
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>



-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Hi Koichi,

i think i should try this with my local system first.

Thanks you very much for help.


On Mon, Feb 24, 2014 at 11:20 AM, 鈴木 幸市 <ko...@in...> wrote:

>  You cannot configure more than one GTM master in a cluster.   You need to
> configure only one GTM, which will takes care of all the nodes in the
> cluster.
>
>  You need to create tables with “DISTRIBUTE BY REPLICATION” clause.
> Documentation will be found at
> http://postgres-xc.sourceforge.net/docs/1_1/sql-createtable.html
>
>  Regards;
> ---
> Koichi Suzuki
>
>  2014/02/22 17:20、Juned Khan <jkh...@gm...> のメール：
>
>    Hi Koichi,
>
>  PFA
>  Thanks for your response.
>
>  Let me explain you my exact requirement. I have attached one image for
> that.
>
>  As per that image i have multiple states which has GTM, datanodes and
> coordinators.
> In future i will add more states. so how data will be replicated with the
> datanodes of different stat.
>
>  can i configure central GTM server which can handle all this stuff., or
> how do i configure each
> GTM to communicate with other GTMs so all DBs can be replicated
>
>
>
> On Mon, Feb 17, 2014 at 1:08 PM, Koichi Suzuki <koi...@gm...>wrote:
>
>> If you mean master-master just replication, you should specify each
>> table as DISTRIBUTE BY REPLICATION.
>>
>> Adding servers could be complicated steps.   I advice to use pgxc_ctl
>> to configure and maintain your cluster.    You will find the document
>> at http://postgres-xc.sourceforge.net/docs/1_1/pgxc-ctl.html
>>
>> When you add new server, you can add new
>> datanode/coordinator/gtm_proxy with pgxc_ctl commands.    Also, you
>> can redistribute your table (in this case, add a node to replicate the
>> data), you can use ALTER TABLE.   Document will be at
>> http://postgres-xc.sourceforge.net/docs/1_1/sql-altertable.html
>>
>> XC is not intended to use as multimaster replication.    It assumes
>> that you have a few big tables updated very frequently (transaction
>> tables) which should be distributed, not replicated and other tables
>> which are relatively stable and are joined frequently with transaction
>> tables (master tables).
>>
>> Regards;
>> ---
>> Koichi Suzuki
>>
>>
>> 2014-02-17 15:51 GMT+09:00 Juned Khan <jkh...@gm...>:
>>  > Hi All,
>> >
>> > I want to use postgre-xc for multi master uses. I want to setup two
>> server
>> > with GTM,cordinator and datanodes.
>> >
>> > And in future i want to add another servers as well with same
>> configuration,
>> > and in future i will add more.
>> >
>> > To test this, i have setup two server with GTM,Coordinator and datanode
>> on
>> > same server. i have follow below link for configuration.
>> >
>> > http://postgresxc.wikia.com/wiki/Laptop_configuration
>> >
>> > Now How do i configuration this two system as master master
>> configuration,
>> > will add more masters in future.
>> >
>> > I found link of GTM standby configuration,then i have setup GTM standby
>> on
>> > one separate system using below link.
>> >
>> > http://postgresxc.wikia.com/wiki/GTM_Standby_Configuration
>> >
>> > then i started GTM, datanodes and ordinators on both server, then i
>> tried to
>> > run below command on Stand By  server.
>> >
>> > gtm_ctl promote -Z gtm -D /home/postgresxc/pgxc/gtm/ -o "-s gtm -t
>> 20001"
>> >
>> > But got this error:
>> > gtm_ctl: could not send promote signal (PID: 11937): No such process
>> >
>> > So my question is am i going to right way to achieve my goal ? how do i
>> > avoid this error ?
>> >
>> >
>> > --
>> > Thanks,
>> > Juned Khan
>> > iNextrix Technologies Pvt Ltd.
>> > www.inextrix.com
>> >
>>  >
>> ------------------------------------------------------------------------------
>> > Android apps run on BlackBerry 10
>> > Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>> > Now with support for Jelly Bean, Bluetooth, Mapview and more.
>> > Get your Android app in front of a whole new audience.  Start now.
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>> >
>>
>
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>  <diagram_new.png>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk_______________________________________________
>
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>


-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

On Mon, Feb 24, 2014 at 10:52 AM, Juned Khan <jkh...@gm...> wrote:

> Hi Abbas,
>
> Thanks for your response.
>
> The problem with using single GTM is if GTM itself will go down then we
> will loose everything.
>

GTM Slave can be used for this purpose.


>
> The application in which we want to use postgre-xc is billing kind of
> application.
> We setup this appllication in multiple states which will use same database
> cluster.
>

Please consider distributing customers based on hash or modulo of
customer_id. This will make sure that the customers get distributed evenly
in the cluster. Ideally you should distribute customers based on the state
where they live, but due to current foreign key limitations of XC you might
not be able to do it.



>
>
>
> On Sun, Feb 23, 2014 at 11:24 PM, Abbas Butt <abb...@en...>wrote:
>
>> Dear Juned,
>> A Postgres-XC cluster needs one GTM. You do not need a GTM at each state.
>> If you need to have data replicated across all the nodes all you need to do
>> is to distribute all tables of the schema by replication using the syntax
>> create table tab(a int, b int) distribute by replication;
>> Having said that, you need to consider the following points
>> a) It seems that the nodes of the cluster would be geographically apart,
>> if yes, please keep in mind that Postgres-XC does not cater for network
>> failures. If the link to one of the cluster nodes is down while inserting
>> data, the insert would fail. A select would however pass depending on which
>> coordinator the application is connected to.
>> b) Normally you would not replicate all the data to all the nodes, you
>> should use a mix of replicated and distributed tables. We can guide better
>> if you can explain a little bit what application you are trying to build
>> with Postgres-XC as backend.
>> Regards
>>
>>
>>
>> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...> wrote:
>>
>>> Hi All,
>>>
>>> PFA
>>>
>>> Let me explain you my exact requirement. I have attached one image for
>>> that.
>>>
>>> As per that image i have multiple states which has GTM, datanodes and
>>> coordinators.
>>> In future i will add more states. so how data will be replicated with
>>> the datanodes of different stat.
>>>
>>> can i configure central GTM server which can handle all this stuff., or
>>> how do i configure each
>>> GTM to communicate with other GTMs so all DBs can be replicated
>>>
>>> I just want to know is that possible using postgre-xc or not ? if yes
>>> then please provide me relevant link and documentation for that.
>>>
>>>
>>> --
>>> Thanks,
>>> Juned Khan
>>> iNextrix Technologies Pvt Ltd.
>>> www.inextrix.com
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Managing the Performance of Cloud-Based Applications
>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>> Read the Whitepaper.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>>
>> --
>> --
>> *Abbas*
>>  Architect
>>
>> Ph: 92.334.5100153
>>  Skype ID: gabbasb
>> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>>
>> *Follow us on Twitter*
>> @EnterpriseDB
>>
>> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>>
>
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>



-- 
-- 
*Abbas*
 Architect

Ph: 92.334.5100153
 Skype ID: gabbasb
www.enterprisedb.co
<http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>

*Follow us on Twitter*
@EnterpriseDB

Visit EnterpriseDB for tutorials, webinars,
whitepapers<http://www.enterprisedb.com/resources-community>and
more<http://www.enterprisedb.com/resources-community>

GTM is separate entity so if environment for GTM itself is okay, it stay
running.   Coordinators/GTM proxies can reconnect to this GTM.   Of course,
you can setup GTM slave for failover.

Regards;

---
Koichi Suzuki


2014-02-24 14:52 GMT+09:00 Juned Khan <jkh...@gm...>:

> Hi Abbas,
>
> Thanks for your response.
>
> The problem with using single GTM is if GTM itself will go down then we
> will loose everything.
>
> The application in which we want to use postgre-xc is billing kind of
> application.
> We setup this appllication in multiple states which will use same database
> cluster.
>
>
>
>
> On Sun, Feb 23, 2014 at 11:24 PM, Abbas Butt <abb...@en...>wrote:
>
>> Dear Juned,
>> A Postgres-XC cluster needs one GTM. You do not need a GTM at each state.
>> If you need to have data replicated across all the nodes all you need to do
>> is to distribute all tables of the schema by replication using the syntax
>> create table tab(a int, b int) distribute by replication;
>> Having said that, you need to consider the following points
>> a) It seems that the nodes of the cluster would be geographically apart,
>> if yes, please keep in mind that Postgres-XC does not cater for network
>> failures. If the link to one of the cluster nodes is down while inserting
>> data, the insert would fail. A select would however pass depending on which
>> coordinator the application is connected to.
>> b) Normally you would not replicate all the data to all the nodes, you
>> should use a mix of replicated and distributed tables. We can guide better
>> if you can explain a little bit what application you are trying to build
>> with Postgres-XC as backend.
>> Regards
>>
>>
>>
>> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...> wrote:
>>
>>> Hi All,
>>>
>>> PFA
>>>
>>> Let me explain you my exact requirement. I have attached one image for
>>> that.
>>>
>>> As per that image i have multiple states which has GTM, datanodes and
>>> coordinators.
>>> In future i will add more states. so how data will be replicated with
>>> the datanodes of different stat.
>>>
>>> can i configure central GTM server which can handle all this stuff., or
>>> how do i configure each
>>> GTM to communicate with other GTMs so all DBs can be replicated
>>>
>>> I just want to know is that possible using postgre-xc or not ? if yes
>>> then please provide me relevant link and documentation for that.
>>>
>>>
>>> --
>>> Thanks,
>>> Juned Khan
>>> iNextrix Technologies Pvt Ltd.
>>> www.inextrix.com
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Managing the Performance of Cloud-Based Applications
>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>> Read the Whitepaper.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>>
>> --
>> --
>> *Abbas*
>>  Architect
>>
>> Ph: 92.334.5100153
>>  Skype ID: gabbasb
>> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>>
>> *Follow us on Twitter*
>> @EnterpriseDB
>>
>> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>>
>
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>

You cannot configure more than one GTM master in a cluster.   You need to configure only one GTM, which will takes care of all the nodes in the cluster.

You need to create tables with “DISTRIBUTE BY REPLICATION” clause.   Documentation will be found at http://postgres-xc.sourceforge.net/docs/1_1/sql-createtable.html

Regards;
---
Koichi Suzuki

2014/02/22 17:20、Juned Khan <jkh...@gm...<mailto:jkh...@gm...>> のメール：

Hi Koichi,

PFA
Thanks for your response.

Let me explain you my exact requirement. I have attached one image for that.

As per that image i have multiple states which has GTM, datanodes and coordinators.
In future i will add more states. so how data will be replicated with the datanodes of different stat.

can i configure central GTM server which can handle all this stuff., or how do i configure each
GTM to communicate with other GTMs so all DBs can be replicated



On Mon, Feb 17, 2014 at 1:08 PM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote:
If you mean master-master just replication, you should specify each
table as DISTRIBUTE BY REPLICATION.

Adding servers could be complicated steps.   I advice to use pgxc_ctl
to configure and maintain your cluster.    You will find the document
at http://postgres-xc.sourceforge.net/docs/1_1/pgxc-ctl.html

When you add new server, you can add new
datanode/coordinator/gtm_proxy with pgxc_ctl commands.    Also, you
can redistribute your table (in this case, add a node to replicate the
data), you can use ALTER TABLE.   Document will be at
http://postgres-xc.sourceforge.net/docs/1_1/sql-altertable.html

XC is not intended to use as multimaster replication.    It assumes
that you have a few big tables updated very frequently (transaction
tables) which should be distributed, not replicated and other tables
which are relatively stable and are joined frequently with transaction
tables (master tables).

Regards;
---
Koichi Suzuki


2014-02-17 15:51 GMT+09:00 Juned Khan <jkh...@gm...<mailto:jkh...@gm...>>:
> Hi All,
>
> I want to use postgre-xc for multi master uses. I want to setup two server
> with GTM,cordinator and datanodes.
>
> And in future i want to add another servers as well with same configuration,
> and in future i will add more.
>
> To test this, i have setup two server with GTM,Coordinator and datanode on
> same server. i have follow below link for configuration.
>
> http://postgresxc.wikia.com/wiki/Laptop_configuration
>
> Now How do i configuration this two system as master master configuration,
> will add more masters in future.
>
> I found link of GTM standby configuration,then i have setup GTM standby on
> one separate system using below link.
>
> http://postgresxc.wikia.com/wiki/GTM_Standby_Configuration
>
> then i started GTM, datanodes and ordinators on both server, then i tried to
> run below command on Stand By  server.
>
> gtm_ctl promote -Z gtm -D /home/postgresxc/pgxc/gtm/ -o "-s gtm -t 20001"
>
> But got this error:
> gtm_ctl: could not send promote signal (PID: 11937): No such process
>
> So my question is am i going to right way to achieve my goal ? how do i
> avoid this error ?
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com<http://www.inextrix.com/>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...<mailto:Pos...@li...>
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



--
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com<http://www.inextrix.com/>
<diagram_new.png>------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hi Abbas,

Thanks for your response.

The problem with using single GTM is if GTM itself will go down then we
will loose everything.

The application in which we want to use postgre-xc is billing kind of
application.
We setup this appllication in multiple states which will use same database
cluster.




On Sun, Feb 23, 2014 at 11:24 PM, Abbas Butt <abb...@en...>wrote:

> Dear Juned,
> A Postgres-XC cluster needs one GTM. You do not need a GTM at each state.
> If you need to have data replicated across all the nodes all you need to do
> is to distribute all tables of the schema by replication using the syntax
> create table tab(a int, b int) distribute by replication;
> Having said that, you need to consider the following points
> a) It seems that the nodes of the cluster would be geographically apart,
> if yes, please keep in mind that Postgres-XC does not cater for network
> failures. If the link to one of the cluster nodes is down while inserting
> data, the insert would fail. A select would however pass depending on which
> coordinator the application is connected to.
> b) Normally you would not replicate all the data to all the nodes, you
> should use a mix of replicated and distributed tables. We can guide better
> if you can explain a little bit what application you are trying to build
> with Postgres-XC as backend.
> Regards
>
>
>
> On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...> wrote:
>
>> Hi All,
>>
>> PFA
>>
>> Let me explain you my exact requirement. I have attached one image for
>> that.
>>
>> As per that image i have multiple states which has GTM, datanodes and
>> coordinators.
>> In future i will add more states. so how data will be replicated with the
>> datanodes of different stat.
>>
>> can i configure central GTM server which can handle all this stuff., or
>> how do i configure each
>> GTM to communicate with other GTMs so all DBs can be replicated
>>
>> I just want to know is that possible using postgre-xc or not ? if yes
>> then please provide me relevant link and documentation for that.
>>
>>
>> --
>> Thanks,
>> Juned Khan
>> iNextrix Technologies Pvt Ltd.
>> www.inextrix.com
>>
>>
>> ------------------------------------------------------------------------------
>> Managing the Performance of Cloud-Based Applications
>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> Read the Whitepaper.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
> --
> --
> *Abbas*
>  Architect
>
> Ph: 92.334.5100153
>  Skype ID: gabbasb
> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>
> *Follow us on Twitter*
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>



-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Sorry I didn't respond for a  while.    I took a look at your
configuration and anything special.

Yes, between coordinator and datamode can stay open.   It is owned by
the pooler process, which manages coordinator and datanode connection.
   Even though a coordinator backend does not need a connection to a
datanode any more, the pooler can keep it opened for subsequent use,
to save connection overhead.

Regards;
---
Koichi Suzuki


2014-02-16 17:19 GMT+09:00 Rishi Ramraj <the...@gm...>:
> FYI: I've been running some more tests on the REL1_2_STABLE branch to see if
> I can find more information on what's wrong with my system. To keep my setup
> simple, I have reverted to using initdb and initgtm to create the cluster
> and am running only the gtm, coordinator and datanode. They are all hosted
> on the same machine and communicate using TCP sockets on localhost.
>
> TL;DR there is a stack trace at the end of the email.
>
> I've added some more debug logging to src/backend/storage/ipc/procarray.c
> 436:
>
>   elog(LOG, "failed to find proc %p in ProcArray", proc);
> +   elog(LOG, "pid %d", proc->pid);
> +   elog(LOG, "pgprocno %d", proc->pgprocno);
>
> The logs only seem to start happening after I connect using psql. Here's
> what they look like on the coordinator without any psql commands issued:
>
> SID 5300263f.163d LOG:  00000: database system was shut down at 2014-02-15
> 19:13:59 EST
> SID 5300263f.163d LOCATION:  StartupXLOG, xlog.c:4959
> SID 5300263f.163b LOG:  00000: database system is ready to accept
> connections
> SID 5300263f.163b LOCATION:  reaper, postmaster.c:2768
> SID 5300263f.1642 LOG:  00000: autovacuum launcher started
> SID 5300263f.1642 LOCATION:  AutoVacLauncherMain, autovacuum.c:417
> SID 5300276c.16c3 LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 5300276c.16c3 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 5300276c.16c3 LOG:  00000: pid 5827
> SID 5300276c.16c3 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 5300276c.16c3 LOG:  00000: pgprocno 102
> SID 5300276c.16c3 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 530027a8.16f9 LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 530027a8.16f9 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 530027a8.16f9 LOG:  00000: pid 5881
> SID 530027a8.16f9 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 530027a8.16f9 LOG:  00000: pgprocno 102
> SID 530027a8.16f9 LOCATION:  ProcArrayRemove, procarray.c:438
>
> Here is what the logs in the coordinator look like after I issued drop
> database test;
>
> SID 53002923.1764 LOG:  00000: statement: drop database test;
> SID 53002923.1764 LOCATION:  exec_simple_query, postgres.c:966
> SID 53002923.1764 LOG:  00000: failed to find proc 0x7f777d08d980 in
> ProcArray
> SID 53002923.1764 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 53002923.1764 STATEMENT:  drop database test;
> SID 53002923.1764 LOG:  00000: pid 0
> SID 53002923.1764 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 53002923.1764 STATEMENT:  drop database test;
> SID 53002923.1764 LOG:  00000: pgprocno 118
> SID 53002923.1764 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 53002923.1764 STATEMENT:  drop database test;
> SID 5300294c.176a LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 5300294c.176a LOCATION:  ProcArrayRemove, procarray.c:436
> SID 5300294c.176a LOG:  00000: pid 5994
> SID 5300294c.176a LOCATION:  ProcArrayRemove, procarray.c:437
> SID 5300294c.176a LOG:  00000: pgprocno 102
> SID 5300294c.176a LOCATION:  ProcArrayRemove, procarray.c:438
> SID 53002988.1784 LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 53002988.1784 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 53002988.1784 LOG:  00000: pid 6020
> SID 53002988.1784 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 53002988.1784 LOG:  00000: pgprocno 102
> SID 53002988.1784 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 530029c4.1791 LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 530029c4.1791 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 530029c4.1791 LOG:  00000: pid 6033
> SID 530029c4.1791 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 530029c4.1791 LOG:  00000: pgprocno 102
> SID 530029c4.1791 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 53002a00.17c6 LOG:  00000: failed to find proc 0x7f777d08ab80 in
> ProcArray
> SID 53002a00.17c6 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 53002a00.17c6 LOG:  00000: pid 6086
> SID 53002a00.17c6 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 53002a00.17c6 LOG:  00000: pgprocno 102
> SID 53002a00.17c6 LOCATION:  ProcArrayRemove, procarray.c:438
>
> At this point, these logs start appearing in the datanode's log:
>
> SID 53002927.1766 LOG:  00000: failed to find proc 0x7f46c031e980 in
> ProcArray
> SID 53002927.1766 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 53002927.1766 STATEMENT:  COMMIT PREPARED 'T10226'
> SID 53002927.1766 LOG:  00000: pid 0
> SID 53002927.1766 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 53002927.1766 STATEMENT:  COMMIT PREPARED 'T10226'
> SID 53002927.1766 LOG:  00000: pgprocno 118
> SID 53002927.1766 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 53002927.1766 STATEMENT:  COMMIT PREPARED 'T10226'
> SID 5300293d.1767 LOG:  00000: failed to find proc 0x7f46c031bb80 in
> ProcArray
> SID 5300293d.1767 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 5300293d.1767 LOG:  00000: pid 5991
> SID 5300293d.1767 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 5300293d.1767 LOG:  00000: pgprocno 102
> SID 5300293d.1767 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 53002979.1781 LOG:  00000: failed to find proc 0x7f46c031bb80 in
> ProcArray
> SID 53002979.1781 LOCATION:  ProcArrayRemove, procarray.c:436
> SID 53002979.1781 LOG:  00000: pid 6017
> SID 53002979.1781 LOCATION:  ProcArrayRemove, procarray.c:437
> SID 53002979.1781 LOG:  00000: pgprocno 102
> SID 53002979.1781 LOCATION:  ProcArrayRemove, procarray.c:438
> SID 530029b5.178e LOG:  00000: failed to find proc 0x7f46c031bb80 in
> ProcArray
> ...
>
> When I disconnect psql, the logs stop in the coordinator but continue in the
> datanode (there's an open connection between the coordinator and the
> datanode). After I set autovacuum = off, the logs with pgprocno 102
> disappeared in both the datanode and coordinator.
>
> I originally set out to get stack traces for both proc 118 and 102.
> Unfortunately, it seems that the autovacuum launcher spawns multiple
> processes making it difficult to intercept the process with gdb. I was able
> to get a stack trace for pgprocno 118 from the open connection between the
> datanode and coordinator:
>
> #0  ProcArrayRemove (proc=proc@entry=0x7ffcb25f9980,
> latestXid=latestXid@entry=10437) at procarray.c:436
> #1  0x00000000004b69ed in FinishPreparedTransaction (gid=<optimized out>,
> isCommit=<optimized out>) at twophase.c:1368
> #2  0x0000000000674551 in standard_ProcessUtility (parsetree=0x29d4110,
>     queryString=0x29d3730 "COMMIT PREPARED 'T10437'",
> context=PROCESS_UTILITY_TOPLEVEL, params=0x0,
>     dest=<optimized out>, sentToRemote=<optimized out>,
> completionTag=0x7fff2c308910 "") at utility.c:574
> #3  0x0000000000670d02 in PortalRunUtility (portal=portal@entry=0x29d99c0,
> utilityStmt=utilityStmt@entry=0x29d4110,
>     isTopLevel=isTopLevel@entry=1 '\001', dest=dest@entry=0x29d4450,
>     completionTag=completionTag@entry=0x7fff2c308910 "") at pquery.c:1285
> #4  0x00000000006718cd in PortalRunMulti (portal=portal@entry=0x29d99c0,
> isTopLevel=isTopLevel@entry=1 '\001',
>     dest=dest@entry=0x29d4450, altdest=altdest@entry=0x29d4450,
> completionTag=completionTag@entry=0x7fff2c308910 "")
>     at pquery.c:1432
> #5  0x00000000006723e9 in PortalRun (portal=portal@entry=0x29d99c0,
> count=count@entry=9223372036854775807,
>     isTopLevel=isTopLevel@entry=1 '\001', dest=dest@entry=0x29d4450,
> altdest=altdest@entry=0x29d4450,
>     completionTag=completionTag@entry=0x7fff2c308910 "") at pquery.c:882
> #6  0x00000000006702d2 in exec_simple_query (query_string=0x29d3730 "COMMIT
> PREPARED 'T10437'") at postgres.c:1140
> #7  PostgresMain (argc=<optimized out>, argv=argv@entry=0x29bb360,
> dbname=0x29bb288 "postgres",
>     username=<optimized out>) at postgres.c:4251
> #8  0x00000000004621c6 in BackendRun (port=0x29dd9a0) at postmaster.c:4205
> #9  BackendStartup (port=0x29dd9a0) at postmaster.c:3894
> #10 ServerLoop () at postmaster.c:1705
> #11 0x000000000062f768 in PostmasterMain (argc=argc@entry=4,
> argv=argv@entry=0x29b9340) at postmaster.c:1374
> #12 0x0000000000462b47 in main (argc=4, argv=0x29b9340) at main.c:196
>
> I'm not familiar with the codebase, so I'm not entirely sure what's
> happening. As far as I can tell, FinishPreparedTransaction is trying to end
> a two phase commit, which it cannot find. I did a select * from
> pg_prepared_xacts on both the coordinator and the datanode, but both tables
> were empty. The commits seem to complete successfully despite the log. I'll
> keep digging to see what I can find.
>
>
> On Sat, Feb 15, 2014 at 2:44 AM, Rishi Ramraj <the...@gm...>
> wrote:
>>
>> I just recompiled XC using the REL1_2_STABLE branch. I used pgxc_ctl to
>> configure and create the cluster. The physical configuration is still the
>> same, except now there's a GTM proxy on the machine as well.
>>
>> The database seems to be functioning correctly, but I'm still getting the
>> ProcArray error. Here's the log from the coordinator:
>>
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3cb0 XID 0LOG:  00000: database
>> system was shut down at 2014-02-15 02:23:33 EST
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3cb0 XID 0LOCATION:  StartupXLOG,
>> xlog.c:4959
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3ca5 XID 0LOG:  00000: database
>> system is ready to accept connections
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3ca5 XID 0LOCATION:  reaper,
>> postmaster.c:2768
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3cb5 XID 0LOG:  00000: autovacuum
>> launcher started
>> 2014-02-15 02:26:28 EST SID 52ff16a4.3cb5 XID 0LOCATION:
>> AutoVacLauncherMain, autovacuum.c:417
>> 2014-02-15 02:28:53 EST SID 52ff1727.3cd2 XID 0ERROR:  42601: syntax error
>> at or near "asdf" at character 1
>> 2014-02-15 02:28:53 EST SID 52ff1727.3cd2 XID 0LOCATION:  scanner_yyerror,
>> scan.l:1044
>> 2014-02-15 02:28:53 EST SID 52ff1727.3cd2 XID 0STATEMENT:  asdf
>> ;
>> 2014-02-15 02:29:30 EST SID 52ff1759.3cd3 XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:29:30 EST SID 52ff1759.3cd3 XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>> 2014-02-15 02:30:30 EST SID 52ff1795.3cd5 XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:30:30 EST SID 52ff1795.3cd5 XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>> 2014-02-15 02:31:01 EST SID 52ff1727.3cd2 XID 0LOG:  00000: statement:
>> create node data with (type='datanode', host='localhost', port=20008);
>> 2014-02-15 02:31:01 EST SID 52ff1727.3cd2 XID 0LOCATION:
>> exec_simple_query, postgres.c:966
>> 2014-02-15 02:31:08 EST SID 52ff1727.3cd2 XID 0ERROR:  42601: syntax error
>> at or near "exit" at character 1
>> 2014-02-15 02:31:08 EST SID 52ff1727.3cd2 XID 0LOCATION:  scanner_yyerror,
>> scan.l:1044
>> 2014-02-15 02:31:08 EST SID 52ff1727.3cd2 XID 0STATEMENT:  exit
>> ;
>> 2014-02-15 02:32:29 EST SID 52ff1808.3d1e XID 0LOG:  00000: statement:
>> create database test;
>> 2014-02-15 02:32:29 EST SID 52ff1808.3d1e XID 0LOCATION:
>> exec_simple_query, postgres.c:966
>> 2014-02-15 02:32:29 EST SID 52ff180d.3d20 XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:32:29 EST SID 52ff180d.3d20 XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>> 2014-02-15 02:32:31 EST SID 52ff1808.3d1e XID 2054LOG:  00000: failed to
>> find proc 0x7f3c12ff8980 in ProcArray
>> 2014-02-15 02:32:31 EST SID 52ff1808.3d1e XID 2054LOCATION:
>> ProcArrayRemove, procarray.c:436
>> 2014-02-15 02:32:31 EST SID 52ff1808.3d1e XID 2054STATEMENT:  create
>> database test;
>> 2014-02-15 02:33:30 EST SID 52ff1849.3e12 XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:33:30 EST SID 52ff1849.3e12 XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>> 2014-02-15 02:34:30 EST SID 52ff1885.3e2e XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:34:30 EST SID 52ff1885.3e2e XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>> 2014-02-15 02:35:30 EST SID 52ff18c1.3e30 XID 0LOG:  00000: failed to find
>> proc 0x7f3c12ff5b80 in ProcArray
>> 2014-02-15 02:35:30 EST SID 52ff18c1.3e30 XID 0LOCATION:  ProcArrayRemove,
>> procarray.c:436
>>
>> Here's the log from the datanode:
>>
>> LOG:  database system was shut down at 2014-02-15 02:23:35 EST
>> LOG:  database system is ready to accept connections
>> LOG:  autovacuum launcher started
>> ERROR:  PGXC Node data: object already defined
>> STATEMENT:  create node data with (type='coordinator', host='localhost',
>> port=20004);
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>> LOG:  failed to find proc 0x7f5a2cc58980 in ProcArray
>> STATEMENT:  COMMIT PREPARED 'T2051'
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>> LOG:  failed to find proc 0x7f5a2cc55b80 in ProcArray
>>
>> I've also attached the pgxcConf file I used to generate the cluster. I'm
>> going to try to run it under gdb and get a stack trace. Let me know if there
>> are any other diagnostics you would like me to run. That being said, the
>> database in its current condition is fine for my testing :)
>>
>>
>> On Thu, Feb 13, 2014 at 11:14 PM, Rishi Ramraj
>> <the...@gm...> wrote:
>>>
>>> Will give it a shot. Thanks for the help!
>>>
>>>
>>> On Thu, Feb 13, 2014 at 11:05 PM, Koichi Suzuki <koi...@gm...>
>>> wrote:
>>>>
>>>> I'm afraid something is wrong inside but sorry I couldn't locate what
>>>> it is.   Series of error message is supposed to be from COMMIT, ABORT,
>>>> COMMIT PREPARED or ABORT PREPARED.
>>>>
>>>> I'd advise to recreate the cluster with pgxc_ctl.    I hope this is
>>>> better to track what is going on.   You will find materials for this
>>>> from PGXC wiki page.   Try www.postgres-xc.org,
>>>>
>>>> Regards;
>>>> ---
>>>> Koichi Suzuki
>>>>
>>>>
>>>> 2014-02-14 12:57 GMT+09:00 Rishi Ramraj <the...@gm...>:
>>>> > Apparently I left the datanode running, and after a while the logs
>>>> > were
>>>> > filled with the following:
>>>> >
>>>> > FATAL:  sorry, too many clients already
>>>> >
>>>> > Here are the logs with the increased log levels. First on the
>>>> > coordinator:
>>>> >
>>>> > 2014-02-13 22:24:31 EST SID 52fd8c6f.386 XID 0LOG:  database system
>>>> > was shut
>>>> > down at 2014-02-13 22:07:11 EST
>>>> > 2014-02-13 22:24:31 EST SID 52fd8c6e.382 XID 0LOG:  database system is
>>>> > ready
>>>> > to accept connections
>>>> > 2014-02-13 22:24:31 EST SID 52fd8c6f.38b XID 0LOG:  autovacuum
>>>> > launcher
>>>> > started
>>>> > 2014-02-13 22:45:18 EST SID 52fd914a.59f XID 0LOG:  statement: drop
>>>> > database
>>>> > test;
>>>> > 2014-02-13 22:45:18 EST SID 52fd914a.59f XID 11571ERROR:  database
>>>> > "test"
>>>> > does not exist
>>>> > 2014-02-13 22:45:18 EST SID 52fd914a.59f XID 11571STATEMENT:  drop
>>>> > database
>>>> > test;
>>>> > 2014-02-13 22:45:32 EST SID 52fd915c.5a2 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:45:33 EST SID 52fd914a.59f XID 0LOG:  statement: create
>>>> > database test;
>>>> > 2014-02-13 22:45:35 EST SID 52fd914a.59f XID 11575LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297dbca60 in ProcArray
>>>> > 2014-02-13 22:45:35 EST SID 52fd914a.59f XID 11575STATEMENT:  create
>>>> > database test;
>>>> > 2014-02-13 22:46:32 EST SID 52fd9198.5b1 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:47:02 EST SID 52fd91b6.5b6 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:47:32 EST SID 52fd91d4.5bb XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:48:02 EST SID 52fd91f2.5be XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:48:32 EST SID 52fd9210.5c7 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:49:02 EST SID 52fd922e.5d1 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:49:32 EST SID 52fd924c.5d6 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:50:02 EST SID 52fd926a.5da XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:50:32 EST SID 52fd9288.5e0 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:51:02 EST SID 52fd92a6.5e3 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:51:32 EST SID 52fd92c4.5ee XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> > 2014-02-13 22:52:02 EST SID 52fd92e2.5f1 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f0297db9c60 in ProcArray
>>>> >
>>>> > Next, on the datanode:
>>>> >
>>>> > 2014-02-13 22:24:21 EST SID 52fd8c65.33a XID 0LOG:  database system
>>>> > was shut
>>>> > down at 2014-02-13 22:23:35 EST
>>>> > 2014-02-13 22:24:21 EST SID 52fd8c64.336 XID 0LOG:  database system is
>>>> > ready
>>>> > to accept connections
>>>> > 2014-02-13 22:24:21 EST SID 52fd8c65.33e XID 0LOG:  autovacuum
>>>> > launcher
>>>> > started
>>>> > 2014-02-13 22:45:34 EST SID 52fd915e.5a4 XID 0LOG:  statement: START
>>>> > TRANSACTION ISOLATION LEVEL read committed READ WRITE
>>>> > 2014-02-13 22:45:34 EST SID 52fd915e.5a4 XID 0LOG:  statement: create
>>>> > database test;
>>>> > 2014-02-13 22:45:35 EST SID 52fd915e.5a4 XID 11574LOG:  statement:
>>>> > PREPARE
>>>> > TRANSACTION 'T11574'
>>>> > 2014-02-13 22:45:35 EST SID 52fd915e.5a4 XID 0LOG:  statement: COMMIT
>>>> > PREPARED 'T11574'
>>>> > 2014-02-13 22:45:35 EST SID 52fd915e.5a4 XID 11575LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de46a60 in ProcArray
>>>> > 2014-02-13 22:45:35 EST SID 52fd915e.5a4 XID 11575STATEMENT:  COMMIT
>>>> > PREPARED 'T11574'
>>>> > 2014-02-13 22:46:22 EST SID 52fd918e.5ae XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:47:22 EST SID 52fd91ca.5b8 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:48:22 EST SID 52fd9206.5c2 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:49:22 EST SID 52fd9242.5d3 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:50:22 EST SID 52fd927e.5dc XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:51:22 EST SID 52fd92ba.5e5 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:52:22 EST SID 52fd92f6.5f5 XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:53:22 EST SID 52fd9332.5fd XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> > 2014-02-13 22:54:22 EST SID 52fd936e.60d XID 0LOG:  failed to find
>>>> > proc
>>>> > 0x7f1c3de43c60 in ProcArray
>>>> >
>>>> > Would you like me to keep experimenting on this installation, or
>>>> > should I
>>>> > recreate the cluster?
>>>> >
>>>> >
>>>> > On Thu, Feb 13, 2014 at 9:49 PM, Koichi Suzuki <koi...@gm...>
>>>> > wrote:
>>>> >>
>>>> >> I don't see anything strange in the configuration file.
>>>> >>
>>>> >> I found regression test sets up gtm port explicitly (value is the
>>>> >> default, 6666, though).     Could you try to configure your cluster
>>>> >> with pgxc_ctl, which is far simpler and you have much less chance to
>>>> >> encounter errors?   This utility comes with built-in command
>>>> >> sequences
>>>> >> needed to configure and operate your cluster.
>>>> >>
>>>> >> Or, could you re-run the same with different log_min_message level?
>>>> >>
>>>> >> Regards;
>>>> >> ---
>>>> >> Koichi Suzuki
>>>> >>
>>>> >>
>>>> >> 2014-02-14 11:16 GMT+09:00 Rishi Ramraj
>>>> >> <the...@gm...>:
>>>> >> > Missed the coordinator.conf file. Find attached.
>>>> >> >
>>>> >> >
>>>> >> > On Thu, Feb 13, 2014 at 9:15 PM, Rishi Ramraj
>>>> >> > <the...@gm...>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Find all of the files attached. I have the GTM, coordinator and
>>>> >> >> datanode
>>>> >> >> services all hosted on the same Linux Mint machine for testing.
>>>> >> >> They
>>>> >> >> communicate using the loopback interface.
>>>> >> >>
>>>> >> >> To initialize the cluster, I ran the following:
>>>> >> >>
>>>> >> >> /var/lib/postgres-xc/gtm $ initgtm -Z
>>>> >> >> /var/lib/postgres-xc/data $ initdb -D . --nodename data
>>>> >> >> /var/lib/postgres-xc/coord $ initdb -D . --nodename coord
>>>> >> >>
>>>> >> >> To start the cluster, I use upstart:
>>>> >> >>
>>>> >> >> /usr/local/pgsql/bin/gtm -D /var/lib/postgres-xc/gtm
>>>> >> >> /usr/local/pgsql/bin/postgres --datanode -D
>>>> >> >> /var/lib/postgres-xc/data
>>>> >> >> /usr/local/pgsql/bin/postgres --coordinator -D
>>>> >> >> /var/lib/postgres-xc/coord
>>>> >> >>
>>>> >> >> I will increase the log level and try to reproduce the errors.
>>>> >> >>
>>>> >> >>
>>>> >> >> On Thu, Feb 13, 2014 at 8:39 PM, 鈴木 幸市 <ko...@in...>
>>>> >> >> wrote:
>>>> >> >>>
>>>> >> >>> Hello;
>>>> >> >>>
>>>> >> >>> You need to set log_statement GUC to appropriate value.   Default
>>>> >> >>> is
>>>> >> >>> “none”.   “all” prints all statements accepted.    Also, it will
>>>> >> >>> be a
>>>> >> >>> good
>>>> >> >>> idea to include session ID in the log_line_prefix GUC.    Your
>>>> >> >>> postgresql.conf has some information how to set this.   I tested
>>>> >> >>> REL1_1_STABLE with four coordinators, four datanodes and four
>>>> >> >>> gtm_proxy and
>>>> >> >>> did not have any additional log for any coordinators/datanodes.
>>>> >> >>> It
>>>> >> >>> will
>>>> >> >>> be helpful what statements you issued.
>>>> >> >>>
>>>> >> >>> Here’s what I got in REL1_1_STABLE (I configured the cluster
>>>> >> >>> using
>>>> >> >>> pgxc_ctl).
>>>> >> >>>
>>>> >> >>> Coordinator:
>>>> >> >>> LOG:  database system was shut down at 2014-02-14 10:2sd
>>>> >> >>> LOG:  autovacuum launcher started
>>>> >> >>>
>>>> >> >>> Datanode:
>>>> >> >>> LOG:  database system was shut down at 2014-02-14 10:21:08 JST
>>>> >> >>> LOG:  database system is ready to accept connections
>>>> >> >>> LOG:  autovacuum launcher started
>>>> >> >>>
>>>> >> >>> I’d like to have your configuration, each postgresql.conf file
>>>> >> >>> and
>>>> >> >>> what
>>>> >> >>> you’ve done to initialize, start and run your application.
>>>> >> >>>
>>>> >> >>> Best;
>>>> >> >>> ---
>>>> >> >>> Koichi Suzuki
>>>> >> >>>
>>>> >> >>> 2014/02/14 0:38、Rishi Ramraj <the...@gm...> のメール：
>>>> >> >>>
>>>> >> >>> I just tried with REL1_1_STABLE. Here are my logs from the
>>>> >> >>> coordinator:
>>>> >> >>>
>>>> >> >>> LOG:  database system was shut down at 2014-02-13 10:14:16 EST
>>>> >> >>> LOG:  database system is ready to accept connections
>>>> >> >>> LOG:  autovacuum launcher started
>>>> >> >>> ERROR:  syntax error at or near "asdf" at character 1
>>>> >> >>> STATEMENT:  asdf;
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79ea60 in ProcArray
>>>> >> >>> STATEMENT:  create database test;
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79bc60 in ProcArray
>>>> >> >>> ERROR:  cannot drop the currently open database
>>>> >> >>> STATEMENT:  drop database test;
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79ea60 in ProcArray
>>>> >> >>> STATEMENT:  drop database test;
>>>> >> >>> ERROR:  syntax error at or near "blah" at character 1
>>>> >> >>> STATEMENT:  blah blah blah;
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f7d4a79bc60 in ProcArray
>>>> >> >>>
>>>> >> >>> The logs from the datanode:
>>>> >> >>>
>>>> >> >>> LOG:  database system was shut down at 2014-02-13 10:19:33 EST
>>>> >> >>> LOG:  database system is ready to accept connections
>>>> >> >>> LOG:  autovacuum launcher started
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5ea60 in ProcArray
>>>> >> >>> STATEMENT:  COMMIT PREPARED 'T10014'
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5ea60 in ProcArray
>>>> >> >>> STATEMENT:  COMMIT PREPARED 'T10023'
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>> LOG:  failed to find proc 0x7f06bee5bc60 in ProcArray
>>>> >> >>>
>>>> >> >>> The datanode seems to continuously produce logs but the
>>>> >> >>> coordinator
>>>> >> >>> does
>>>> >> >>> not. I issued CREATE NODE commands to both services but they
>>>> >> >>> don't
>>>> >> >>> seem to
>>>> >> >>> show up in the log.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> On Thu, Feb 13, 2014 at 9:49 AM, Rishi Ramraj
>>>> >> >>> <the...@gm...> wrote:
>>>> >> >>>>
>>>> >> >>>> I haven't been issuing commits or aborts every 30 seconds. So
>>>> >> >>>> far,
>>>> >> >>>> I've
>>>> >> >>>> only issued five commands to the cluster using psql, but I have
>>>> >> >>>> over
>>>> >> >>>> 100
>>>> >> >>>> logs. I will try 1.1 today and let you know.
>>>> >> >>>>
>>>> >> >>>> Do you use a specific branching methodology, like gitflow? Do
>>>> >> >>>> you
>>>> >> >>>> have a
>>>> >> >>>> bug tracking system? If you need any help with release
>>>> >> >>>> engineering,
>>>> >> >>>> let me
>>>> >> >>>> know; I don't mind volunteering some time.
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> On Thu, Feb 13, 2014 at 1:41 AM, 鈴木 幸市
>>>> >> >>>> <ko...@in...>
>>>> >> >>>> wrote:
>>>> >> >>>>>
>>>> >> >>>>> GTM message is just a report.   When GTM starts, it tries to
>>>> >> >>>>> read if
>>>> >> >>>>> there is any slave connected in previous run and it didn’t find
>>>> >> >>>>> one.
>>>> >> >>>>>
>>>> >> >>>>> The first message looks not harmful but could be some potential
>>>> >> >>>>> issues.
>>>> >> >>>>> Please let me look into it.   The chance to have this message
>>>> >> >>>>> is at
>>>> >> >>>>> the
>>>> >> >>>>> initialization of each datanode/coordinator, COMMIT, ABORT,
>>>> >> >>>>> COMMIT
>>>> >> >>>>> PREPARED
>>>> >> >>>>> or ABORT PREPARED.
>>>> >> >>>>>
>>>> >> >>>>> Do you have a chance to issue them every 30 seconds?
>>>> >> >>>>>
>>>> >> >>>>> If possible, could you try release 1.1 and see if you have the
>>>> >> >>>>> same
>>>> >> >>>>> issue (first message)?  I think release 1.1 is better because
>>>> >> >>>>> master
>>>> >> >>>>> is
>>>> >> >>>>> anyway development branch.
>>>> >> >>>>>
>>>> >> >>>>> Best;
>>>> >> >>>>> ---
>>>> >> >>>>> Koichi Suzuki
>>>> >> >>>>>
>>>> >> >>>>> 2014/02/13 14:51、Rishi Ramraj <the...@gm...>
>>>> >> >>>>> のメール：
>>>> >> >>>>>
>>>> >> >>>>> > Hello All,
>>>> >> >>>>> >
>>>> >> >>>>> > I just installed postgres-xc from the git master branch on a
>>>> >> >>>>> > test
>>>> >> >>>>> > machine. All processes are running on the same box. On both
>>>> >> >>>>> > the
>>>> >> >>>>> > coordinator
>>>> >> >>>>> > and data processes, I'm getting the following logs about
>>>> >> >>>>> > every 30
>>>> >> >>>>> > seconds:
>>>> >> >>>>> >
>>>> >> >>>>> > LOG:  failed to find proc 0x7fd9ee703f80 in ProcArray
>>>> >> >>>>> >
>>>> >> >>>>> > On the GTM process, I'm getting the following logs at about
>>>> >> >>>>> > the
>>>> >> >>>>> > same
>>>> >> >>>>> > frequency:
>>>> >> >>>>> >
>>>> >> >>>>> > LOG:  Any GTM standby node not found in registered node(s).
>>>> >> >>>>> > LOCATION:  gtm_standby_connect_to_standby_int,
>>>> >> >>>>> > gtm_standby.c:381
>>>> >> >>>>> >
>>>> >> >>>>> > The cluster seems to be working properly. I was able to
>>>> >> >>>>> > create a
>>>> >> >>>>> > new
>>>> >> >>>>> > database and a table within that database without any
>>>> >> >>>>> > problem. I
>>>> >> >>>>> > restarted
>>>> >> >>>>> > all services and the data was persisted, but the logs
>>>> >> >>>>> > persist. Any
>>>> >> >>>>> > idea
>>>> >> >>>>> > what's causing these logs?
>>>> >> >>>>> >
>>>> >> >>>>> > Thanks,
>>>> >> >>>>> > - Rishi
>>>> >> >>>>> >
>>>> >> >>>>> >
>>>> >> >>>>> >
>>>> >> >>>>> > ------------------------------------------------------------------------------
>>>> >> >>>>> > Android apps run on BlackBerry 10
>>>> >> >>>>> > Introducing the new BlackBerry 10.2.1 Runtime for Android
>>>> >> >>>>> > apps.
>>>> >> >>>>> > Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>> >> >>>>> > Get your Android app in front of a whole new audience.  Start
>>>> >> >>>>> > now.
>>>> >> >>>>> >
>>>> >> >>>>> >
>>>> >> >>>>> >
>>>> >> >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________
>>>> >> >>>>> > Postgres-xc-general mailing list
>>>> >> >>>>> > Pos...@li...
>>>> >> >>>>> >
>>>> >> >>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>> >> >>>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> --
>>>> >> >>>> Cheers,
>>>> >> >>>> - Rishi
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> --
>>>> >> >>> Cheers,
>>>> >> >>> - Rishi
>>>> >> >>>
>>>> >> >>>
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> --
>>>> >> >> Cheers,
>>>> >> >> - Rishi
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > Cheers,
>>>> >> > - Rishi
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > ------------------------------------------------------------------------------
>>>> >> > Android apps run on BlackBerry 10
>>>> >> > Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>> >> > Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>> >> > Get your Android app in front of a whole new audience.  Start now.
>>>> >> >
>>>> >> >
>>>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>> >> > _______________________________________________
>>>> >> > Postgres-xc-general mailing list
>>>> >> > Pos...@li...
>>>> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>> >> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Cheers,
>>>> > - Rishi
>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> - Rishi
>>
>>
>>
>>
>> --
>> Cheers,
>> - Rishi
>
>
>
>
> --
> Cheers,
> - Rishi

On 22 February 2014 01:38, Sergio Sinuco
<ser...@da...> wrote:
> I forgot to say that when the cluster start after crash, sequences of tables
> advance two thousand units. For example, before kill gtm process, sequence
> was in 4. After restart,  sequence's value of first insertion was in 2005.

It's a feature.
GTM doesn't write sequence values into the disk at each assignment for
performance. GTM ensures the sequence value will unique but doesn't
ensure it will continuous. It could skip 2000 at maximum.

>
>
> 2014-02-21 11:32 GMT-05:00 Sergio Sinuco <ser...@da...>:
>
>> Hi. Thanks for reply. Effectively I applied the patches to 1.1 version and
>> It worked. However I had to remove register.node files in GTM and GTM Proxy
>> too because if I didn't do it then the cluster didn't work.
>>
>> Now I will try 1.1 version with these patches and postgis patch in order
>> to find if they are compatible. I will let you know my results.
>>
>> I will try 1.2 version too.

The patch Mr.Suzuki suggested is not for the issue I told.

Regards.

>>
>> Thanks.
>>
>>
>> 2014-02-19 21:15 GMT-05:00 Masataka Saito <pg...@gm...>:
>>
>>> I'm afraid you got this issue. Can you check logs in GTM and GTM proxy?
>>> https://sourceforge.net/p/postgres-xc/bugs/474/
>>>
>>> If it prove right, please take the following steps.
>>> 1. Stop all component
>>> 2. Remove register.node in GTM and GTM Proxy
>>> 3. Start GTM
>>> 4. Start GTM proxy
>>> 5. Start coordinators and datanodes
>>>
>>> Following materials will be help you to configure.
>>> * Postgres Open 2013 (Especially SPOF Analysis in Slide deck)
>>>
>>> http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=Presentation_Materials
>>>
>>> And the configuration topics described in these materials are
>>> deprecated yet, but very helpful to know what pgxc_ctl does.
>>> * PGDay.EU, Oct 24th, 2012, Prague, Czeck
>>>
>>> http://postgres-xc.sourceforge.net/misc-docs/Prague_Presentation_20121024.pdf
>>> * Presentation at PGConfChina 2012, Jun16th, 2012, Beijing, China (I
>>> believe this is the best material for geeks)
>>>
>>> http://postgres-xc.sourceforge.net/misc-docs/PGConfChina_20120610_Distribution.pdf
>>>
>>> Regards.
>>>
>>> On 20 February 2014 02:03, Sergio Sinuco
>>> <ser...@da...> wrote:
>>> > Hello everybody.
>>> >
>>> > I just begin to try postgrexc 1.1. I have followed configuration
>>> > tutorial
>>> > for real servers in postgresxc.wikia.com
>>> >
>>> > http://postgresxc.wikia.com/wiki/Real_Server_configuration
>>> >
>>> > I setup three nodes. The first one with a GTM. The second and third
>>> > with a
>>> > GTM Proxy, a datanode and a coordinator in each one.
>>> >
>>> > Everything is OK, until I kill the GTM process in first node. After
>>> > restart
>>> > everything in all nodes I can't insert new data in tables. When I try
>>> > it,
>>> > following message is showed:
>>> >
>>> > ERROR:
>>> > STATEMENT:  INSERT INTO public.test(name) VALUES ('CARLOS'::text)
>>> >
>>> > Then, I can't connect to coordinators and following message is showed:
>>> >
>>> > ERROR:  GTM error, could not obtain snapshot XID = 0
>>> > WARNING:  can not connect to GTM: Connection refused
>>> >
>>> > I know that there is a way to configure Postgres-XC in order to have
>>> > high
>>> > availability (GTM Stand By and other stuff), but the question is, when
>>> > GTM
>>> > process and even its contingency crash there is no way to get a right
>>> > behaviour of the cluster?
>>> >
>>> > Thanks.
>>> >
>>> > --
>>> > Sergio E. Sinuco Leon
>>> > Arquitecto de soluciones
>>> > Datatraffic S.A.S.
>>> > Móvil: (57) 310 884 26 50
>>> > Fijo (+571) 7426160 Ext 115
>>> > Calle 93 # 15-27 Ofc. 502
>>> > Calle 29 # 6 - 94 Ofc. 601
>>> > Bogotá, Colombia.
>>> > www.datatraffic.com.co
>>> >
>>> >
>>> > ------------------------------------------------------------------------------
>>> > Managing the Performance of Cloud-Based Applications
>>> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>> > Read the Whitepaper.
>>> >
>>> > http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>> > _______________________________________________
>>> > Postgres-xc-general mailing list
>>> > Pos...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>> >
>>
>>
>>
>>
>> --
>> Sergio E. Sinuco Leon
>> Arquitecto de soluciones
>> Datatraffic S.A.S.
>> Móvil: (57) 310 884 26 50
>> Fijo (+571) 7426160 Ext 115
>> Calle 93 # 15-27 Ofc. 502
>> Calle 29 # 6 - 94 Ofc. 601
>> Bogotá, Colombia.
>> www.datatraffic.com.co
>
>
>
>
> --
> Sergio E. Sinuco Leon
> Arquitecto de soluciones
> Datatraffic S.A.S.
> Móvil: (57) 310 884 26 50
> Fijo (+571) 7426160 Ext 115
> Calle 93 # 15-27 Ofc. 502
> Calle 29 # 6 - 94 Ofc. 601
> Bogotá, Colombia.
> www.datatraffic.com.co

Dear Juned,
A Postgres-XC cluster needs one GTM. You do not need a GTM at each state.
If you need to have data replicated across all the nodes all you need to do
is to distribute all tables of the schema by replication using the syntax
create table tab(a int, b int) distribute by replication;
Having said that, you need to consider the following points
a) It seems that the nodes of the cluster would be geographically apart, if
yes, please keep in mind that Postgres-XC does not cater for network
failures. If the link to one of the cluster nodes is down while inserting
data, the insert would fail. A select would however pass depending on which
coordinator the application is connected to.
b) Normally you would not replicate all the data to all the nodes, you
should use a mix of replicated and distributed tables. We can guide better
if you can explain a little bit what application you are trying to build
with Postgres-XC as backend.
Regards



On Sat, Feb 22, 2014 at 6:15 PM, Juned Khan <jkh...@gm...> wrote:

> Hi All,
>
> PFA
>
> Let me explain you my exact requirement. I have attached one image for
> that.
>
> As per that image i have multiple states which has GTM, datanodes and
> coordinators.
> In future i will add more states. so how data will be replicated with the
> datanodes of different stat.
>
> can i configure central GTM server which can handle all this stuff., or
> how do i configure each
> GTM to communicate with other GTMs so all DBs can be replicated
>
> I just want to know is that possible using postgre-xc or not ? if yes then
> please provide me relevant link and documentation for that.
>
>
> --
> Thanks,
> Juned Khan
> iNextrix Technologies Pvt Ltd.
> www.inextrix.com
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
-- 
*Abbas*
 Architect

Ph: 92.334.5100153
 Skype ID: gabbasb
www.enterprisedb.co
<http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>

*Follow us on Twitter*
@EnterpriseDB

Visit EnterpriseDB for tutorials, webinars,
whitepapers<http://www.enterprisedb.com/resources-community>and
more<http://www.enterprisedb.com/resources-community>

I forgot to say that when the cluster start after crash, sequences of
tables advance two thousand units. For example, before kill gtm process,
sequence was in 4. After restart,  sequence's value of first insertion was
in 2005.


2014-02-21 11:32 GMT-05:00 Sergio Sinuco <ser...@da...>:

> Hi. Thanks for reply. Effectively I applied the patches to 1.1 version and
> It worked. However I had to remove register.node files in GTM and GTM Proxy
> too because if I didn't do it then the cluster didn't work.
>
> Now I will try 1.1 version with these patches and postgis patch in order
> to find if they are compatible. I will let you know my results.
>
> I will try 1.2 version too.
>
> Thanks.
>
>
> 2014-02-19 21:15 GMT-05:00 Masataka Saito <pg...@gm...>:
>
> I'm afraid you got this issue. Can you check logs in GTM and GTM proxy?
>> https://sourceforge.net/p/postgres-xc/bugs/474/
>>
>> If it prove right, please take the following steps.
>> 1. Stop all component
>> 2. Remove register.node in GTM and GTM Proxy
>> 3. Start GTM
>> 4. Start GTM proxy
>> 5. Start coordinators and datanodes
>>
>> Following materials will be help you to configure.
>> * Postgres Open 2013 (Especially SPOF Analysis in Slide deck)
>>
>> http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=Presentation_Materials
>>
>> And the configuration topics described in these materials are
>> deprecated yet, but very helpful to know what pgxc_ctl does.
>> * PGDay.EU, Oct 24th, 2012, Prague, Czeck
>>
>> http://postgres-xc.sourceforge.net/misc-docs/Prague_Presentation_20121024.pdf
>> * Presentation at PGConfChina 2012, Jun16th, 2012, Beijing, China (I
>> believe this is the best material for geeks)
>>
>> http://postgres-xc.sourceforge.net/misc-docs/PGConfChina_20120610_Distribution.pdf
>>
>> Regards.
>>
>> On 20 February 2014 02:03, Sergio Sinuco
>> <ser...@da...> wrote:
>> > Hello everybody.
>> >
>> > I just begin to try postgrexc 1.1. I have followed configuration
>> tutorial
>> > for real servers in postgresxc.wikia.com
>> >
>> > http://postgresxc.wikia.com/wiki/Real_Server_configuration
>> >
>> > I setup three nodes. The first one with a GTM. The second and third
>> with a
>> > GTM Proxy, a datanode and a coordinator in each one.
>> >
>> > Everything is OK, until I kill the GTM process in first node. After
>> restart
>> > everything in all nodes I can't insert new data in tables. When I try
>> it,
>> > following message is showed:
>> >
>> > ERROR:
>> > STATEMENT:  INSERT INTO public.test(name) VALUES ('CARLOS'::text)
>> >
>> > Then, I can't connect to coordinators and following message is showed:
>> >
>> > ERROR:  GTM error, could not obtain snapshot XID = 0
>> > WARNING:  can not connect to GTM: Connection refused
>> >
>> > I know that there is a way to configure Postgres-XC in order to have
>> high
>> > availability (GTM Stand By and other stuff), but the question is, when
>> GTM
>> > process and even its contingency crash there is no way to get a right
>> > behaviour of the cluster?
>> >
>> > Thanks.
>> >
>> > --
>> > Sergio E. Sinuco Leon
>> > Arquitecto de soluciones
>> > Datatraffic S.A.S.
>> > Móvil: (57) 310 884 26 50
>> > Fijo (+571) 7426160 Ext 115
>> > Calle 93 # 15-27 Ofc. 502
>> > Calle 29 # 6 - 94 Ofc. 601
>> > Bogotá, Colombia.
>> > www.datatraffic.com.co
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Managing the Performance of Cloud-Based Applications
>> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> > Read the Whitepaper.
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>> >
>>
>
>
>
> --
> Sergio E. Sinuco Leon
> Arquitecto de soluciones
> Datatraffic S.A.S.
> Móvil: (57) 310 884 26 50
> Fijo (+571) 7426160 Ext 115
> Calle 93 # 15-27 Ofc. 502
> Calle 29 # 6 - 94 Ofc. 601
> Bogotá, Colombia.
> www.datatraffic.com.co
>



-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co

Hi. Thanks for reply. Effectively I applied the patches to 1.1 version and
It worked. However I had to remove register.node files in GTM and GTM Proxy
too because if I didn't do it then the cluster didn't work.

Now I will try 1.1 version with these patches and postgis patch in order to
find if they are compatible. I will let you know my results.

I will try 1.2 version too.

Thanks.


2014-02-19 21:15 GMT-05:00 Masataka Saito <pg...@gm...>:

> I'm afraid you got this issue. Can you check logs in GTM and GTM proxy?
> https://sourceforge.net/p/postgres-xc/bugs/474/
>
> If it prove right, please take the following steps.
> 1. Stop all component
> 2. Remove register.node in GTM and GTM Proxy
> 3. Start GTM
> 4. Start GTM proxy
> 5. Start coordinators and datanodes
>
> Following materials will be help you to configure.
> * Postgres Open 2013 (Especially SPOF Analysis in Slide deck)
>
> http://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=Presentation_Materials
>
> And the configuration topics described in these materials are
> deprecated yet, but very helpful to know what pgxc_ctl does.
> * PGDay.EU, Oct 24th, 2012, Prague, Czeck
>
> http://postgres-xc.sourceforge.net/misc-docs/Prague_Presentation_20121024.pdf
> * Presentation at PGConfChina 2012, Jun16th, 2012, Beijing, China (I
> believe this is the best material for geeks)
>
> http://postgres-xc.sourceforge.net/misc-docs/PGConfChina_20120610_Distribution.pdf
>
> Regards.
>
> On 20 February 2014 02:03, Sergio Sinuco
> <ser...@da...> wrote:
> > Hello everybody.
> >
> > I just begin to try postgrexc 1.1. I have followed configuration tutorial
> > for real servers in postgresxc.wikia.com
> >
> > http://postgresxc.wikia.com/wiki/Real_Server_configuration
> >
> > I setup three nodes. The first one with a GTM. The second and third with
> a
> > GTM Proxy, a datanode and a coordinator in each one.
> >
> > Everything is OK, until I kill the GTM process in first node. After
> restart
> > everything in all nodes I can't insert new data in tables. When I try it,
> > following message is showed:
> >
> > ERROR:
> > STATEMENT:  INSERT INTO public.test(name) VALUES ('CARLOS'::text)
> >
> > Then, I can't connect to coordinators and following message is showed:
> >
> > ERROR:  GTM error, could not obtain snapshot XID = 0
> > WARNING:  can not connect to GTM: Connection refused
> >
> > I know that there is a way to configure Postgres-XC in order to have high
> > availability (GTM Stand By and other stuff), but the question is, when
> GTM
> > process and even its contingency crash there is no way to get a right
> > behaviour of the cluster?
> >
> > Thanks.
> >
> > --
> > Sergio E. Sinuco Leon
> > Arquitecto de soluciones
> > Datatraffic S.A.S.
> > Móvil: (57) 310 884 26 50
> > Fijo (+571) 7426160 Ext 115
> > Calle 93 # 15-27 Ofc. 502
> > Calle 29 # 6 - 94 Ofc. 601
> > Bogotá, Colombia.
> > www.datatraffic.com.co
> >
> >
> ------------------------------------------------------------------------------
> > Managing the Performance of Cloud-Based Applications
> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> > Read the Whitepaper.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>



-- 
Sergio E. Sinuco Leon
Arquitecto de soluciones
Datatraffic S.A.S.
Móvil: (57) 310 884 26 50
Fijo (+571) 7426160 Ext 115
Calle 93 # 15-27 Ofc. 502
Calle 29 # 6 - 94 Ofc. 601
Bogotá, Colombia.
www.datatraffic.com.co