postgres-xc-general Mailing List for Postgres-XC

Hello,

----- Original Message ----- 
From: "Koichi Suzuki" <koi...@gm...>
> If table is distributed, statements will be decomposed to smaller
> pieces to run at each node and they will be executed in parallel.
> Now our evaluation is more concentrated on running as many (small)
> transactions in parallel, we've not evaluated parallel query
> performance yet and there might be something we need to improve.
>
> Unfortunately, foreign tables are not supported yet.

Great. I wish parallel queries will be able to handle foreign tables. If 
multiple Postgres-XC nodes can scan different parts of a large file on 
Hadoop through a foreign table, it will be a better tool than Hive.

Regards
MauMau

HI,

If table is distributed, statements will be decomposed to smaller
pieces to run at each node and they will be executed in parallel.
Now our evaluation is more concentrated on running as many (small)
transactions in parallel, we've not evaluated parallel query
performance yet and there might be something we need to improve.

Unfortunately, foreign tables are not supported yet.

Regards;
----------
Koichi Suzuki



2011/12/30 MauMau <mau...@gm...>:
> Hello,
>
> I'm very interested in the evolution of Postgres-XC as a data warehouse
> infrastructure. Thank you for your great job. Please let me ask you some
> questions.
>
> 1.What kind of SELECT statements does the current Postgres-XC process in
> parallel on multiple nodes? I mean I'd like to know if a single query can be
> broken into pieces and processed in parallel on different nodes. To be more
> concrete, does the following simple query become faster as the number of
> nodes increases?
>
> SELECT SUM(some_column) FROM some_table [WHERE ...];
>
> 2.Does the foreign table feature of PostgreSQL 9.1 work with Postgres-XC? If
> it doesn't currently, do you have any plan?
>
> Regards
> MauMau
>
>
>
>
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual
> desktops for less than the cost of PCs and save 60% on VDI infrastructure
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hello,

I'm very interested in the evolution of Postgres-XC as a data warehouse 
infrastructure. Thank you for your great job. Please let me ask you some 
questions.

1.What kind of SELECT statements does the current Postgres-XC process in 
parallel on multiple nodes? I mean I'd like to know if a single query can be 
broken into pieces and processed in parallel on different nodes. To be more 
concrete, does the following simple query become faster as the number of 
nodes increases?

SELECT SUM(some_column) FROM some_table [WHERE ...];

2.Does the foreign table feature of PostgreSQL 9.1 work with Postgres-XC? If 
it doesn't currently, do you have any plan?

Regards
MauMau

On Thu, Dec 22, 2011 at 7:14 PM, peppux <pe...@gm...> wrote:

> Hello guys,
> i've two remote servers with postgres.
>
> I'd like to configure them so that when changing one, automatically
> changes the other database.
> I think that i need to configure them as sync multimaster.
>
> Someone can help me please?
> Thanks in advice.
>
As said before, this is an extremely difficult implementation, and you have
to think about a lot of corner cases.
Regarding your application, I would define your application as a 1-layer
multimaster.
Each node is a master, contains the data, and can run independently in case
it is unique.
The main difference we have in Postgres-XC is that we use a 2-layer master
with datanode and Coordinator.
On the top of that, XC uses GTM to feed global transaction ID, snapshot and
sequence values to other nodes.

In the case of 1 layer though, I think you may be able to implement
transaction ID feed, snapshot and snapshot value directly from one of the
nodes, by using a new type of mechanism, what has not been designed yet
though. To give you an idea if you wish to work on that, multimaster
replication will need to have a consistent mechanism to send WAL among all
the nodes to keep data consistency...
But that's pretty complicated.
-- 
Michael Paquier
http://michael.otacoo.com

Multimaster synchronized replication is very very difficult to
implement.   To my knowledge, Postgres-R is paying effort for this.
I don't know if it is in production and its performance.
----------
Koichi Suzuki



2011/12/22 peppux <pe...@gm...>:
> Hello guys,
> i've two remote servers with postgres.
>
> I'd like to configure them so that when changing one, automatically
> changes the other database.
> I think that i need to configure them as sync multimaster.
>
> Someone can help me please?
> Thanks in advice.
>
> Bye
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hello guys,
i've two remote servers with postgres.

I'd like to configure them so that when changing one, automatically
changes the other database.
I think that i need to configure them as sync multimaster.

Someone can help me please?
Thanks in advice.

Bye

Good to know.
Kind regards,
-- 
Michael Paquier
http://michael.otacoo.com

Hi,
Thank's Michael, with your informations i can compile Postgres-XC.

Regards

-- 
Magorn

Hi,

This issue is related to pthread library referencing in GTM.
There is already a big report regarding that:
https://sourceforge.net/tracker/?func=detail&aid=3425212&group_id=311227&atid=1310232
.
It is one of the issues we have to get ride off before having a version 1.0
released.

Regards,

On Mon, Dec 19, 2011 at 11:50 PM, Magorn <ma...@gm...> wrote:

> Hi,
> I have problem when i want toi compile 0.9.6 version. I have this error :
>
> gcc  -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
> -fwrapv -L../../common -L../../libpq -lpthread main.o gtm_thread.o
> gtm_txn.o gtm_seq.o gtm_snap.o gtm_time.o gtm_standby.o
> ../libpq/libpqcomm.a ../path/libgtmpath.a ../recovery/libgtmrecovery.a
> ../client/libgtmclient.a ../common/libgtm.a -o gtm
> main.o: In function `MainThreadInit':
> main.c:(.text+0x13): undefined reference to `pthread_key_create'
> main.c:(.text+0x9a): undefined reference to `pthread_setspecific'
> main.o: In function `BaseInit':
> main.c:(.text+0x11f): undefined reference to `pthread_getspecific'
> main.o: In function `GTM_ThreadMain':
> main.c:(.text+0x1157): undefined reference to `pthread_getspecific'
> main.c:(.text+0x1167): undefined reference to `pthread_getspecific'
> main.c:(.text+0x13b2): undefined reference to `pthread_getspecific'
> main.c:(.text+0x13d0): undefined reference to `pthread_getspecific'
> main.o:main.c:(.text+0x13e8): more undefined references to
> `pthread_getspecific' follow
> gtm_thread.o: In function `GTM_ThreadCreate':
> gtm_thread.c:(.text+0x465): undefined reference to `pthread_create'
> gtm_thread.o: In function `GTM_ThreadJoin':
> gtm_thread.c:(.text+0x4e6): undefined reference to `pthread_join'
> gtm_thread.o: In function `GTM_ThreadCleanup':
> gtm_thread.c:(.text+0x682): undefined reference to `pthread_setspecific'
> gtm_thread.o: In function `GTM_ThreadMainWrapper':
> gtm_thread.c:(.text+0x6b0): undefined reference to `pthread_detach'
> gtm_thread.c:(.text+0x6c4): undefined reference to `pthread_setspecific'
> gtm_thread.c:(.text+0x6d1): undefined reference to `pthread_getspecific'
> gtm_thread.c:(.text+0x739): undefined reference to
> `__pthread_register_cancel'
> gtm_thread.c:(.text+0x759): undefined reference to
> `__pthread_unregister_cancel'
> gtm_txn.o: In function `GTM_GetGIDData':
> gtm_txn.c:(.text+0x170a): undefined reference to `pthread_getspecific'
> gtm_txn.c:(.text+0x177d): undefined reference to `pthread_getspecific'
> gtm_txn.o: In function `ProcessBeginTransactionCommand':
> gtm_txn.c:(.text+0x1865): undefined reference to `pthread_getspecific'
> gtm_txn.o: In function `ProcessBeginTransactionGetGXIDCommand':
> gtm_txn.c:(.text+0x19e6): undefined reference to `pthread_getspecific'
> gtm_txn.c:(.text+0x1bb4): undefined reference to `pthread_getspecific'
> gtm_txn.o:gtm_txn.c:(.text+0x1bd5): more undefined references to
> `pthread_getspecific' follow
> ../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockAcquire':
> gtm_lock.c:(.text+0x25): undefined reference to `pthread_rwlock_wrlock'
> gtm_lock.c:(.text+0x36): undefined reference to `pthread_rwlock_rdlock'
> ../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockRelease':
> gtm_lock.c:(.text+0x8b): undefined reference to `pthread_rwlock_unlock'
> ../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockInit':
> gtm_lock.c:(.text+0xb4): undefined reference to `pthread_rwlock_init'
> ../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockDestroy':
> gtm_lock.c:(.text+0xce): undefined reference to `pthread_rwlock_destroy'
> ../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockConditionalAcquire':
> gtm_lock.c:(.text+0xf9): undefined reference to `pthread_rwlock_trywrlock'
> gtm_lock.c:(.text+0x10a): undefined reference to `pthread_rwlock_tryrdlock'
> ../common/libgtm.a(gtm_lock.o): In function
> `GTM_MutexLockConditionalAcquire':
> gtm_lock.c:(.text+0x1d6): undefined reference to `pthread_mutex_trylock'
> ../common/libgtm.a(gtm_list.o): In function `new_list':
> gtm_list.c:(.text+0x52): undefined reference to `pthread_getspecific'
> gtm_list.c:(.text+0x83): undefined reference to `pthread_getspecific'
> ../common/libgtm.a(gtm_list.o): In function `new_head_cell':
> gtm_list.c:(.text+0xdc): undefined reference to `pthread_getspecific'
> ../common/libgtm.a(gtm_list.o): In function `new_tail_cell':
> gtm_list.c:(.text+0x13a): undefined reference to `pthread_getspecific'
> ../common/libgtm.a(gtm_list.o): In function `add_new_cell':
> gtm_list.c:(.text+0x1f0): undefined reference to `pthread_getspecific'
> ../common/libgtm.a(gtm_list.o):gtm_list.c:(.text+0xc0e): more
> undefined references to `pthread_getspecific' follow
> collect2: ld returned 1 exit status
> make[3]: *** [gtm] Error 1
> make[3]: Leaving directory `/home/pgxc/pgxc-0.9.6/src/gtm/main'
> make[2]: *** [all] Error 2
> make[2]: Leaving directory `/home/pgxc/pgxc-0.9.6/src/gtm'
> make[1]: *** [all-gtm-recurse] Error 2
> make[1]: Leaving directory `/home/pgxc/pgxc-0.9.6/src'
> make: *** [all-src-recurse] Error 2
>
> I use ubuntu server 11.10 64 bits.
> Make version : GNU Make 3.81
>


-- 
Michael Paquier
http://michael.otacoo.com

Hi,
I have problem when i want toi compile 0.9.6 version. I have this error :

gcc  -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv -L../../common -L../../libpq -lpthread main.o gtm_thread.o
gtm_txn.o gtm_seq.o gtm_snap.o gtm_time.o gtm_standby.o
../libpq/libpqcomm.a ../path/libgtmpath.a ../recovery/libgtmrecovery.a
../client/libgtmclient.a ../common/libgtm.a -o gtm
main.o: In function `MainThreadInit':
main.c:(.text+0x13): undefined reference to `pthread_key_create'
main.c:(.text+0x9a): undefined reference to `pthread_setspecific'
main.o: In function `BaseInit':
main.c:(.text+0x11f): undefined reference to `pthread_getspecific'
main.o: In function `GTM_ThreadMain':
main.c:(.text+0x1157): undefined reference to `pthread_getspecific'
main.c:(.text+0x1167): undefined reference to `pthread_getspecific'
main.c:(.text+0x13b2): undefined reference to `pthread_getspecific'
main.c:(.text+0x13d0): undefined reference to `pthread_getspecific'
main.o:main.c:(.text+0x13e8): more undefined references to
`pthread_getspecific' follow
gtm_thread.o: In function `GTM_ThreadCreate':
gtm_thread.c:(.text+0x465): undefined reference to `pthread_create'
gtm_thread.o: In function `GTM_ThreadJoin':
gtm_thread.c:(.text+0x4e6): undefined reference to `pthread_join'
gtm_thread.o: In function `GTM_ThreadCleanup':
gtm_thread.c:(.text+0x682): undefined reference to `pthread_setspecific'
gtm_thread.o: In function `GTM_ThreadMainWrapper':
gtm_thread.c:(.text+0x6b0): undefined reference to `pthread_detach'
gtm_thread.c:(.text+0x6c4): undefined reference to `pthread_setspecific'
gtm_thread.c:(.text+0x6d1): undefined reference to `pthread_getspecific'
gtm_thread.c:(.text+0x739): undefined reference to `__pthread_register_cancel'
gtm_thread.c:(.text+0x759): undefined reference to `__pthread_unregister_cancel'
gtm_txn.o: In function `GTM_GetGIDData':
gtm_txn.c:(.text+0x170a): undefined reference to `pthread_getspecific'
gtm_txn.c:(.text+0x177d): undefined reference to `pthread_getspecific'
gtm_txn.o: In function `ProcessBeginTransactionCommand':
gtm_txn.c:(.text+0x1865): undefined reference to `pthread_getspecific'
gtm_txn.o: In function `ProcessBeginTransactionGetGXIDCommand':
gtm_txn.c:(.text+0x19e6): undefined reference to `pthread_getspecific'
gtm_txn.c:(.text+0x1bb4): undefined reference to `pthread_getspecific'
gtm_txn.o:gtm_txn.c:(.text+0x1bd5): more undefined references to
`pthread_getspecific' follow
../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockAcquire':
gtm_lock.c:(.text+0x25): undefined reference to `pthread_rwlock_wrlock'
gtm_lock.c:(.text+0x36): undefined reference to `pthread_rwlock_rdlock'
../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockRelease':
gtm_lock.c:(.text+0x8b): undefined reference to `pthread_rwlock_unlock'
../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockInit':
gtm_lock.c:(.text+0xb4): undefined reference to `pthread_rwlock_init'
../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockDestroy':
gtm_lock.c:(.text+0xce): undefined reference to `pthread_rwlock_destroy'
../common/libgtm.a(gtm_lock.o): In function `GTM_RWLockConditionalAcquire':
gtm_lock.c:(.text+0xf9): undefined reference to `pthread_rwlock_trywrlock'
gtm_lock.c:(.text+0x10a): undefined reference to `pthread_rwlock_tryrdlock'
../common/libgtm.a(gtm_lock.o): In function `GTM_MutexLockConditionalAcquire':
gtm_lock.c:(.text+0x1d6): undefined reference to `pthread_mutex_trylock'
../common/libgtm.a(gtm_list.o): In function `new_list':
gtm_list.c:(.text+0x52): undefined reference to `pthread_getspecific'
gtm_list.c:(.text+0x83): undefined reference to `pthread_getspecific'
../common/libgtm.a(gtm_list.o): In function `new_head_cell':
gtm_list.c:(.text+0xdc): undefined reference to `pthread_getspecific'
../common/libgtm.a(gtm_list.o): In function `new_tail_cell':
gtm_list.c:(.text+0x13a): undefined reference to `pthread_getspecific'
../common/libgtm.a(gtm_list.o): In function `add_new_cell':
gtm_list.c:(.text+0x1f0): undefined reference to `pthread_getspecific'
../common/libgtm.a(gtm_list.o):gtm_list.c:(.text+0xc0e): more
undefined references to `pthread_getspecific' follow
collect2: ld returned 1 exit status
make[3]: *** [gtm] Error 1
make[3]: Leaving directory `/home/pgxc/pgxc-0.9.6/src/gtm/main'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/pgxc/pgxc-0.9.6/src/gtm'
make[1]: *** [all-gtm-recurse] Error 2
make[1]: Leaving directory `/home/pgxc/pgxc-0.9.6/src'
make: *** [all-src-recurse] Error 2

I use ubuntu server 11.10 64 bits.
Make version : GNU Make 3.81

Regards,

-- 
Magorn