postgres-xc-developers Mailing List for Postgres-XC

Brought to you by: ahsanhadi, amitdkhan, ashutoshbapat, gabbasb, and 3 others

postgres-xc-developers — Postgres-XC hackers and developers

You can subscribe to this list here.

2010	Jan	Feb	Mar	Apr (10)	May (17)	Jun (3)	Jul	Aug	Sep (8)	Oct (18)	Nov (51)	Dec (74)
2011	Jan (47)	Feb (44)	Mar (44)	Apr (102)	May (35)	Jun (25)	Jul (56)	Aug (69)	Sep (32)	Oct (37)	Nov (31)	Dec (16)
2012	Jan (34)	Feb (127)	Mar (218)	Apr (252)	May (80)	Jun (137)	Jul (205)	Aug (159)	Sep (35)	Oct (50)	Nov (82)	Dec (52)
2013	Jan (107)	Feb (159)	Mar (118)	Apr (163)	May (151)	Jun (89)	Jul (106)	Aug (177)	Sep (49)	Oct (63)	Nov (46)	Dec (7)
2014	Jan (65)	Feb (128)	Mar (40)	Apr (11)	May (4)	Jun (8)	Jul (16)	Aug (11)	Sep (4)	Oct (1)	Nov (5)	Dec (16)
2015	Jan (5)	Feb	Mar (2)	Apr (5)	May (4)	Jun (12)	Jul	Aug	Sep	Oct	Nov	Dec (4)
2019	Jan	Feb	Mar	Apr	May	Jun	Jul (2)	Aug	Sep	Oct	Nov	Dec

Flat | Threaded

[Postgres-xc-developers] Node Registering patch

From: Michael P. <mic...@gm...> - 2011-01-11 06:14:01

Attachments: register10.patch

Hi all,

Here is the final version of the node registering patch.
Mason and Abbas have even been through the code.
I myself checked the code once again to be sure that it worked OK based on
the latest HEAD.

This will be committed soon if nobody has comments on it.
Thanks,
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

[Postgres-xc-developers] Fix for PREPARE TRANSACTION when gid used twice

From: Michael P. <mic...@gm...> - 2011-01-11 05:42:37

Attachments: 2pcfix_20110111.patch

Hi all,

Please see attached a patch that improves a little bit error handling for
explicit 2pc.
There was a problem on GTM where a GID was used twice.
For example with the following sequence:
CREATE TABLE test (a int);
BEGIN;
INSERT INTO test VALUES (1);
INSERT INTO test VALUES (2);
PREPARE TRANSACTION 'h';
BEGIN;
INSERT INTO test VALUES (1);
INSERT INTO test VALUES (2);
PREPARE TRANSACTION 'h'; => X

This does not return an error with the current HEAD because the gid is
already in use on GTM.
With the patch attached, in that case GTM returns an error and rollbacks the
transaction that was going to be prepared.

Any comments or opinions are welcome.

Note:
This patch also seems to fix the bug 3136262.
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: xiong w. <wan...@gm...> - 2011-01-11 02:57:14

2011/1/11 Michael Paquier <mic...@gm...>:
>> Thank you for your reply. We have not get any feedbacks for a long
>> time. We have submited several patches on bugs, but we have not
>> received any reply. We don't know why.
>
> Don't worry the project is not dead!
> Everybody was more or less on vacations for the new year.
> My apologies for the late responses.

Ok, I learn it.

>>
>> We really enjoy the project. Therefore, we spend our time to
>> contribute to the project. The only thing what we expect to get is
>> some advice on what we did. If our patches work well, that's ok. If
>> our patches work badly, we hope you can point out where we are wrong,
>> which is really something to us.
>>
>> BTW, there is no any changes on the source code on sourceforge. Is
>> there anything happened? Could you tell us something processing or
>> current plans on this project so that we can keep up with you?
>
> We are currently working a lot on High-availability features (some of those
> features are going to be committed soon).
> It is pretty long and delicate to make such implementation for a Database
> cluster.
>
> For instance, I am currently working a a patch that permits to register
> nodes in the cluster.
> And also on some feature to allow recovery for a Coordinator.
>
> I am also working on a fix for the feature called EXECUTE DIRECT, that
> allows to launch queries directly on targeted nodes.
> There is also a lot of effort being done for a feature called Datanode
> Mirroring.
> This feature will allow to continue running the cluster even if a Datanode
> is in difficulty (HW, network, or software pb).
>
> Release 0.9.4 is also being prepared.
>

Thanks for your reply. It's much clear on what you are doing now.:)

We have submited serveral patches, we have not get any information
from PGXC team. We are a litter confused. We hope we can be updated
with you.

> I think there is still the multiple INSERT patch you wrote that needs some
> review, no?
>
Yes. It will be much appreciated if you can review. Should I revise
and submit my patch now or after your recent release?

Thanks again for your reply.

Best regards,
Benny
> --
> Michael Paquier
> http://michaelpq.users.sourceforge.net
>
>

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: Koichi S. <koi...@gm...> - 2011-01-11 02:55:44

Thanks.   Then we will change XC license to PostgreSQL's one.
----------
Koichi Suzuki



2011/1/11 xiong wang <wan...@gm...>:
> Hi Koichi,
>
> 在 2011年1月11日 上午10:04，Koichi Suzuki <ko...@in...> 写道：
>> Benny;
>>
>> Thank you very much for the excellent job.  Michael will take care of
>> your patch.
>>
> Many thanks.
>
>> BTW, as you may know, I recently announced that I'm considering to
>> change XC license from LGPL to BSD (or PostgreSQL's one).
>>
>> I'd like have your input on this if you're okay or not.   I believe this
>> change will make XC code quite easier to contribute to PostgreSQL core.
>>
> I consider it's great. I aggree.
>
>> Regards;
>> ---
>> Koichi
>>
>
> Regards,
> Benny
>
>
>> (2011年01月10日 11:06), xiong wang wrote:
>>> Dears，
>>>
>>> We found the same problems when we did test on pg_regress. After
>>> checked, we found statements execution on datanodes can not use
>>> indexseq scan. The only way to scan tuples is seqscan. Such a bug was
>>> introduced by the patch "Improve performance of "multi-step" queries
>>> (an on-going process). 44ca05af2742271abdc5c14f5ca313d5ea307875".
>>>
>>> The enclosure is our patch for this bug.
>>>
>>> Your suggestions will be appreciated.
>>>
>>> Regards,
>>> Benny
>>>
>>> 2011/1/7<pos...@li...>:
>>>> Send Postgres-xc-developers mailing list submissions to
>>>>         pos...@li...
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>         https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>> or, via email, send a message with subject or body 'help' to
>>>>         pos...@li...
>>>>
>>>> You can reach the person managing the list at
>>>>         pos...@li...
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of Postgres-xc-developers digest..."
>>>>
>>>> Today's Topics:
>>>>
>>>>    1. CPU usage in the latest HEAD with stability patch (Koichi Suzuki)
>>>>
>>>>
>>>> ---------- 已转发邮件 ----------
>>>> From: Koichi Suzuki<ko...@in...>
>>>> To: "'Andrei.Martsinchyk'"<And...@en...>, Pavan Deolasee<pav...@en...>, Mason Sharp<mas...@en...>, Abbas Butt<abb...@te...>, Michael Paquier<mi...@in...>, SUDOU Takayuki<sud...@in...>, Postgres-XC Developers<pos...@li...>
>>>> Date: Fri, 07 Jan 2011 18:45:55 +0900
>>>> Subject: [Postgres-xc-developers] CPU usage in the latest HEAD with stability patch
>>>> Hi, (I hope this finally succeeds)
>>>>
>>>> Sudo-san ran DBT-1 with the latest HEAD with stability3.patch
>>>> from Andrei (sorry it may be local so some of the readers may not know).
>>>>   We observed no error but also found specific data node processes are
>>>> consuming most of CPUs, as found in ps and top list attached.  Andrei,
>>>> do you have any suggestion how it was introduced?
>>>>
>>>> Other test environment data is as follows:
>>>>
>>>> PGXC
>>>>   commit 45e1d4e389e966d072aaf98a49d9702aa253d976
>>>>   Author: Mason Sharp<ma...@us...>
>>>>   Date:   Thu Dec 23 15:10:38 2010 -0500
>>>>
>>>> Additional Patch
>>>>   stability3.patch (from Andrei)
>>>>
>>>> Hardwares and configuration
>>>>   PGXC : Co5/Dn5 (coordinator1,2,3,4,5)
>>>>   Loader : loader1, 2
>>>>   GtmProxy使用
>>>>
>>>> DBT1 parameters
>>>>   rampup time 5分、測定時間 5分、合計10分の測定
>>>>
>>>>   transaction_queue_size : 2000
>>>>   transaction_array_size : 2000
>>>>   [dbdriver]
>>>>   items : 10000
>>>>   customers : 28800
>>>>   eu : 500
>>>>   eu/min : 100
>>>>   mean think_tim : 0.5
>>>>   run_duration in seconds : 300
>>>>   access mode : access_appServer
>>>>   access_clean
>>>>
>>>> -------------
>>>> Following are picked-up lines of ps output.  Whole output will be found
>>>> in the attached file.
>>>> (only for coordinator1 and coordinator2)
>>>>
>>>> To indicate the data, ran
>>>> date; ps ux to get the list.
>>>>
>>>> Kind Regards;
>>>> ---
>>>> Koichi Suzuki
>>>>
>>>> # I tried to send this file in several previous mails with tarball, but
>>>> somehow the file was trimmed considerably and many mail-system found it
>>>> could be bad one.   I changed the format to zip and I hope this works.
>>>>
>>>> I apologize if anyone received corrupted file many times.
>>>>
>>>> =========================================================
>>>> Hostname: coordinator1
>>>>   Fri Jan  7 16:08:15 JST 2011
>>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>>   502       4957 84.4  0.0 226404  9144 ?        Rs   16:08   0:08
>>>> postgres: sudoutaka dbt1 192.168.247.72(60593) SELECT
>>>>   502       4965 79.6  0.0 226344  8968 ?        Rs   16:08   0:04
>>>> postgres: sudoutaka dbt1 192.168.247.71(57079) SELECT
>>>>   502       4971 62.0  0.0 226432  8876 ?        Rs   16:08   0:02
>>>> postgres: sudoutaka dbt1 192.168.247.73(34752) SELECT
>>>>
>>>>   Fri Jan  7 16:08:45 JST 2011
>>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>>   502       4957 29.5  0.0 226204  9280 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.72(60593) idle
>>>>   502       4965 32.8  0.0 226144  9216 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.71(57079) idle
>>>>   502       4971 34.6  0.0 226212  9276 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.73(34752) idle in transaction
>>>>   502       4986 41.1  0.0 226148  9236 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.75(48995) idle
>>>>   502       4990 44.0  0.0 226148  9224 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.73(34781) idle in transaction
>>>>   502       4995 49.0  0.0 226212  9284 ?        Ss   16:08   0:11
>>>> postgres: sudoutaka dbt1 192.168.247.72(60651) idle
>>>>   502       5018 20.0  0.0 226148 11680 ?        Ss   16:08   0:00
>>>> postgres: sudoutaka dbt1 192.168.247.74(48020) idle in transaction
>>>>
>>>> Hostname: coordinator2
>>>>   Fri Jan  7 16:07:33 JST 2011
>>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>>   502      18933 80.1  0.0 226324  9164 ?        Rs   16:07   0:08
>>>> postgres: sudoutaka dbt1 192.168.247.72(60619) SELECT
>>>>   502      18940 54.0  0.0 226324  8908 ?        Rs   16:07   0:03
>>>> postgres: sudoutaka dbt1 192.168.247.71(34207) SELECT
>>>>   502      18944 81.6  0.0 226132 10848 ?        Rs   16:07   0:04
>>>> postgres: sudoutaka dbt1 192.168.247.71(34214) SELECT
>>>>   502      18946 80.0  0.0 226412  8920 ?        Rs   16:07   0:03
>>>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>>>
>>>> Fri Jan  7 16:08:03 JST 2011
>>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>>   502      18927 31.1  0.1 226196 15980 ?        Ss   16:07   0:13
>>>> postgres: sudoutaka dbt1 192.168.247.73(58650) idle in transaction
>>>>   502      18933 30.2  0.0 226124  9252 ?        Ss   16:07   0:12
>>>> postgres: sudoutaka dbt1 192.168.247.72(60619) idle
>>>>   502      18940 34.8  0.0 226124  9304 ?        Ss   16:07   0:12
>>>> postgres: sudoutaka dbt1 192.168.247.71(34207) idle
>>>>   502      18944 51.4  0.0 226076 11060 ?        Ss   16:07   0:18
>>>> postgres: sudoutaka dbt1 192.168.247.71(34214) idle in transaction
>>>>   502      18946 70.2  0.1 226192 15124 ?        Rs   16:07   0:23
>>>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>>>   502      18957 43.4  0.0 226128  9248 ?        Ss   16:07   0:12
>>>> postgres: sudoutaka dbt1 192.168.247.75(58779) idle
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Gaining the trust of online customers is vital for the success of any company
>>>> that requires sensitive data to be transmitted over the Web.   Learn how to
>>>> best implement a security strategy that keeps consumers' information secure
>>>> and instills the confidence they need to proceed with transactions.
>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any company
> that requires sensitive data to be transmitted over the Web.   Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: xiong w. <wan...@gm...> - 2011-01-11 02:43:53

Hi Koichi,

在 2011年1月11日 上午10:04，Koichi Suzuki <ko...@in...> 写道：
> Benny;
>
> Thank you very much for the excellent job.  Michael will take care of
> your patch.
>
Many thanks.

> BTW, as you may know, I recently announced that I'm considering to
> change XC license from LGPL to BSD (or PostgreSQL's one).
>
> I'd like have your input on this if you're okay or not.   I believe this
> change will make XC code quite easier to contribute to PostgreSQL core.
>
I consider it's great. I aggree.

> Regards;
> ---
> Koichi
>

Regards,
Benny


> (2011年01月10日 11:06), xiong wang wrote:
>> Dears，
>>
>> We found the same problems when we did test on pg_regress. After
>> checked, we found statements execution on datanodes can not use
>> indexseq scan. The only way to scan tuples is seqscan. Such a bug was
>> introduced by the patch "Improve performance of "multi-step" queries
>> (an on-going process). 44ca05af2742271abdc5c14f5ca313d5ea307875".
>>
>> The enclosure is our patch for this bug.
>>
>> Your suggestions will be appreciated.
>>
>> Regards,
>> Benny
>>
>> 2011/1/7<pos...@li...>:
>>> Send Postgres-xc-developers mailing list submissions to
>>>         pos...@li...
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>         https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> or, via email, send a message with subject or body 'help' to
>>>         pos...@li...
>>>
>>> You can reach the person managing the list at
>>>         pos...@li...
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of Postgres-xc-developers digest..."
>>>
>>> Today's Topics:
>>>
>>>    1. CPU usage in the latest HEAD with stability patch (Koichi Suzuki)
>>>
>>>
>>> ---------- 已转发邮件 ----------
>>> From: Koichi Suzuki<ko...@in...>
>>> To: "'Andrei.Martsinchyk'"<And...@en...>, Pavan Deolasee<pav...@en...>, Mason Sharp<mas...@en...>, Abbas Butt<abb...@te...>, Michael Paquier<mi...@in...>, SUDOU Takayuki<sud...@in...>, Postgres-XC Developers<pos...@li...>
>>> Date: Fri, 07 Jan 2011 18:45:55 +0900
>>> Subject: [Postgres-xc-developers] CPU usage in the latest HEAD with stability patch
>>> Hi, (I hope this finally succeeds)
>>>
>>> Sudo-san ran DBT-1 with the latest HEAD with stability3.patch
>>> from Andrei (sorry it may be local so some of the readers may not know).
>>>   We observed no error but also found specific data node processes are
>>> consuming most of CPUs, as found in ps and top list attached.  Andrei,
>>> do you have any suggestion how it was introduced?
>>>
>>> Other test environment data is as follows:
>>>
>>> PGXC
>>>   commit 45e1d4e389e966d072aaf98a49d9702aa253d976
>>>   Author: Mason Sharp<ma...@us...>
>>>   Date:   Thu Dec 23 15:10:38 2010 -0500
>>>
>>> Additional Patch
>>>   stability3.patch (from Andrei)
>>>
>>> Hardwares and configuration
>>>   PGXC : Co5/Dn5 (coordinator1,2,3,4,5)
>>>   Loader : loader1, 2
>>>   GtmProxy使用
>>>
>>> DBT1 parameters
>>>   rampup time 5分、定 5分、合10分の定
>>>
>>>   transaction_queue_size : 2000
>>>   transaction_array_size : 2000
>>>   [dbdriver]
>>>   items : 10000
>>>   customers : 28800
>>>   eu : 500
>>>   eu/min : 100
>>>   mean think_tim : 0.5
>>>   run_duration in seconds : 300
>>>   access mode : access_appServer
>>>   access_clean
>>>
>>> -------------
>>> Following are picked-up lines of ps output.  Whole output will be found
>>> in the attached file.
>>> (only for coordinator1 and coordinator2)
>>>
>>> To indicate the data, ran
>>> date; ps ux to get the list.
>>>
>>> Kind Regards;
>>> ---
>>> Koichi Suzuki
>>>
>>> # I tried to send this file in several previous mails with tarball, but
>>> somehow the file was trimmed considerably and many mail-system found it
>>> could be bad one.   I changed the format to zip and I hope this works.
>>>
>>> I apologize if anyone received corrupted file many times.
>>>
>>> =========================================================
>>> Hostname: coordinator1
>>>   Fri Jan  7 16:08:15 JST 2011
>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>   502       4957 84.4  0.0 226404  9144 ?        Rs   16:08   0:08
>>> postgres: sudoutaka dbt1 192.168.247.72(60593) SELECT
>>>   502       4965 79.6  0.0 226344  8968 ?        Rs   16:08   0:04
>>> postgres: sudoutaka dbt1 192.168.247.71(57079) SELECT
>>>   502       4971 62.0  0.0 226432  8876 ?        Rs   16:08   0:02
>>> postgres: sudoutaka dbt1 192.168.247.73(34752) SELECT
>>>
>>>   Fri Jan  7 16:08:45 JST 2011
>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>   502       4957 29.5  0.0 226204  9280 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.72(60593) idle
>>>   502       4965 32.8  0.0 226144  9216 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.71(57079) idle
>>>   502       4971 34.6  0.0 226212  9276 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.73(34752) idle in transaction
>>>   502       4986 41.1  0.0 226148  9236 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.75(48995) idle
>>>   502       4990 44.0  0.0 226148  9224 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.73(34781) idle in transaction
>>>   502       4995 49.0  0.0 226212  9284 ?        Ss   16:08   0:11
>>> postgres: sudoutaka dbt1 192.168.247.72(60651) idle
>>>   502       5018 20.0  0.0 226148 11680 ?        Ss   16:08   0:00
>>> postgres: sudoutaka dbt1 192.168.247.74(48020) idle in transaction
>>>
>>> Hostname: coordinator2
>>>   Fri Jan  7 16:07:33 JST 2011
>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>   502      18933 80.1  0.0 226324  9164 ?        Rs   16:07   0:08
>>> postgres: sudoutaka dbt1 192.168.247.72(60619) SELECT
>>>   502      18940 54.0  0.0 226324  8908 ?        Rs   16:07   0:03
>>> postgres: sudoutaka dbt1 192.168.247.71(34207) SELECT
>>>   502      18944 81.6  0.0 226132 10848 ?        Rs   16:07   0:04
>>> postgres: sudoutaka dbt1 192.168.247.71(34214) SELECT
>>>   502      18946 80.0  0.0 226412  8920 ?        Rs   16:07   0:03
>>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>>
>>> Fri Jan  7 16:08:03 JST 2011
>>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>>   502      18927 31.1  0.1 226196 15980 ?        Ss   16:07   0:13
>>> postgres: sudoutaka dbt1 192.168.247.73(58650) idle in transaction
>>>   502      18933 30.2  0.0 226124  9252 ?        Ss   16:07   0:12
>>> postgres: sudoutaka dbt1 192.168.247.72(60619) idle
>>>   502      18940 34.8  0.0 226124  9304 ?        Ss   16:07   0:12
>>> postgres: sudoutaka dbt1 192.168.247.71(34207) idle
>>>   502      18944 51.4  0.0 226076 11060 ?        Ss   16:07   0:18
>>> postgres: sudoutaka dbt1 192.168.247.71(34214) idle in transaction
>>>   502      18946 70.2  0.1 226192 15124 ?        Rs   16:07   0:23
>>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>>   502      18957 43.4  0.0 226128  9248 ?        Ss   16:07   0:12
>>> postgres: sudoutaka dbt1 192.168.247.75(58779) idle
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Gaining the trust of online customers is vital for the success of any company
>>> that requires sensitive data to be transmitted over the Web.   Learn how to
>>> best implement a security strategy that keeps consumers' information secure
>>> and instills the confidence they need to proceed with transactions.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>>
>
>

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: Michael P. <mic...@gm...> - 2011-01-11 02:21:00

>
> Thank you for your reply. We have not get any feedbacks for a long
> time. We have submited several patches on bugs, but we have not
> received any reply. We don't know why.
>
Don't worry the project is not dead!
Everybody was more or less on vacations for the new year.
My apologies for the late responses.

>
> We really enjoy the project. Therefore, we spend our time to
> contribute to the project. The only thing what we expect to get is
> some advice on what we did. If our patches work well, that's ok. If
> our patches work badly, we hope you can point out where we are wrong,
> which is really something to us.
>
> BTW, there is no any changes on the source code on sourceforge. Is
> there anything happened? Could you tell us something processing or
> current plans on this project so that we can keep up with you?
>

We are currently working a lot on High-availability features (some of those
features are going to be committed soon).
It is pretty long and delicate to make such implementation for a Database
cluster.

For instance, I am currently working a a patch that permits to register
nodes in the cluster.
And also on some feature to allow recovery for a Coordinator.

I am also working on a fix for the feature called EXECUTE DIRECT, that
allows to launch queries directly on targeted nodes.
There is also a lot of effort being done for a feature called Datanode
Mirroring.
This feature will allow to continue running the cluster even if a Datanode
is in difficulty (HW, network, or software pb).

Release 0.9.4 is also being prepared.

I think there is still the multiple INSERT patch you wrote that needs some
review, no?

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: Koichi S. <ko...@in...> - 2011-01-11 02:04:41

Benny;

Thank you very much for the excellent job.  Michael will take care of
your patch.

BTW, as you may know, I recently announced that I'm considering to
change XC license from LGPL to BSD (or PostgreSQL's one).

I'd like have your input on this if you're okay or not.   I believe this
change will make XC code quite easier to contribute to PostgreSQL core.

Regards;
---
Koichi

(2011年01月10日 11:06), xiong wang wrote:
> Dears，
> 
> We found the same problems when we did test on pg_regress. After
> checked, we found statements execution on datanodes can not use
> indexseq scan. The only way to scan tuples is seqscan. Such a bug was
> introduced by the patch "Improve performance of "multi-step" queries
> (an on-going process). 44ca05af2742271abdc5c14f5ca313d5ea307875".
> 
> The enclosure is our patch for this bug.
> 
> Your suggestions will be appreciated.
> 
> Regards,
> Benny
> 
> 2011/1/7<pos...@li...>:
>> Send Postgres-xc-developers mailing list submissions to
>>         pos...@li...
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> or, via email, send a message with subject or body 'help' to
>>         pos...@li...
>>
>> You can reach the person managing the list at
>>         pos...@li...
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Postgres-xc-developers digest..."
>>
>> Today's Topics:
>>
>>    1. CPU usage in the latest HEAD with stability patch (Koichi Suzuki)
>>
>>
>> ---------- 已转发邮件 ----------
>> From: Koichi Suzuki<ko...@in...>
>> To: "'Andrei.Martsinchyk'"<And...@en...>, Pavan Deolasee<pav...@en...>, Mason Sharp<mas...@en...>, Abbas Butt<abb...@te...>, Michael Paquier<mi...@in...>, SUDOU Takayuki<sud...@in...>, Postgres-XC Developers<pos...@li...>
>> Date: Fri, 07 Jan 2011 18:45:55 +0900
>> Subject: [Postgres-xc-developers] CPU usage in the latest HEAD with stability patch
>> Hi, (I hope this finally succeeds)
>>
>> Sudo-san ran DBT-1 with the latest HEAD with stability3.patch
>> from Andrei (sorry it may be local so some of the readers may not know).
>>   We observed no error but also found specific data node processes are
>> consuming most of CPUs, as found in ps and top list attached.  Andrei,
>> do you have any suggestion how it was introduced?
>>
>> Other test environment data is as follows:
>>
>> PGXC
>>   commit 45e1d4e389e966d072aaf98a49d9702aa253d976
>>   Author: Mason Sharp<ma...@us...>
>>   Date:   Thu Dec 23 15:10:38 2010 -0500
>>
>> Additional Patch
>>   stability3.patch (from Andrei)
>>
>> Hardwares and configuration
>>   PGXC : Co5/Dn5 (coordinator1,2,3,4,5)
>>   Loader : loader1, 2
>>   GtmProxy使用
>>
>> DBT1 parameters
>>   rampup time 5分、定 5分、合10分の定
>>
>>   transaction_queue_size : 2000
>>   transaction_array_size : 2000
>>   [dbdriver]
>>   items : 10000
>>   customers : 28800
>>   eu : 500
>>   eu/min : 100
>>   mean think_tim : 0.5
>>   run_duration in seconds : 300
>>   access mode : access_appServer
>>   access_clean
>>
>> -------------
>> Following are picked-up lines of ps output.  Whole output will be found
>> in the attached file.
>> (only for coordinator1 and coordinator2)
>>
>> To indicate the data, ran
>> date; ps ux to get the list.
>>
>> Kind Regards;
>> ---
>> Koichi Suzuki
>>
>> # I tried to send this file in several previous mails with tarball, but
>> somehow the file was trimmed considerably and many mail-system found it
>> could be bad one.   I changed the format to zip and I hope this works.
>>
>> I apologize if anyone received corrupted file many times.
>>
>> =========================================================
>> Hostname: coordinator1
>>   Fri Jan  7 16:08:15 JST 2011
>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>   502       4957 84.4  0.0 226404  9144 ?        Rs   16:08   0:08
>> postgres: sudoutaka dbt1 192.168.247.72(60593) SELECT
>>   502       4965 79.6  0.0 226344  8968 ?        Rs   16:08   0:04
>> postgres: sudoutaka dbt1 192.168.247.71(57079) SELECT
>>   502       4971 62.0  0.0 226432  8876 ?        Rs   16:08   0:02
>> postgres: sudoutaka dbt1 192.168.247.73(34752) SELECT
>>
>>   Fri Jan  7 16:08:45 JST 2011
>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>   502       4957 29.5  0.0 226204  9280 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.72(60593) idle
>>   502       4965 32.8  0.0 226144  9216 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.71(57079) idle
>>   502       4971 34.6  0.0 226212  9276 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.73(34752) idle in transaction
>>   502       4986 41.1  0.0 226148  9236 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.75(48995) idle
>>   502       4990 44.0  0.0 226148  9224 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.73(34781) idle in transaction
>>   502       4995 49.0  0.0 226212  9284 ?        Ss   16:08   0:11
>> postgres: sudoutaka dbt1 192.168.247.72(60651) idle
>>   502       5018 20.0  0.0 226148 11680 ?        Ss   16:08   0:00
>> postgres: sudoutaka dbt1 192.168.247.74(48020) idle in transaction
>>
>> Hostname: coordinator2
>>   Fri Jan  7 16:07:33 JST 2011
>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>   502      18933 80.1  0.0 226324  9164 ?        Rs   16:07   0:08
>> postgres: sudoutaka dbt1 192.168.247.72(60619) SELECT
>>   502      18940 54.0  0.0 226324  8908 ?        Rs   16:07   0:03
>> postgres: sudoutaka dbt1 192.168.247.71(34207) SELECT
>>   502      18944 81.6  0.0 226132 10848 ?        Rs   16:07   0:04
>> postgres: sudoutaka dbt1 192.168.247.71(34214) SELECT
>>   502      18946 80.0  0.0 226412  8920 ?        Rs   16:07   0:03
>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>
>> Fri Jan  7 16:08:03 JST 2011
>>   USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>>   502      18927 31.1  0.1 226196 15980 ?        Ss   16:07   0:13
>> postgres: sudoutaka dbt1 192.168.247.73(58650) idle in transaction
>>   502      18933 30.2  0.0 226124  9252 ?        Ss   16:07   0:12
>> postgres: sudoutaka dbt1 192.168.247.72(60619) idle
>>   502      18940 34.8  0.0 226124  9304 ?        Ss   16:07   0:12
>> postgres: sudoutaka dbt1 192.168.247.71(34207) idle
>>   502      18944 51.4  0.0 226076 11060 ?        Ss   16:07   0:18
>> postgres: sudoutaka dbt1 192.168.247.71(34214) idle in transaction
>>   502      18946 70.2  0.1 226192 15124 ?        Rs   16:07   0:23
>> postgres: sudoutaka dbt1 192.168.247.73(48371) SELECT
>>   502      18957 43.4  0.0 226128  9248 ?        Ss   16:07   0:12
>> postgres: sudoutaka dbt1 192.168.247.75(58779) idle
>>
>>
>> ------------------------------------------------------------------------------
>> Gaining the trust of online customers is vital for the success of any company
>> that requires sensitive data to be transmitted over the Web.   Learn how to
>> best implement a security strategy that keeps consumers' information secure
>> and instills the confidence they need to proceed with transactions.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: xiong w. <wan...@gm...> - 2011-01-11 02:00:48

Dears,

在 2011年1月11日 上午9:19，Michael Paquier <mic...@gm...> 写道：
> 2011/1/10 xiong wang <wan...@gm...>
>>
>> Dears，
>>
>> We found the same problems when we did test on pg_regress. After
>> checked, we found statements execution on datanodes can not use
>> indexseq scan. The only way to scan tuples is seqscan. Such a bug was
>> introduced by the patch "Improve performance of "multi-step" queries
>> (an on-going process). 44ca05af2742271abdc5c14f5ca313d5ea307875".
>>
>> The enclosure is our patch for this bug.
>>
>> Your suggestions will be appreciated.
>
> Hi Benny,
>
> Well done!
> This is exactly the kind of stuff that worsened performance.
>
> I made a test with your patch and this has huge effects on the performance.
> If you see something else like this, don't hesitate to post it.
>


> --
> Michael Paquier
> http://michaelpq.users.sourceforge.net
>
>

Thank you for your reply. We have not get any feedbacks for a long
time. We have submited several patches on bugs, but we have not
received any reply. We don't know why.

We really enjoy the project. Therefore, we spend our time to
contribute to the project. The only thing what we expect to get is
some advice on what we did. If our patches work well, that's ok. If
our patches work badly, we hope you can point out where we are wrong,
which is really something to us.

BTW, there is no any changes on the source code on sourceforge. Is
there anything happened? Could you tell us something processing or
current plans on this project so that we can keep up with you?

Many thanks

Regards,
Benny

Re: [Postgres-xc-developers] Postgres-xc-developers Digest, Vol 7, Issue 3

From: Michael P. <mic...@gm...> - 2011-01-11 01:19:12

2011/1/10 xiong wang <wan...@gm...>

> Dears，
>
> We found the same problems when we did test on pg_regress. After
> checked, we found statements execution on datanodes can not use
> indexseq scan. The only way to scan tuples is seqscan. Such a bug was
> introduced by the patch "Improve performance of "multi-step" queries
> (an on-going process). 44ca05af2742271abdc5c14f5ca313d5ea307875".
>
> The enclosure is our patch for this bug.
>
> Your suggestions will be appreciated.
>

Hi Benny,

Well done!
This is exactly the kind of stuff that worsened performance.

I made a test with your patch and this has huge effects on the performance.
If you see something else like this, don't hesitate to post it.

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

1 message has been excluded from this view by a project administrator.

Flat | Threaded