Menu
▾
▴
postgres-xc-developers — Postgres-XC hackers and developers
You can subscribe to this list here.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
(10) |
May
(17) |
Jun
(3) |
Jul
|
Aug
|
Sep
(8) |
Oct
(18) |
Nov
(51) |
Dec
(74) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(47) |
Feb
(44) |
Mar
(44) |
Apr
(102) |
May
(35) |
Jun
(25) |
Jul
(56) |
Aug
(69) |
Sep
(32) |
Oct
(37) |
Nov
(31) |
Dec
(16) |
| 2012 |
Jan
(34) |
Feb
(127) |
Mar
(218) |
Apr
(252) |
May
(80) |
Jun
(137) |
Jul
(205) |
Aug
(159) |
Sep
(35) |
Oct
(50) |
Nov
(82) |
Dec
(52) |
| 2013 |
Jan
(107) |
Feb
(159) |
Mar
(118) |
Apr
(163) |
May
(151) |
Jun
(89) |
Jul
(106) |
Aug
(177) |
Sep
(49) |
Oct
(63) |
Nov
(46) |
Dec
(7) |
| 2014 |
Jan
(65) |
Feb
(128) |
Mar
(40) |
Apr
(11) |
May
(4) |
Jun
(8) |
Jul
(16) |
Aug
(11) |
Sep
(4) |
Oct
(1) |
Nov
(5) |
Dec
(16) |
| 2015 |
Jan
(5) |
Feb
|
Mar
(2) |
Apr
(5) |
May
(4) |
Jun
(12) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(4) |
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
1
(6) |
2
(10) |
3
(13) |
4
(1) |
5
|
|
6
|
7
(2) |
8
(4) |
9
(12) |
10
(1) |
11
|
12
|
|
13
|
14
|
15
|
16
|
17
|
18
|
19
|
|
20
|
21
|
22
(1) |
23
(2) |
24
(4) |
25
(3) |
26
|
|
27
|
28
|
29
|
30
(4) |
31
|
|
|
|
From: Tomonari K. <kat...@po...> - 2013-10-30 05:42:03
|
Hi Ashutosh, Here is the patch without the testcase. Thank you! ---------------- Tomonari Katsumata (2013/10/30 13:51), Ashutosh Bapat wrote: > Hi Tomonari, > I am waiting for you to provide me a patch without the testcase. > > > On Wed, Oct 30, 2013 at 6:51 AM, Tomonari Katsumata < > kat...@po...> wrote: > >> Hi, >> >> Almost one month is passing without any >> comments about this thread. >> >> I need this fix as soon as possible, because >> these kinds queries are executed by >> pg_statsinfo for Postgres-XC. >> >> Anybody, please commit it. >> >> regards, >> ------------ >> Tomonari Katsumata >> >> >> (2013/09/28 10:53), Tomonari Katsumata wrote: >>> Hi Ashotosh, >>> >>>> I never said "don't use functions". Here's what I mailed >>>> >>> Oh, Sorry for bothering you by my misunderstanding. >>> >>>>> What is better for this testcase? >>>> >>>> May be we should just drop the testcase. Anyway, EXEC DIRECT is not >>> production feature, but debug feature and whosoever wrote this feature >> has >>> not added any testcases for it anyway. >>>> If anyone else has objections to dropping the testcase, please >> suggest >>> Tomonari, a better to write it. >>>> >>> Past 2 days, any suggestions were not sent to me. >>> So it seems no objections to dropping the testcase. >>> >>> regards, >>> ---------------- >>> NTT Software Corporation >>> Tomonari Katsumata >>> >>> >>> >>> 2013/9/26 Ashutosh Bapat <ashutosh.bapat@enterprisedb.**com<ash...@en...> >>> >>> >>>> >>>> >>>> >>>> On Thu, Sep 26, 2013 at 1:29 PM, Tomonari Katsumata < >>>> kat...@po....**jp <kat...@po...>> >> wrote: >>>> >>>>> Hi Ashtosh, >>>>> >>>>> Thanks for your review! >>>>> >>>>> Previous patch(revised_fqs_r3.patch in [2013/08/14 19:55]) >>>>> doesn't use a shell script. >>>>> Instead, It used test_execute_direct_all_xc_****nodes() function. >> >>>>> >>>>> I've got a comment "don't use functions" from you, >>>>> but I could not come up with good idea for the testcase. >>>>> The testcase needs to get all node_names to "EXECUTE DIRECT" >>>>> against all nodes. >>>>> >>>>> >>>> I never said "don't use functions". Here's what I mailed >>>> >>>> "In the test file, isn't there any way, you can add the offending >> statent >>>> without any wrapper function test_execute_direct_all_xc_ >>>> nodes()? We should minimize the test-code to use only the minimal set of >>>> features. Since this statement no more gets into infinite loop, please >> do >>>> not use the statement timeout." >>>> >>>> >>>>> What is better for this testcase? >>>>> >>>>> >>>> May be we should just drop the testcase. Anyway, EXEC DIRECT is not >>>> production feature, but debug feature and whosoever wrote this feature >> has >>>> not added any testcases for it anyway. >>>> >>>> If anyone else has objections to dropping the testcase, please suggest >>>> Tomonari, a better to write it. >>>> >>>> >>>>> regards, >>>>> >>>>> ---------------- >>>>> NTT Software Corporation >>>>> Tomonari Katsumata >>>>> >>>>> (2013/09/26 15:44), Ashutosh Bapat wrote: >>>>>> Hi Tomonary, >>>>>> The testcase you have written uses a shell script. This is not the >>>>> standard >>>>>> way to add testcases. Can you please see if the testcase can be added >>>>>> directly as SQL? Please send a revised patch ASAP, so that it can be >>>>>> committed. I would have liked to see more testcases for EXEC DIRECT, >> but >>>>>> since it's a debug only feature (not for production), it gets lesser >>>>>> priority in testing and thus we do not see testcases for it. >>>>>> >>>>>> >>>>>> On Thu, Sep 26, 2013 at 8:26 AM, Tomonari Katsumata < >>>>>> kat...@po....****jp <katsumata.tomonari@po.ntts.** >> co.jp <kat...@po...>>> >> >>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Does anyone check these patches? >>>>>>> If there are no comments, please commit it. >>>>>>> >>>>>>> regards, >>>>>>> >>>>>>> ---------------------- >>>>>>> NTT Software Corporation >>>>>>> Tomonari Katsumata >>>>>>> >>>>>>> -------- Original Message -------- >>>>>>> Subject: Re: [Postgres-xc-developers] Query Planning bug ? >>>>>>> Date: Thu, 5 Sep 2013 21:34:32 +0900 >>>>>>> From: Tomonari Katsumata <t.k...@gm...> >>>>>>> To: Tomonari Katsumata <katsumata.tomonari@po.ntts.******co.jp< >> http://co.jp> >>>>> <katsumata.tomonari@po.**ntts.**co.jp <http://ntts.co.jp> < >> katsumata.tomonari@po.ntts.**co.jp <kat...@po...>>> >>>>>>>> >>>>>>> CC: Ashutosh Bapat <ashutosh.bapat@enterprisedb.******com< >>>>> ashutosh.bapat@**enterprisedb.**com <http://enterprisedb.com> < >> ashutosh.bapat@enterprisedb.**com <ash...@en...>>>>, >>>>>>> pgxc-hackers mailing list <postgres-xc-developers@lists.****** >>>>> sourceforge.net<postgres-xc-****dev...@li...urceforge.****net< >> postgres-xc-developers@**lists.sourceforge.net<pos...@li...> >>> >> >>>>>> >>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm sorry for long long leaving this thread alone. >>>>>>> I've devided the patch to source part and regression part. >>>>>>> (against f8e0f994bff1faefa9b1fc2f6e584a******1d3de70241) >> >>>>> >>>>>>> >>>>>>> I did not change any process on source part. >>>>>>> (revised_fqs_r4.patch) >>>>>>> >>>>>>> I added a script into src/test/regress/sql directory for >>>>>>> changing regression part. >>>>>>> (add_test_xc_misc.patch and execute_direct_regress.sh) >>>>>>> When regression test runs, the script is executed instead of >>>>>>> user defined function. >>>>>>> >>>>>>> Please check and commit it. >>>>>>> >>>>>>> regards, >>>>>>> ------------------------ >>>>>>> NTT Software Corporation >>>>>>> Tomonari Katsumata >>>>>>> >>>>>>> >>>>>>> 2013/8/14 Tomonari Katsumata <katsumata.tomonari@po.ntts.******co.jp >> <http://co.jp> >>>>> <katsumata.tomonari@po.**ntts.**co.jp <http://ntts.co.jp> < >> katsumata.tomonari@po.ntts.**co.jp <kat...@po...>>> >> >>>>> >>>>>>>> >>>>>>> >>>>>>>> Hi Ashtosh, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> >>>>>>>> >>>>>>>>> The code changes are fine, but, the comment in >>>>>>> pgxc_collect_RTE_walker() >>>>>>>>> seems to specific. I think it should read, "create a copy of >> query's >>>>>>>> range >>>>>>>>> table, so that it can be linked with other RTEs in the collector's >>>>>>>> context." >>>>>>>>> >>>>>>>> I've revised the comment as you suggested. >>>>>>>> >>>>>>>> >>>>>>>>> In the test file, isn't there any way, you can add the offending >>>>>>> statent >>>>>>>>> without any wrapper function test_execute_direct_all_xc_******* >> >>>>> *nodes()? >>>>> >>>>>>> We >>>>>>> >>>>>>>> should >>>>>>>>> minimize the test-code to use only the minimal set of features. >> Since >>>>>>>> this >>>>>>>>> statement no more gets into infinite loop, please do not use the >>>>>>>> statement >>>>>>>>> timeout. >>>>>>>> >>>>>>>> I've gotten rid of using statement timeout. >>>>>>>> But I couldn't come up any idea to testing more simply, >>>>>>>> so the function is remaining as same as before patch. >>>>>>>> Any ideas? >>>>>>>> >>>>>>>> Here is the new patch. >>>>>>>> (against dec40008b3d689911566514614c511********1c0a61327d) >> >>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> regards, >>>>>>>> ------------- >>>>>>>> NTT Software Corporation >>>>>>>> Tomonari Katsumata >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ------------------------------****----------------------------**--** >> >>>>> ------------------ >>>>>>> October Webinars: Code for Performance >>>>>>> Free Intel webinars can help you accelerate application performance. >>>>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>>>> most >>>>>>> from >>>>>>> the latest Intel processors and coprocessors. See abstracts and >>>>> register > >>>>>>> http://pubads.g.doubleclick.****net/gampad/clk?id=60133471&iu=**** >>>>> /4140/ostg.clktrk<http://**pubads.g.doubleclick.net/** >> gampad/clk?id=60133471&iu=/**4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> >>> >> >>>>>>> ______________________________****_________________ >>>>>>> Postgres-xc-developers mailing list >>>>>>> Postgres-xc-developers@lists.****sourceforge.net<Postgres-xc-** >> dev...@li...urceforge.**net<Pos...@li...> >>> >>>>>>> https://lists.sourceforge.net/****lists/listinfo/postgres-xc-****<https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**> >>>>> developers<https://lists.**sourceforge.net/lists/** >> listinfo/postgres-xc-**developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> >>> >> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best Wishes, >>>> Ashutosh Bapat >>>> EnterpriseDB Corporation >>>> The Postgres Database Company >>>> >>>> >>>> ------------------------------**------------------------------** >> ------------------ >>>> October Webinars: Code for Performance >>>> Free Intel webinars can help you accelerate application performance. >>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >>>> from >>>> the latest Intel processors and coprocessors. See abstracts and >> register > >>>> http://pubads.g.doubleclick.**net/gampad/clk?id=60133471&iu=** >> /4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> >>>> ______________________________**_________________ >>>> Postgres-xc-developers mailing list >>>> Postgres-xc-developers@lists.**sourceforge.net<Pos...@li...> >>>> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> >>>> >>>> >>> >> >> >> > > |
|
From: Ashutosh B. <ash...@en...> - 2013-10-30 04:51:15
|
Hi Tomonari, I am waiting for you to provide me a patch without the testcase. On Wed, Oct 30, 2013 at 6:51 AM, Tomonari Katsumata < kat...@po...> wrote: > Hi, > > Almost one month is passing without any > comments about this thread. > > I need this fix as soon as possible, because > these kinds queries are executed by > pg_statsinfo for Postgres-XC. > > Anybody, please commit it. > > regards, > ------------ > Tomonari Katsumata > > > (2013/09/28 10:53), Tomonari Katsumata wrote: > > Hi Ashotosh, > > > >> I never said "don't use functions". Here's what I mailed > >> > > Oh, Sorry for bothering you by my misunderstanding. > > > >>> What is better for this testcase? > >> > >> May be we should just drop the testcase. Anyway, EXEC DIRECT is not > > production feature, but debug feature and whosoever wrote this feature > has > > not added any testcases for it anyway. > >> If anyone else has objections to dropping the testcase, please > suggest > > Tomonari, a better to write it. > >> > > Past 2 days, any suggestions were not sent to me. > > So it seems no objections to dropping the testcase. > > > > regards, > > ---------------- > > NTT Software Corporation > > Tomonari Katsumata > > > > > > > > 2013/9/26 Ashutosh Bapat <ashutosh.bapat@enterprisedb.**com<ash...@en...> > > > > > >> > >> > >> > >> On Thu, Sep 26, 2013 at 1:29 PM, Tomonari Katsumata < > >> kat...@po....**jp <kat...@po...>> > wrote: > >> > >>> Hi Ashtosh, > >>> > >>> Thanks for your review! > >>> > >>> Previous patch(revised_fqs_r3.patch in [2013/08/14 19:55]) > >>> doesn't use a shell script. > >>> Instead, It used test_execute_direct_all_xc_****nodes() function. > > >>> > >>> I've got a comment "don't use functions" from you, > >>> but I could not come up with good idea for the testcase. > >>> The testcase needs to get all node_names to "EXECUTE DIRECT" > >>> against all nodes. > >>> > >>> > >> I never said "don't use functions". Here's what I mailed > >> > >> "In the test file, isn't there any way, you can add the offending > statent > >> without any wrapper function test_execute_direct_all_xc_ > >> nodes()? We should minimize the test-code to use only the minimal set of > >> features. Since this statement no more gets into infinite loop, please > do > >> not use the statement timeout." > >> > >> > >>> What is better for this testcase? > >>> > >>> > >> May be we should just drop the testcase. Anyway, EXEC DIRECT is not > >> production feature, but debug feature and whosoever wrote this feature > has > >> not added any testcases for it anyway. > >> > >> If anyone else has objections to dropping the testcase, please suggest > >> Tomonari, a better to write it. > >> > >> > >>> regards, > >>> > >>> ---------------- > >>> NTT Software Corporation > >>> Tomonari Katsumata > >>> > >>> (2013/09/26 15:44), Ashutosh Bapat wrote: > >>>> Hi Tomonary, > >>>> The testcase you have written uses a shell script. This is not the > >>> standard > >>>> way to add testcases. Can you please see if the testcase can be added > >>>> directly as SQL? Please send a revised patch ASAP, so that it can be > >>>> committed. I would have liked to see more testcases for EXEC DIRECT, > but > >>>> since it's a debug only feature (not for production), it gets lesser > >>>> priority in testing and thus we do not see testcases for it. > >>>> > >>>> > >>>> On Thu, Sep 26, 2013 at 8:26 AM, Tomonari Katsumata < > >>>> kat...@po....****jp <katsumata.tomonari@po.ntts.** > co.jp <kat...@po...>>> > > >>> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> Does anyone check these patches? > >>>>> If there are no comments, please commit it. > >>>>> > >>>>> regards, > >>>>> > >>>>> ---------------------- > >>>>> NTT Software Corporation > >>>>> Tomonari Katsumata > >>>>> > >>>>> -------- Original Message -------- > >>>>> Subject: Re: [Postgres-xc-developers] Query Planning bug ? > >>>>> Date: Thu, 5 Sep 2013 21:34:32 +0900 > >>>>> From: Tomonari Katsumata <t.k...@gm...> > >>>>> To: Tomonari Katsumata <katsumata.tomonari@po.ntts.******co.jp< > http://co.jp> > >>> <katsumata.tomonari@po.**ntts.**co.jp <http://ntts.co.jp> < > katsumata.tomonari@po.ntts.**co.jp <kat...@po...>>> > >>>>>> > >>>>> CC: Ashutosh Bapat <ashutosh.bapat@enterprisedb.******com< > >>> ashutosh.bapat@**enterprisedb.**com <http://enterprisedb.com> < > ashutosh.bapat@enterprisedb.**com <ash...@en...>>>>, > >>>>> pgxc-hackers mailing list <postgres-xc-developers@lists.****** > >>> sourceforge.net<postgres-xc-****dev...@li...urceforge.****net< > postgres-xc-developers@**lists.sourceforge.net<pos...@li...> > > > > >>>> > >>> > >>>>>> > >>>>> > >>>>> > >>>>> Hi, > >>>>> > >>>>> I'm sorry for long long leaving this thread alone. > >>>>> I've devided the patch to source part and regression part. > >>>>> (against f8e0f994bff1faefa9b1fc2f6e584a******1d3de70241) > > >>> > >>>>> > >>>>> I did not change any process on source part. > >>>>> (revised_fqs_r4.patch) > >>>>> > >>>>> I added a script into src/test/regress/sql directory for > >>>>> changing regression part. > >>>>> (add_test_xc_misc.patch and execute_direct_regress.sh) > >>>>> When regression test runs, the script is executed instead of > >>>>> user defined function. > >>>>> > >>>>> Please check and commit it. > >>>>> > >>>>> regards, > >>>>> ------------------------ > >>>>> NTT Software Corporation > >>>>> Tomonari Katsumata > >>>>> > >>>>> > >>>>> 2013/8/14 Tomonari Katsumata <katsumata.tomonari@po.ntts.******co.jp > <http://co.jp> > >>> <katsumata.tomonari@po.**ntts.**co.jp <http://ntts.co.jp> < > katsumata.tomonari@po.ntts.**co.jp <kat...@po...>>> > > >>> > >>>>>> > >>>>> > >>>>>> Hi Ashtosh, > >>>>>> > >>>>>> Thanks for your comment! > >>>>>> > >>>>>> > >>>>>>> The code changes are fine, but, the comment in > >>>>> pgxc_collect_RTE_walker() > >>>>>>> seems to specific. I think it should read, "create a copy of > query's > >>>>>> range > >>>>>>> table, so that it can be linked with other RTEs in the collector's > >>>>>> context." > >>>>>>> > >>>>>> I've revised the comment as you suggested. > >>>>>> > >>>>>> > >>>>>>> In the test file, isn't there any way, you can add the offending > >>>>> statent > >>>>>>> without any wrapper function test_execute_direct_all_xc_******* > > >>> *nodes()? > >>> > >>>>> We > >>>>> > >>>>>> should > >>>>>>> minimize the test-code to use only the minimal set of features. > Since > >>>>>> this > >>>>>>> statement no more gets into infinite loop, please do not use the > >>>>>> statement > >>>>>>> timeout. > >>>>>> > >>>>>> I've gotten rid of using statement timeout. > >>>>>> But I couldn't come up any idea to testing more simply, > >>>>>> so the function is remaining as same as before patch. > >>>>>> Any ideas? > >>>>>> > >>>>>> Here is the new patch. > >>>>>> (against dec40008b3d689911566514614c511********1c0a61327d) > > >>> > >>>>> > >>>>>> > >>>>>> > >>>>>> regards, > >>>>>> ------------- > >>>>>> NTT Software Corporation > >>>>>> Tomonari Katsumata > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------****----------------------------**--** > > >>> ------------------ > >>>>> October Webinars: Code for Performance > >>>>> Free Intel webinars can help you accelerate application performance. > >>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the > >>> most > >>>>> from > >>>>> the latest Intel processors and coprocessors. See abstracts and > >>> register > > >>>>> http://pubads.g.doubleclick.****net/gampad/clk?id=60133471&iu=**** > >>> /4140/ostg.clktrk<http://**pubads.g.doubleclick.net/** > gampad/clk?id=60133471&iu=/**4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> > > > > >>>>> ______________________________****_________________ > >>>>> Postgres-xc-developers mailing list > >>>>> Postgres-xc-developers@lists.****sourceforge.net<Postgres-xc-** > dev...@li...urceforge.**net<Pos...@li...> > > > >>>>> https://lists.sourceforge.net/****lists/listinfo/postgres-xc-****<https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**> > >>> developers<https://lists.**sourceforge.net/lists/** > listinfo/postgres-xc-**developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> > > > > >>>>> > >>>>> > >>>> > >>>> > >>> > >>> > >>> > >> > >> > >> -- > >> Best Wishes, > >> Ashutosh Bapat > >> EnterpriseDB Corporation > >> The Postgres Database Company > >> > >> > >> ------------------------------**------------------------------** > ------------------ > >> October Webinars: Code for Performance > >> Free Intel webinars can help you accelerate application performance. > >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > >> from > >> the latest Intel processors and coprocessors. See abstracts and > register > > >> http://pubads.g.doubleclick.**net/gampad/clk?id=60133471&iu=** > /4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> > >> ______________________________**_________________ > >> Postgres-xc-developers mailing list > >> Postgres-xc-developers@lists.**sourceforge.net<Pos...@li...> > >> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> > >> > >> > > > > > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company |
|
From: 鈴木 幸市 <ko...@in...> - 2013-10-30 01:50:02
|
Ashutosh; I understand that there is no remaining issue in his patch. If so, I think we should commit it timely. Best; --- Koichi Suzuki On 2013/10/30, at 10:21, Tomonari Katsumata <kat...@po...> wrote: > Hi, > > Almost one month is passing without any > comments about this thread. > > I need this fix as soon as possible, because > these kinds queries are executed by > pg_statsinfo for Postgres-XC. > > Anybody, please commit it. > > regards, > ------------ > Tomonari Katsumata > > (2013/09/28 10:53), Tomonari Katsumata wrote: >> Hi Ashotosh, >> >>> I never said "don't use functions". Here's what I mailed >>> >> Oh, Sorry for bothering you by my misunderstanding. >> >>>> What is better for this testcase? >>> >>> May be we should just drop the testcase. Anyway, EXEC DIRECT is not >> production feature, but debug feature and whosoever wrote this > feature has >> not added any testcases for it anyway. >>> If anyone else has objections to dropping the testcase, please > suggest >> Tomonari, a better to write it. >>> >> Past 2 days, any suggestions were not sent to me. >> So it seems no objections to dropping the testcase. >> >> regards, >> ---------------- >> NTT Software Corporation >> Tomonari Katsumata >> >> >> >> 2013/9/26 Ashutosh Bapat <ash...@en...> >> >>> >>> >>> >>> On Thu, Sep 26, 2013 at 1:29 PM, Tomonari Katsumata < >>> kat...@po...> wrote: >>> >>>> Hi Ashtosh, >>>> >>>> Thanks for your review! >>>> >>>> Previous patch(revised_fqs_r3.patch in [2013/08/14 19:55]) >>>> doesn't use a shell script. >>>> Instead, It used test_execute_direct_all_xc_**nodes() function. >>>> >>>> I've got a comment "don't use functions" from you, >>>> but I could not come up with good idea for the testcase. >>>> The testcase needs to get all node_names to "EXECUTE DIRECT" >>>> against all nodes. >>>> >>>> >>> I never said "don't use functions". Here's what I mailed >>> >>> "In the test file, isn't there any way, you can add the offending > statent >>> without any wrapper function test_execute_direct_all_xc_ >>> nodes()? We should minimize the test-code to use only the minimal set of >>> features. Since this statement no more gets into infinite loop, > please do >>> not use the statement timeout." >>> >>> >>>> What is better for this testcase? >>>> >>>> >>> May be we should just drop the testcase. Anyway, EXEC DIRECT is not >>> production feature, but debug feature and whosoever wrote this > feature has >>> not added any testcases for it anyway. >>> >>> If anyone else has objections to dropping the testcase, please suggest >>> Tomonari, a better to write it. >>> >>> >>>> regards, >>>> >>>> ---------------- >>>> NTT Software Corporation >>>> Tomonari Katsumata >>>> >>>> (2013/09/26 15:44), Ashutosh Bapat wrote: >>>>> Hi Tomonary, >>>>> The testcase you have written uses a shell script. This is not the >>>> standard >>>>> way to add testcases. Can you please see if the testcase can be added >>>>> directly as SQL? Please send a revised patch ASAP, so that it can be >>>>> committed. I would have liked to see more testcases for EXEC > DIRECT, but >>>>> since it's a debug only feature (not for production), it gets lesser >>>>> priority in testing and thus we do not see testcases for it. >>>>> >>>>> >>>>> On Thu, Sep 26, 2013 at 8:26 AM, Tomonari Katsumata < >>>>> kat...@po....**jp <kat...@po...>> >>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Does anyone check these patches? >>>>>> If there are no comments, please commit it. >>>>>> >>>>>> regards, >>>>>> >>>>>> ---------------------- >>>>>> NTT Software Corporation >>>>>> Tomonari Katsumata >>>>>> >>>>>> -------- Original Message -------- >>>>>> Subject: Re: [Postgres-xc-developers] Query Planning bug ? >>>>>> Date: Thu, 5 Sep 2013 21:34:32 +0900 >>>>>> From: Tomonari Katsumata <t.k...@gm...> >>>>>> To: Tomonari Katsumata > <katsumata.tomonari@po.ntts.****co.jp<http://co.jp> >>>> <katsumata.tomonari@po.**ntts.co.jp <kat...@po...>> >>>>>>> >>>>>> CC: Ashutosh Bapat <ashutosh.bapat@enterprisedb.****com< >>>> ashutosh.bapat@**enterprisedb.com <ash...@en...>>>, >>>>>> pgxc-hackers mailing list <postgres-xc-developers@lists.**** >>>> > sourceforge.net<postgres-xc-**dev...@li...urceforge.**net<pos...@li...> >>>>> >>>> >>>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm sorry for long long leaving this thread alone. >>>>>> I've devided the patch to source part and regression part. >>>>>> (against f8e0f994bff1faefa9b1fc2f6e584a****1d3de70241) >>>> >>>>>> >>>>>> I did not change any process on source part. >>>>>> (revised_fqs_r4.patch) >>>>>> >>>>>> I added a script into src/test/regress/sql directory for >>>>>> changing regression part. >>>>>> (add_test_xc_misc.patch and execute_direct_regress.sh) >>>>>> When regression test runs, the script is executed instead of >>>>>> user defined function. >>>>>> >>>>>> Please check and commit it. >>>>>> >>>>>> regards, >>>>>> ------------------------ >>>>>> NTT Software Corporation >>>>>> Tomonari Katsumata >>>>>> >>>>>> >>>>>> 2013/8/14 Tomonari Katsumata > <katsumata.tomonari@po.ntts.****co.jp<http://co.jp> >>>> <katsumata.tomonari@po.**ntts.co.jp <kat...@po...>> >>>> >>>>>>> >>>>>> >>>>>>> Hi Ashtosh, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> >>>>>>> >>>>>>>> The code changes are fine, but, the comment in >>>>>> pgxc_collect_RTE_walker() >>>>>>>> seems to specific. I think it should read, "create a copy of > query's >>>>>>> range >>>>>>>> table, so that it can be linked with other RTEs in the collector's >>>>>>> context." >>>>>>>> >>>>>>> I've revised the comment as you suggested. >>>>>>> >>>>>>> >>>>>>>> In the test file, isn't there any way, you can add the offending >>>>>> statent >>>>>>>> without any wrapper function test_execute_direct_all_xc_***** >>>> *nodes()? >>>> >>>>>> We >>>>>> >>>>>>> should >>>>>>>> minimize the test-code to use only the minimal set of features. > Since >>>>>>> this >>>>>>>> statement no more gets into infinite loop, please do not use the >>>>>>> statement >>>>>>>> timeout. >>>>>>> >>>>>>> I've gotten rid of using statement timeout. >>>>>>> But I couldn't come up any idea to testing more simply, >>>>>>> so the function is remaining as same as before patch. >>>>>>> Any ideas? >>>>>>> >>>>>>> Here is the new patch. >>>>>>> (against dec40008b3d689911566514614c511******1c0a61327d) >>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> regards, >>>>>>> ------------- >>>>>>> NTT Software Corporation >>>>>>> Tomonari Katsumata >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------**------------------------------** >>>> ------------------ >>>>>> October Webinars: Code for Performance >>>>>> Free Intel webinars can help you accelerate application performance. >>>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>>> most >>>>>> from >>>>>> the latest Intel processors and coprocessors. See abstracts and >>>> register > >>>>>> http://pubads.g.doubleclick.**net/gampad/clk?id=60133471&iu=** >>>> > /4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> >>>>>> ______________________________**_________________ >>>>>> Postgres-xc-developers mailing list >>>>>> > Postgres-xc-developers@lists.**sourceforge.net<Pos...@li...> >>>>>> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-** >>>> > developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Best Wishes, >>> Ashutosh Bapat >>> EnterpriseDB Corporation >>> The Postgres Database Company >>> >>> >>> > ------------------------------------------------------------------------------ >>> October Webinars: Code for Performance >>> Free Intel webinars can help you accelerate application performance. >>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >>> from >>> the latest Intel processors and coprocessors. See abstracts and > register > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >>> >> > > > > ------------------------------------------------------------------------------ > Android is increasing in popularity, but the open development platform that > developers love is also attractive to malware creators. Download this white > paper to learn more about secure code signing practices that can help keep > Android apps secure. > http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: Tomonari K. <kat...@po...> - 2013-10-30 01:22:08
|
Hi, Almost one month is passing without any comments about this thread. I need this fix as soon as possible, because these kinds queries are executed by pg_statsinfo for Postgres-XC. Anybody, please commit it. regards, ------------ Tomonari Katsumata (2013/09/28 10:53), Tomonari Katsumata wrote: > Hi Ashotosh, > >> I never said "don't use functions". Here's what I mailed >> > Oh, Sorry for bothering you by my misunderstanding. > >>> What is better for this testcase? >> >> May be we should just drop the testcase. Anyway, EXEC DIRECT is not > production feature, but debug feature and whosoever wrote this feature has > not added any testcases for it anyway. >> If anyone else has objections to dropping the testcase, please suggest > Tomonari, a better to write it. >> > Past 2 days, any suggestions were not sent to me. > So it seems no objections to dropping the testcase. > > regards, > ---------------- > NTT Software Corporation > Tomonari Katsumata > > > > 2013/9/26 Ashutosh Bapat <ash...@en...> > >> >> >> >> On Thu, Sep 26, 2013 at 1:29 PM, Tomonari Katsumata < >> kat...@po...> wrote: >> >>> Hi Ashtosh, >>> >>> Thanks for your review! >>> >>> Previous patch(revised_fqs_r3.patch in [2013/08/14 19:55]) >>> doesn't use a shell script. >>> Instead, It used test_execute_direct_all_xc_**nodes() function. >>> >>> I've got a comment "don't use functions" from you, >>> but I could not come up with good idea for the testcase. >>> The testcase needs to get all node_names to "EXECUTE DIRECT" >>> against all nodes. >>> >>> >> I never said "don't use functions". Here's what I mailed >> >> "In the test file, isn't there any way, you can add the offending statent >> without any wrapper function test_execute_direct_all_xc_ >> nodes()? We should minimize the test-code to use only the minimal set of >> features. Since this statement no more gets into infinite loop, please do >> not use the statement timeout." >> >> >>> What is better for this testcase? >>> >>> >> May be we should just drop the testcase. Anyway, EXEC DIRECT is not >> production feature, but debug feature and whosoever wrote this feature has >> not added any testcases for it anyway. >> >> If anyone else has objections to dropping the testcase, please suggest >> Tomonari, a better to write it. >> >> >>> regards, >>> >>> ---------------- >>> NTT Software Corporation >>> Tomonari Katsumata >>> >>> (2013/09/26 15:44), Ashutosh Bapat wrote: >>>> Hi Tomonary, >>>> The testcase you have written uses a shell script. This is not the >>> standard >>>> way to add testcases. Can you please see if the testcase can be added >>>> directly as SQL? Please send a revised patch ASAP, so that it can be >>>> committed. I would have liked to see more testcases for EXEC DIRECT, but >>>> since it's a debug only feature (not for production), it gets lesser >>>> priority in testing and thus we do not see testcases for it. >>>> >>>> >>>> On Thu, Sep 26, 2013 at 8:26 AM, Tomonari Katsumata < >>>> kat...@po....**jp <kat...@po...>> >>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Does anyone check these patches? >>>>> If there are no comments, please commit it. >>>>> >>>>> regards, >>>>> >>>>> ---------------------- >>>>> NTT Software Corporation >>>>> Tomonari Katsumata >>>>> >>>>> -------- Original Message -------- >>>>> Subject: Re: [Postgres-xc-developers] Query Planning bug ? >>>>> Date: Thu, 5 Sep 2013 21:34:32 +0900 >>>>> From: Tomonari Katsumata <t.k...@gm...> >>>>> To: Tomonari Katsumata <katsumata.tomonari@po.ntts.****co.jp<http://co.jp> >>> <katsumata.tomonari@po.**ntts.co.jp <kat...@po...>> >>>>>> >>>>> CC: Ashutosh Bapat <ashutosh.bapat@enterprisedb.****com< >>> ashutosh.bapat@**enterprisedb.com <ash...@en...>>>, >>>>> pgxc-hackers mailing list <postgres-xc-developers@lists.**** >>> sourceforge.net<postgres-xc-**dev...@li...urceforge.**net<pos...@li...> >>>> >>> >>>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I'm sorry for long long leaving this thread alone. >>>>> I've devided the patch to source part and regression part. >>>>> (against f8e0f994bff1faefa9b1fc2f6e584a****1d3de70241) >>> >>>>> >>>>> I did not change any process on source part. >>>>> (revised_fqs_r4.patch) >>>>> >>>>> I added a script into src/test/regress/sql directory for >>>>> changing regression part. >>>>> (add_test_xc_misc.patch and execute_direct_regress.sh) >>>>> When regression test runs, the script is executed instead of >>>>> user defined function. >>>>> >>>>> Please check and commit it. >>>>> >>>>> regards, >>>>> ------------------------ >>>>> NTT Software Corporation >>>>> Tomonari Katsumata >>>>> >>>>> >>>>> 2013/8/14 Tomonari Katsumata <katsumata.tomonari@po.ntts.****co.jp<http://co.jp> >>> <katsumata.tomonari@po.**ntts.co.jp <kat...@po...>> >>> >>>>>> >>>>> >>>>>> Hi Ashtosh, >>>>>> >>>>>> Thanks for your comment! >>>>>> >>>>>> >>>>>>> The code changes are fine, but, the comment in >>>>> pgxc_collect_RTE_walker() >>>>>>> seems to specific. I think it should read, "create a copy of query's >>>>>> range >>>>>>> table, so that it can be linked with other RTEs in the collector's >>>>>> context." >>>>>>> >>>>>> I've revised the comment as you suggested. >>>>>> >>>>>> >>>>>>> In the test file, isn't there any way, you can add the offending >>>>> statent >>>>>>> without any wrapper function test_execute_direct_all_xc_***** >>> *nodes()? >>> >>>>> We >>>>> >>>>>> should >>>>>>> minimize the test-code to use only the minimal set of features. Since >>>>>> this >>>>>>> statement no more gets into infinite loop, please do not use the >>>>>> statement >>>>>>> timeout. >>>>>> >>>>>> I've gotten rid of using statement timeout. >>>>>> But I couldn't come up any idea to testing more simply, >>>>>> so the function is remaining as same as before patch. >>>>>> Any ideas? >>>>>> >>>>>> Here is the new patch. >>>>>> (against dec40008b3d689911566514614c511******1c0a61327d) >>> >>>>> >>>>>> >>>>>> >>>>>> regards, >>>>>> ------------- >>>>>> NTT Software Corporation >>>>>> Tomonari Katsumata >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------**------------------------------** >>> ------------------ >>>>> October Webinars: Code for Performance >>>>> Free Intel webinars can help you accelerate application performance. >>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the >>> most >>>>> from >>>>> the latest Intel processors and coprocessors. See abstracts and >>> register > >>>>> http://pubads.g.doubleclick.**net/gampad/clk?id=60133471&iu=** >>> /4140/ostg.clktrk<http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk> >>>>> ______________________________**_________________ >>>>> Postgres-xc-developers mailing list >>>>> Postgres-xc-developers@lists.**sourceforge.net<Pos...@li...> >>>>> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-** >>> developers<https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers> >>>>> >>>>> >>>> >>>> >>> >>> >>> >> >> >> -- >> Best Wishes, >> Ashutosh Bapat >> EnterpriseDB Corporation >> The Postgres Database Company >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > |
|
From: 鈴木 幸市 <ko...@in...> - 2013-10-25 08:59:23
|
I think error logs of communication errors between GTM and GTM standby can be written at GTM master as well. Please let me allow some more time to submit the patch for it. --- Koichi Suzuki On 2013/10/25, at 0:00, Nikhil Sontakke <ni...@st...<mailto:ni...@st...>> wrote: Seems like a good thing to add in the GTM daemons (covers GTM Standby as well?). So +1 from me. Regards, Nikhils On Thu, Oct 24, 2013 at 6:43 AM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote: Hi, Libpq for GTM and GTM_Proxy has a feature to store its own error messages to gtm_conn structure. Because Libpq is to be linked to any other applications, GTM libpq itself does not call elog(). However, modules calling gtm libpq can do this. I found we can add this feature to gtm_proxy's proxy_main.c and backend's gtm.c and hope this helps to analyze what's going on in gtm communications. If I have reasonable approval, I'd like to submit a patch for this. Regards; --- Koichi Suzuki ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk _______________________________________________ Postgres-xc-developers mailing list Pos...@li...<mailto:Pos...@li...> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers -- StormDB - http://www.stormdb.com<http://www.stormdb.com/> The Database Cloud ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk_______________________________________________ Postgres-xc-developers mailing list Pos...@li... https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: Koichi S. <koi...@gm...> - 2013-10-25 03:32:38
|
Thanks. I wrote a preliminary patch and found it seems to work fine. I'm now inserting sleep between retries if we have to retry many times. Will see the patch soon. Regards; --- Koichi Suzuki 2013/10/23 Nikhil Sontakke <ni...@st...> > +1 > > I had diagnosed this same issue quite a while ago. We should wait for the > buffer to drain out before adding more into it. The buffer size was going > beyond 1GB in size pretty quickly! > > Regards, > Nikhils > > > On Wed, Oct 23, 2013 at 10:35 AM, Koichi Suzuki <koi...@gm...>wrote: > >> I've found that in copy command, we have a risk to overflow the buffer in >> coordinator if datanode is very slow (for various reasons). >> >> When we issue copy command against tens of gigabytes of data, coordinator >> send all of then without synch. When some of the data fails to send, it >> adds next data and tries to send all of them. When a datanode is very >> slow (for any reason), the data will continue to be cashed at the >> coordinator and finally overflows. >> >> The patch avoids this issue. When amount of unsent data exceeds the >> criteria, coordinator will keep retry until this amount is below the second >> criteria. >> >> This can be applied to all the releases, as well as the master. >> >> Regards; >> --- >> Koichi Suzuki >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > |
|
From: Koichi S. <koi...@gm...> - 2013-10-25 03:30:35
|
I think error logs of communication errors between GTM and GTM standby can be written at GTM master as well. Please let me allow some more time to submit the patch for it. --- Koichi Suzuki 2013/10/25 Nikhil Sontakke <ni...@st...> > Seems like a good thing to add in the GTM daemons (covers GTM Standby as > well?). So +1 from me. > > Regards, > Nikhils > > > On Thu, Oct 24, 2013 at 6:43 AM, Koichi Suzuki <koi...@gm...>wrote: > >> Hi, >> >> Libpq for GTM and GTM_Proxy has a feature to store its own error messages >> to gtm_conn structure. Because Libpq is to be linked to any other >> applications, GTM libpq itself does not call elog(). However, modules >> calling gtm libpq can do this. >> >> I found we can add this feature to gtm_proxy's proxy_main.c and backend's >> gtm.c and hope this helps to analyze what's going on in gtm communications. >> >> If I have reasonable approval, I'd like to submit a patch for this. >> >> Regards; >> --- >> Koichi Suzuki >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > |
|
From: Michael P. <mic...@gm...> - 2013-10-24 22:59:28
|
On Fri, Oct 25, 2013 at 1:14 AM, Matt Warner <MW...@xi...> wrote: > Good morning. I didn't see any responses to this. > > I know everyone is busy. Were the changes acceptable? Sorry, I'm quite busy the last couple of weeks, so I got no time to look at your patches. -- Michael |
|
From: Matt W. <MW...@xi...> - 2013-10-24 16:16:17
|
Good morning. I didn't see any responses to this. I know everyone is busy. Were the changes acceptable? Matt -----Original Message----- From: Matt Warner [mailto:MW...@XI...] Sent: Wednesday, October 09, 2013 4:29 PM To: 'Michael Paquier' Cc: Postgres-XC Developers; Suzuki; Koichi Subject: Re: [Postgres-xc-developers] Minor Fixes Below is a summary of the changes made against git pull from this morning. I don't claim to be the best resource for this, so please do take the time to verify what I've sent. I also welcome comments and suggestions. Please note that the entire config directory was replaced with the one from Postgres 9.3.0. Likewise, I used configure.in from Postgres 9.3.0 as the starting point for the updates. autoconf successfully creates the configure file, which is also attached. I've tried to capture all the XC pieces that had been added manually to the previous configure file, but the new configure file should be checked and verified. In particular, I moved the '-DPGXC' from configure to pgxc_config.h and updated postgres.h to include this new header. The changes all compile successfully under Solaris 10 using the Solaris Studio compiler. Being that the Solaris compiler seems to be more demanding than gcc about syntax and "correctness", I believe this is a good sign. I also ran this with test data and saw no issues. FYI, Matt % git status # On branch master # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: src/include/pgxc_config.h # new file: src/include/pgxc_config.h.in # new file: src/include/stamp-ext-h # # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: config/c-compiler.m4 # modified: config/config.guess # modified: config/config.sub # modified: config/install-sh # modified: config/perl.m4 # modified: config/programs.m4 # modified: config/python.m4 # modified: configure # modified: configure.in # modified: src/backend/pgxc/pool/pgxcnode.c # modified: src/include/postgres.h # -----Original Message----- From: Michael Paquier [mailto:mic...@gm...] Sent: Tuesday, October 08, 2013 5:43 PM To: Matt Warner Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki Subject: Re: [Postgres-xc-developers] Minor Fixes On Wed, Oct 9, 2013 at 9:13 AM, Matt Warner <MW...@xi...> wrote: > So I owe you a different patch for proxy_main.c as well. Btw, I can see why it is needed for pgxcnode.h as we use FIONREAD there, but why is it necessary to have an include to filio.h in proxy_main.c? I couldn't figure out why so I simply removed that from the patch committed. -- Michael |
|
From: Nikhil S. <ni...@st...> - 2013-10-24 15:00:43
|
Seems like a good thing to add in the GTM daemons (covers GTM Standby as well?). So +1 from me. Regards, Nikhils On Thu, Oct 24, 2013 at 6:43 AM, Koichi Suzuki <koi...@gm...>wrote: > Hi, > > Libpq for GTM and GTM_Proxy has a feature to store its own error messages > to gtm_conn structure. Because Libpq is to be linked to any other > applications, GTM libpq itself does not call elog(). However, modules > calling gtm libpq can do this. > > I found we can add this feature to gtm_proxy's proxy_main.c and backend's > gtm.c and hope this helps to analyze what's going on in gtm communications. > > If I have reasonable approval, I'd like to submit a patch for this. > > Regards; > --- > Koichi Suzuki > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Koichi S. <koi...@gm...> - 2013-10-24 01:13:41
|
Hi, Libpq for GTM and GTM_Proxy has a feature to store its own error messages to gtm_conn structure. Because Libpq is to be linked to any other applications, GTM libpq itself does not call elog(). However, modules calling gtm libpq can do this. I found we can add this feature to gtm_proxy's proxy_main.c and backend's gtm.c and hope this helps to analyze what's going on in gtm communications. If I have reasonable approval, I'd like to submit a patch for this. Regards; --- Koichi Suzuki |
|
From: Nikhil S. <ni...@st...> - 2013-10-23 07:19:34
|
+1 I had diagnosed this same issue quite a while ago. We should wait for the buffer to drain out before adding more into it. The buffer size was going beyond 1GB in size pretty quickly! Regards, Nikhils On Wed, Oct 23, 2013 at 10:35 AM, Koichi Suzuki <koi...@gm...>wrote: > I've found that in copy command, we have a risk to overflow the buffer in > coordinator if datanode is very slow (for various reasons). > > When we issue copy command against tens of gigabytes of data, coordinator > send all of then without synch. When some of the data fails to send, it > adds next data and tries to send all of them. When a datanode is very > slow (for any reason), the data will continue to be cashed at the > coordinator and finally overflows. > > The patch avoids this issue. When amount of unsent data exceeds the > criteria, coordinator will keep retry until this amount is below the second > criteria. > > This can be applied to all the releases, as well as the master. > > Regards; > --- > Koichi Suzuki > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Koichi S. <koi...@gm...> - 2013-10-22 02:12:32
|
(Please take a look at WIP branch master_pg93_merge).
I reviewed this patch and found that this patch handles CLEAN CONNECTION as
simply NOP.
To disable event trigger for some DDLs, we need to move the code from
ProcessUtilitySlow() to Standard_ProcessUtility(). My previous patch was
not sufficient. We need to move the code for XC-specific DDLs in this
manner.
I don't think this change affects regression test.
DDL handling was done as follows:
(outside) --> ProcessUtility() /* ProcessUtility_hook is handled here */
--> standard_ProcessUtility() /* DDLs without event
trigger support are handled here */
--> ProcessUtilitySlow() /* DDLs with event trigger
support are handled here */
3_1_clean_con.patch just returns from standard_ProcessUtility() without
doing anything. Unfortunately, CLEAN CONNECTION HANDLER is now in
ProcessUtilitySlow() and this should be moved back to
standard_ProcessUtility().
Any thoughts?
---
Koichi Suzuki
2013/9/30 Abbas Butt <abb...@en...>
> Here is the patch, forgot to attach it with the previous mail.
> Regards
>
>
>
> On Mon, Sep 30, 2013 at 7:23 AM, Abbas Butt <abb...@en...>
> wrote:
> >
> > Hi,
> > The current implementation of event triggers assumes that any statment
> > that is not handeled by the main switch-case in standard_ProcessUtility
> > supports event triggers.
> > CLEAN CONNECTION is a new statement added by XC and does not support
> > event triggers as of now.
> >
> > This patch is intended for the master_pg93_merge branch and applies
> cleanly
> > on top of the following two patches.
> > 20130924_01.patch and fix_for_pooler_socket_20130926.patch
> >
> > Regards
> > --
> > Abbas
> > Architect
> >
> > Ph: 92.334.5100153
> > Skype ID: gabbasb
> > www.enterprisedb.com
> >
> > Follow us on Twitter
> > @EnterpriseDB
> >
> > Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
>
>
>
> --
> --
> Abbas
> Architect
>
> Ph: 92.334.5100153
> Skype ID: gabbasb
> www.enterprisedb.com
>
> Follow us on Twitter
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
>
> On Mon, Sep 30, 2013 at 7:23 AM, Abbas Butt <abb...@en...>
> wrote:
> >
> > Hi,
> > The current implementation of event triggers assumes that any statment
> > that is not handeled by the main switch-case in standard_ProcessUtility
> > supports event triggers.
> > CLEAN CONNECTION is a new statement added by XC and does not support
> > event triggers as of now.
> >
> > This patch is intended for the master_pg93_merge branch and applies
> cleanly
> > on top of the following two patches.
> > 20130924_01.patch and fix_for_pooler_socket_20130926.patch
> >
> > Regards
> > --
> > Abbas
> > Architect
> >
> > Ph: 92.334.5100153
> > Skype ID: gabbasb
> > www.enterprisedb.com
> >
> > Follow us on Twitter
> > @EnterpriseDB
> >
> > Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
>
>
>
> --
> --
> Abbas
> Architect
>
> Ph: 92.334.5100153
> Skype ID: gabbasb
> www.enterprisedb.com
>
> Follow us on Twitter
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
|
|
From: Masataka S. <pg...@gm...> - 2013-10-10 08:11:11
|
Hi, all. I observed wrong sorted results on a SELECT query with COLLATE clause. test=# SELECT * FROM FOOBAR ORDER BY all_alpha COLLATE "en_US"; all_alpha ----------- A b B z Z a (6 rows) Related ticket is here. https://sourceforge.net/p/postgres-xc/bugs/458/ Please refer the ticket for more details including instructions to reproduce. Regards. |
|
From: Matt W. <MW...@XI...> - 2013-10-09 23:29:59
|
Below is a summary of the changes made against git pull from this morning. I don't claim to be the best resource for this, so please do take the time to verify what I've sent. I also welcome comments and suggestions. Please note that the entire config directory was replaced with the one from Postgres 9.3.0. Likewise, I used configure.in from Postgres 9.3.0 as the starting point for the updates. autoconf successfully creates the configure file, which is also attached. I've tried to capture all the XC pieces that had been added manually to the previous configure file, but the new configure file should be checked and verified. In particular, I moved the '-DPGXC' from configure to pgxc_config.h and updated postgres.h to include this new header. The changes all compile successfully under Solaris 10 using the Solaris Studio compiler. Being that the Solaris compiler seems to be more demanding than gcc about syntax and "correctness", I believe this is a good sign. I also ran this with test data and saw no issues. FYI, Matt % git status # On branch master # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: src/include/pgxc_config.h # new file: src/include/pgxc_config.h.in # new file: src/include/stamp-ext-h # # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: config/c-compiler.m4 # modified: config/config.guess # modified: config/config.sub # modified: config/install-sh # modified: config/perl.m4 # modified: config/programs.m4 # modified: config/python.m4 # modified: configure # modified: configure.in # modified: src/backend/pgxc/pool/pgxcnode.c # modified: src/include/postgres.h # -----Original Message----- From: Michael Paquier [mailto:mic...@gm...] Sent: Tuesday, October 08, 2013 5:43 PM To: Matt Warner Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki Subject: Re: [Postgres-xc-developers] Minor Fixes On Wed, Oct 9, 2013 at 9:13 AM, Matt Warner <MW...@xi...> wrote: > So I owe you a different patch for proxy_main.c as well. Btw, I can see why it is needed for pgxcnode.h as we use FIONREAD there, but why is it necessary to have an include to filio.h in proxy_main.c? I couldn't figure out why so I simply removed that from the patch committed. -- Michael |
|
From: Matt W. <MW...@XI...> - 2013-10-09 17:13:27
|
I've checked out latest with git and as you point out, there's no need for filio.h in proxy_main.c. There is a need for it in backend/pgxc/pgxcnode.c, so I've put in an "#ifdef HAVE_SYS_FILIO_H" to conditionally include filio.h. As soon as I complete a successful compilation of the latest pull from git and do some testing, I'll send the changes. Matt -----Original Message----- From: Michael Paquier [mailto:mic...@gm...] Sent: Tuesday, October 08, 2013 5:43 PM To: Matt Warner Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki Subject: Re: [Postgres-xc-developers] Minor Fixes On Wed, Oct 9, 2013 at 9:13 AM, Matt Warner <MW...@xi...> wrote: > So I owe you a different patch for proxy_main.c as well. Btw, I can see why it is needed for pgxcnode.h as we use FIONREAD there, but why is it necessary to have an include to filio.h in proxy_main.c? I couldn't figure out why so I simply removed that from the patch committed. -- Michael |
|
From: Masataka S. <pg...@gm...> - 2013-10-09 09:28:21
|
I made a ticket for this issue. https://sourceforge.net/p/postgres-xc/bugs/457/ The test case is also failing to XC 1.0.0. XC 1.0.0 returns same error code as XC on the master branch. The test is introduced only to master branch after branch REL9_2_STABLE, which I use to test XC 1.1 was created. It is why REL1_1_STABLE branch passes the JDBC regression test. Regards. On Mon, Oct 7, 2013 at 11:42 PM, Ahsan Hadi <ahs...@en...> wrote: > Thanks Masataka. > > Can you create a source-forge ticket for this please? Attach a reproducible > test case with the ticket. > > > > On Thu, Oct 3, 2013 at 2:45 PM, Masataka Saito <pg...@gm...> wrote: >> >> Hello, all. >> >> I found Postgres-XC returns different error code from PostgreSQL's one >> for repeated rollback, and it makes a pgjdbc's regression test case >> fail. >> >> PostgreSQL returns error code 42704 which is defined as >> ERRCODE_UNDEFINED_OBJECT. It is maybe returned in >> src/backend/access/transam/twophase.c::LockGXact. >> Postgres-XC returns error code XX000 which is defined as >> ERRCODE_INTERNAL_ERROR It is maybe returned in >> src/backend/pgxc/pool/execRemote.c::FinishRemotePreparedTransaction. >> >> Error messages are quite similar. I will show it at the next line. >> > prepared transaction with identifier >> > "0_8fHx8fF5/OCyZd9xZmJ2MSPCE1olci037CNQYoXoVCKbi63JcZIoUeJxU7PKl/t4qI4xV1agFPE+9gRrzheGHA==_BAUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==" >> > does not exist. >> >> Does anyone know ... >> * Why they have different error codes, and is it reasonable? >> * To fix the code affects other components? >> >> You can reproduce using pgjdbc's regression test. The source code of >> the test is >> org.postgresql.test.xa.XADataSourceTest.testRepeatedRolledBack. >> > >> > http://sourceforge.net/p/postgres-xc/pgjdbc-xc/ci/master/tree/org/postgresql/test/xa/XADataSourceTest.java#l331 >> >> Regards. >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > -- > Ahsan Hadi > Snr Director Product Development > EnterpriseDB Corporation > The Enterprise Postgres Company > > Phone: +92-51-8358874 > Mobile: +92-333-5162114 > > Website: www.enterprisedb.com > EnterpriseDB Blog: http://blogs.enterprisedb.com/ > Follow us on Twitter: http://www.twitter.com/enterprisedb > > This e-mail message (and any attachment) is intended for the use of the > individual or entity to whom it is addressed. This message contains > information from EnterpriseDB Corporation that may be privileged, > confidential, or exempt from disclosure under applicable law. If you are not > the intended recipient or authorized to receive this for the intended > recipient, any use, dissemination, distribution, retention, archiving, or > copying of this communication is strictly prohibited. If you have received > this e-mail in error, please notify the sender immediately by reply e-mail > and delete this message. |
|
From: Nikhil S. <ni...@st...> - 2013-10-09 08:13:40
|
+1 for an approach using the Primary Key. As Pavan mentioned, maybe Update already works using PK as an identifier so we might have a precedent already. Regards, Nikhils On Wed, Oct 9, 2013 at 12:12 PM, Pavan Deolasee <pav...@gm...>wrote: > On Wed, Oct 9, 2013 at 12:09 PM, Andrei Martsinchyk < > and...@gm...> wrote: > >> I believe even plain vacuum may move tuples within a page and purge empty >> pages, therefore CTIDs could be changed. >> > > Plain vacuum does not move tuples around.. Note that all indexes must be > updated if a tuple is moved, even within a page. But besides that all your > points below still hold true. > > >> Autovacuum on Datamodes are not in sync and concurrent writes may occur >> in different order on Datanodes. >> CTID can not be used to identify tuple replicas. Primary key could be >> used, or special system column. Otherwise not shippable updates against >> replicated tables should not be allowed. >> >> >> 2013/10/9 Pavan Deolasee <pav...@gm...> >> >>> >>> >>> >>> On Wed, Oct 9, 2013 at 11:49 AM, Koichi Suzuki <koi...@gm...>wrote: >>> >>>> Hmm.. Yes, it could be serious. I suppose that CTID changes only when >>>> vacuum full occurs. >>>> >>> >>> CTID will also change every time a row is updated. So the bug should be >>> very easy to reproduce. One easiest way to compile different datanodes with >>> diferent block size. Another way could be to update/delete from a >>> replicated table and vacuum the table on one datanode but not other and >>> then again update several rows in the table. Most likely the new versions >>> on two different nodes will not have the same CTID anymore. >>> >>> >>>> Vacuum freeze will updates xmim to the smallest value and usual vacuum >>>> recycles "dead" row area. We may have a chance to assign different CTIDs >>>> to the same rows in replicated table if vacuum full runs locally in >>>> datanodes. (My assumption may be wrong. Does vacuum freeze move tuples >>>> and change their CTID?) >>>> >>>> >>> No, it does not. >>> >>> Thanks, >>> Pavan >>> >>> -- >>> Pavan Deolasee >>> http://www.linkedin.com/in/pavandeolasee >>> >>> >>> ------------------------------------------------------------------------------ >>> October Webinars: Code for Performance >>> Free Intel webinars can help you accelerate application performance. >>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >>> from >>> the latest Intel processors and coprocessors. See abstracts and register >>> > >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk >>> >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >>> >> >> >> -- >> Andrei Martsinchyk >> >> >> StormDB - http://www.stormdb.com >> The Database Cloud >> >> > > > -- > Pavan Deolasee > http://www.linkedin.com/in/pavandeolasee > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Pavan D. <pav...@gm...> - 2013-10-09 06:42:35
|
On Wed, Oct 9, 2013 at 12:09 PM, Andrei Martsinchyk < and...@gm...> wrote: > I believe even plain vacuum may move tuples within a page and purge empty > pages, therefore CTIDs could be changed. > Plain vacuum does not move tuples around.. Note that all indexes must be updated if a tuple is moved, even within a page. But besides that all your points below still hold true. > Autovacuum on Datamodes are not in sync and concurrent writes may occur in > different order on Datanodes. > CTID can not be used to identify tuple replicas. Primary key could be > used, or special system column. Otherwise not shippable updates against > replicated tables should not be allowed. > > > 2013/10/9 Pavan Deolasee <pav...@gm...> > >> >> >> >> On Wed, Oct 9, 2013 at 11:49 AM, Koichi Suzuki <koi...@gm...>wrote: >> >>> Hmm.. Yes, it could be serious. I suppose that CTID changes only when >>> vacuum full occurs. >>> >> >> CTID will also change every time a row is updated. So the bug should be >> very easy to reproduce. One easiest way to compile different datanodes with >> diferent block size. Another way could be to update/delete from a >> replicated table and vacuum the table on one datanode but not other and >> then again update several rows in the table. Most likely the new versions >> on two different nodes will not have the same CTID anymore. >> >> >>> Vacuum freeze will updates xmim to the smallest value and usual vacuum >>> recycles "dead" row area. We may have a chance to assign different CTIDs >>> to the same rows in replicated table if vacuum full runs locally in >>> datanodes. (My assumption may be wrong. Does vacuum freeze move tuples >>> and change their CTID?) >>> >>> >> No, it does not. >> >> Thanks, >> Pavan >> >> -- >> Pavan Deolasee >> http://www.linkedin.com/in/pavandeolasee >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk >> >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > > -- > Andrei Martsinchyk > > > StormDB - http://www.stormdb.com > The Database Cloud > > -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee |
|
From: Andrei M. <and...@gm...> - 2013-10-09 06:39:46
|
I believe even plain vacuum may move tuples within a page and purge empty pages, therefore CTIDs could be changed. Autovacuum on Datamodes are not in sync and concurrent writes may occur in different order on Datanodes. CTID can not be used to identify tuple replicas. Primary key could be used, or special system column. Otherwise not shippable updates against replicated tables should not be allowed. 2013/10/9 Pavan Deolasee <pav...@gm...> > > > > On Wed, Oct 9, 2013 at 11:49 AM, Koichi Suzuki <koi...@gm...>wrote: > >> Hmm.. Yes, it could be serious. I suppose that CTID changes only when >> vacuum full occurs. >> > > CTID will also change every time a row is updated. So the bug should be > very easy to reproduce. One easiest way to compile different datanodes with > diferent block size. Another way could be to update/delete from a > replicated table and vacuum the table on one datanode but not other and > then again update several rows in the table. Most likely the new versions > on two different nodes will not have the same CTID anymore. > > >> Vacuum freeze will updates xmim to the smallest value and usual vacuum >> recycles "dead" row area. We may have a chance to assign different CTIDs >> to the same rows in replicated table if vacuum full runs locally in >> datanodes. (My assumption may be wrong. Does vacuum freeze move tuples >> and change their CTID?) >> >> > No, it does not. > > Thanks, > Pavan > > -- > Pavan Deolasee > http://www.linkedin.com/in/pavandeolasee > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- Andrei Martsinchyk StormDB - http://www.stormdb.com The Database Cloud |
|
From: Pavan D. <pav...@gm...> - 2013-10-09 06:39:33
|
On Wed, Oct 9, 2013 at 12:03 PM, 鈴木 幸市 <ko...@in...> wrote: > Thanks Pavan for the comment. > > So possible approach to resolve this problem could be separate system > column to maintain column identification, or asking primary key for > replicated tables when trigger (and other CTID-dependent feature) is used. > > Right. But I wonder why is it limited to triggers only ? How does UPDATE on a replicated table work anyways ? Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee |
|
From: 鈴木 幸市 <ko...@in...> - 2013-10-09 06:33:18
|
Thanks Pavan for the comment. So possible approach to resolve this problem could be separate system column to maintain column identification, or asking primary key for replicated tables when trigger (and other CTID-dependent feature) is used. Any other ideas/thoughts? --- Koichi Suzuki On 2013/10/09, at 15:26, Pavan Deolasee <pav...@gm...<mailto:pav...@gm...>> wrote: On Wed, Oct 9, 2013 at 11:49 AM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote: Hmm.. Yes, it could be serious. I suppose that CTID changes only when vacuum full occurs. CTID will also change every time a row is updated. So the bug should be very easy to reproduce. One easiest way to compile different datanodes with diferent block size. Another way could be to update/delete from a replicated table and vacuum the table on one datanode but not other and then again update several rows in the table. Most likely the new versions on two different nodes will not have the same CTID anymore. Vacuum freeze will updates xmim to the smallest value and usual vacuum recycles "dead" row area. We may have a chance to assign different CTIDs to the same rows in replicated table if vacuum full runs locally in datanodes. (My assumption may be wrong. Does vacuum freeze move tuples and change their CTID?) No, it does not. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk_______________________________________________ Postgres-xc-developers mailing list Pos...@li... https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: Pavan D. <pav...@gm...> - 2013-10-09 06:27:06
|
On Wed, Oct 9, 2013 at 11:49 AM, Koichi Suzuki <koi...@gm...>wrote: > Hmm.. Yes, it could be serious. I suppose that CTID changes only when > vacuum full occurs. > CTID will also change every time a row is updated. So the bug should be very easy to reproduce. One easiest way to compile different datanodes with diferent block size. Another way could be to update/delete from a replicated table and vacuum the table on one datanode but not other and then again update several rows in the table. Most likely the new versions on two different nodes will not have the same CTID anymore. > Vacuum freeze will updates xmim to the smallest value and usual vacuum > recycles "dead" row area. We may have a chance to assign different CTIDs > to the same rows in replicated table if vacuum full runs locally in > datanodes. (My assumption may be wrong. Does vacuum freeze move tuples > and change their CTID?) > > No, it does not. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee |
|
From: Koichi S. <koi...@gm...> - 2013-10-09 06:19:34
|
Hmm.. Yes, it could be serious. I suppose that CTID changes only when vacuum full occurs. Vacuum freeze will updates xmim to the smallest value and usual vacuum recycles "dead" row area. We may have a chance to assign different CTIDs to the same rows in replicated table if vacuum full runs locally in datanodes. (My assumption may be wrong. Does vacuum freeze move tuples and change their CTID?) I think we have two different fix for this. 1. Do not assume "the same CTID to the same row". In this case, to provide "row identification" to replicated tables, we may need different system column, 2. Ask primary key for replicated tables when trigger is used, or 3. Run vacuum full (and freeze, if vacuum freeze moves tuples among blocks) in more strict way, that is, lock all the replica in the datanode in advance so that CTID can be maintained the same. I understand the approach 1 and 2 is safer. Maybe we can begin with 2 and add 1. If vacuum freeze moves tuples, approach 3 may not be acceptable. Anyway, it can take a bit and need more discussion on the spec. Regards; --- Koichi Suzuki 2013/9/28 Mason Sharp <ma...@st...> > > > > On Thu, Sep 26, 2013 at 9:51 PM, Amit Khandekar < > ami...@en...> wrote: > >> http://sourceforge.net/p/postgres-xc/bugs/402/ >> >> But this issue has nothing do with triggers. I think there should be some >> way to reproduce without triggers. Although, if any user triggers or unique >> constraint triggers makes our job of reproducing the issue easy, that would >> be good. >> > > Under what other circumstances can one reproduce? When the statement > cannot be pushed down directly, like an UPDATE with volatile functions? > > It was not hard to reproduce... insert a few hundred rows in a table. > Execute an UPDATE on one that has a trigger defined. I guess it could > depend on when each node's particular auto-vacuum was run, perhaps with > concurrent activity. > > Anyway, I think this is a major issue. > > > >> >> >> On 27 September 2013 09:33, Ashutosh Bapat < >> ash...@en...> wrote: >> >>> Hi Hackers, >>> While working on triggers me and Amit had thought of this problem and >>> there should be a mail from Amit in that regards. What stopped us from >>> working on this more, was a reproduction scenario where same row landed on >>> two different nodes with different CTID. We couldn't get that scenario even >>> with complex ingredients like vacuum analyze etc. If you are hitting this >>> problem, can one provide us a reproduction. >>> >>> >>> On Thu, Sep 26, 2013 at 11:30 PM, mason_s <ma...@us...> wrote: >>> >>>> ------------------------------ >>>> >>>> * [bugs:#454] <http://sourceforge.net/p/postgres-xc/bugs/454/> Update >>>> triggers on replicated tables may corrupt data* >>>> >>>> *Status:* open >>>> *Created:* Thu Sep 26, 2013 06:00 PM UTC by mason_s >>>> *Last Updated:* Thu Sep 26, 2013 06:00 PM UTC >>>> *Owner:* nobody >>>> >>>> We noticed that when updating a single row on a replicated table that >>>> we were getting duplicate key violations, even though the primary key >>>> columns were not involved. >>>> >>>> We dug deeper and noticed that the mechanism uses ctid to identify what >>>> to update for update triggers that are non-"shippable" and executed on a >>>> coordinator. In the case of a replicated table, the ctid value may be >>>> different on different nodes for each tuple. It appears that Postgres-XC >>>> just uses one ctid value from one of the nodes and then sends down the same >>>> UPDATE statement to all of the individual nodes. >>>> >>>> It should either get each ctid for each node and update, or determine >>>> the corresponding unique key values and use that in the generated WHERE >>>> clause. >>>> ------------------------------ >>>> >>>> Sent from sourceforge.net because you indicated interest in >>>> https://sourceforge.net/p/postgres-xc/bugs/454/ >>>> >>>> To unsubscribe from further messages, please visit >>>> https://sourceforge.net/auth/subscriptions/ >>>> >>> >>> >>> >>> -- >>> Best Wishes, >>> Ashutosh Bapat >>> EnterpriseDB Corporation >>> The Postgres Database Company >>> >>> >>> ------------------------------------------------------------------------------ >>> October Webinars: Code for Performance >>> Free Intel webinars can help you accelerate application performance. >>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >>> from >>> the latest Intel processors and coprocessors. See abstracts and register >>> > >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >>> >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > > -- > Mason Sharp > > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Services > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > |
|
From: Michael P. <mic...@gm...> - 2013-10-09 00:43:19
|
On Wed, Oct 9, 2013 at 9:13 AM, Matt Warner <MW...@xi...> wrote: > So I owe you a different patch for proxy_main.c as well. Btw, I can see why it is needed for pgxcnode.h as we use FIONREAD there, but why is it necessary to have an include to filio.h in proxy_main.c? I couldn't figure out why so I simply removed that from the patch committed. -- Michael |
2 messages has been excluded from this view by a project administrator.
