postgres-xc-developers Mailing List for Postgres-XC

On Fri, Nov 29, 2013 at 6:15 PM, Pavan Deolasee
<pav...@gm...> wrote:
> I (actually my new compiler) noticed and complained about this which I
> believe is most likely a mistake. Please see attached patch. I haven't
> looked at the related code in detail, so don't know if this can cause any
> problems, known or unknown.
Indeed, I just went ahead, committed it and backpatched to 1.1 as well
where the issue was introduced.

> P.S. I can commit this, though not sure if my commit rights are still
> intact.
Not sure either...
-- 
Michael

Yeah,

Mavericks OS X gives out a ton of additional, helpful info related to
warnings (along with possible resolutions). Pretty neat.

Regards,
Nikhils


On Fri, Nov 29, 2013 at 2:45 PM, Pavan Deolasee <pav...@gm...>wrote:

> Hello,
>
> I (actually my new compiler) noticed and complained about this which I
> believe is most likely a mistake. Please see attached patch. I haven't
> looked at the related code in detail, so don't know if this can cause any
> problems, known or unknown.
>
> Thanks,
> Pavan
>
> P.S. I can commit this, though not sure if my commit rights are still
> intact.
>
> --
> Pavan Deolasee
> http://www.linkedin.com/in/pavandeolasee
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
StormDB - http://www.stormdb.com
The Database Cloud

Verry sorry that I didn't include the patch.  I'll wait for a day for
more input before commit.
---
Koichi Suzuki


2013/11/25 Koichi Suzuki <koi...@gm...>:
> The patch contains two fixes:
>
> 1. gtmPxyExtraConfig and gtmPxySpecificExtraConfig configuration
> handling:  They were not handled.
>
> 2. kill gtm slave handling:  Internal command generation error fixed.
>
> The are intended to be in release 1.1.1 and 1.2.
>
> Further input is welcome.
>
> Regards;
> ---
> Koichi Suzuki

Hi,

I looked into an error message "GTM error, could not obtain snapshot".
The root cause was very simple, but I found another 2 bugs on the way
of analysis.

I explain the summary of bugs and attached patches.

01_fix_error_target:
 When GTM proxy received an error response to a command, GTM proxy
reported error using an unrelated connection to the command. Because
proxy_main.c:ProcessResponse uses ereport function and it is
internally use GetMyThreadInfo->thr_conn, but thr_conn is not set
correctly.

02_fix_backend_proxy_id:
 When Postgres-XC disconnected connection to GTM proxy, GTM proxy
reports it to GTM by MSG_BACKEND_DISCONNECT. The message must have
 It could snuff out GTXID from the snapshot, and it is the root cause
of the subject.

03_fix_commit_rollback_status:
 TXN_COMMIT_RESULT / TXN_ROLLBACK_RESULT should have a gxid and a
status, but GTM Proxy sends it without a status.


I want to commit these patches tomorrow.
I'm sorry but please notice me quickly if you have a comment to the patches.

Regards

The patch contains two fixes:

1. gtmPxyExtraConfig and gtmPxySpecificExtraConfig configuration
handling:  They were not handled.

2. kill gtm slave handling:  Internal command generation error fixed.

The are intended to be in release 1.1.1 and 1.2.

Further input is welcome.

Regards;
---
Koichi Suzuki

On Sat, 2013-11-23 at 11:49 -0500, Tom Lane wrote:
> > My point is, they compile the *backend* as position-independent
> code.
> > The backend is not a shared library.  Maybe it is in Postgres-XC?
> But
> > at least this makes their build process significantly different, so
> it's
> > doubtful that this is a PG-proper issue.
> 
> Note that that's not an unreasonable decision in itself, if it's done
> pursuant to some distro policy that daemons should run with ASLR
> enabled.

Right.  False alarm.

Michael;

As you know, XC does not add any changes to configure, except for XC version number and related settings.

If the error is from XC-specific code, they could be improved as well.

Could you let me know how the build failed?   I don’t have S390 build environment.

Regards;
---
Koichi Suzuki

2013/11/23 2:41、Michael Meskes <me...@po...> のメール：

> On Fri, Nov 22, 2013 at 11:27:45AM -0500, Tom Lane wrote:
>> I think this is probably nonsense.  I spent ten years maintaining Postgres
>> for Red Hat, and I never saw any such failure on s390 in their packages.
>> If -fpic weren't good enough for shared libraries on s390, how'd any of
>> those builds get through their regression tests?
> 
> You've got a point here.
> 
>> It may well be that *mixing* -fpic and -fPIC is a bad idea, but I'd say
>> that points to an error in something XC is doing, because the core
>> Postgres build doesn't use -fPIC anywhere for Linux/s390, AFAICS.
> 
> I actually only compared to the Debian build which *does* have -fPIC and indeed
> it seems it adds -fPIC unconditionally. But then the PostgreSQL package works
> flawlessly which obviously points to XC for the problem. I give you that.
> 
>> Furthermore, if we change that convention now, we're going to increase
>> the risk of such mixing failures for other people.
> 
> Sure, but if this a bug we should. I'm not saying it is, I simply don't know.
> 
> The thread is starting with my email here
> http://lists.debian.org/debian-s390/2013/10/msg00008.html and the reply said:
> 
> It uses -fpic instead of -fPIC.
> 
> No, I'm not shortening that email reply here. :)
> 
> Checking the Debian logs it appears that all calls use *both* which seems to do
> the right thing. And yes, it appears there is a change in XC that makes it
> break. But still, I would think there has to be a correct set of options.
> 
> Michael
> -- 
> Michael Meskes
> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
> Jabber: michael.meskes at gmail dot com
> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
> 
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing 
> conversations that shape the rapidly evolving mobile landscape. Sign up now. 
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Peter Eisentraut <pe...@gm...> writes:
> On Fri, 2013-11-22 at 23:32 +0000, Greg Stark wrote:
>> Debian policy is to always use -fPIC

> My point is, they compile the *backend* as position-independent code.
> The backend is not a shared library.  Maybe it is in Postgres-XC?  But
> at least this makes their build process significantly different, so it's
> doubtful that this is a PG-proper issue.

Note that that's not an unreasonable decision in itself, if it's done
pursuant to some distro policy that daemons should run with ASLR enabled.
(Right before I left Red Hat, we were looking into building PG with -fPIE
for that reason.  It didn't happen yet because of a kernel bug[1], but
it will eventually.)

But there's too many moving parts here for us to know exactly what's going
wrong without more evidence.  The only thing that *is* pretty clear is
that the failure is not with the stock PG build anyway, so changing the
properties of the stock build sounds like the wrong response.  Personally
I'd think it is the job of the Debian package maintainer to determine why
this is breaking.

			regards, tom lane

[1] https://bugzilla.redhat.com/show_bug.cgi?id=952946

On Fri, 2013-11-22 at 23:32 +0000, Greg Stark wrote:
>         According to the Debian build logs, postgres-xc compiles the
>         entire
>         backend with -fPIC.  Not sure what sense that makes.
> 
> 
> Debian policy is to always use -fPIC

My point is, they compile the *backend* as position-independent code.
The backend is not a shared library.  Maybe it is in Postgres-XC?  But
at least this makes their build process significantly different, so it's
doubtful that this is a PG-proper issue.

Greg Stark <st...@mi...> writes:
> On Fri, Nov 22, 2013 at 8:51 PM, Peter Eisentraut <pe...@gm...> wrote:
>> According to the Debian build logs, postgres-xc compiles the entire
>> backend with -fPIC.  Not sure what sense that makes.

> Debian policy is to always use -fPIC

> IIRC -fpic is good enough as long as the total size of the library is below
> some limit. I'm not sure precisely what this size is that has to be below
> the limit but if I recall correctly it's something you have no way to
> determine in advance for a general purpose library. So Debian decided long
> long ago to just use -fPIC always.

Well, in that case they did a really crappy job of applying that policy to
their Postgres packages, because it sure sounds like there's still some
-fpic switches laying about in their builds.  But in any case, that
seems to me like a pretty brain-dead policy (hint: if you need -fPIC,
you'll get build failures that tell you so), so I feel no need to adopt
it for standard Postgres builds.

			regards, tom lane

On Fri, Nov 22, 2013 at 8:51 PM, Peter Eisentraut <pe...@gm...> wrote:

> On 11/22/13, 12:41 PM, Michael Meskes wrote:
> > Checking the Debian logs it appears that all calls use *both* which
> seems to do
> > the right thing. And yes, it appears there is a change in XC that makes
> it
> > break. But still, I would think there has to be a correct set of options.
>
> According to the Debian build logs, postgres-xc compiles the entire
> backend with -fPIC.  Not sure what sense that makes.


Debian policy is to always use -fPIC

IIRC -fpic is good enough as long as the total size of the library is below
some limit. I'm not sure precisely what this size is that has to be below
the limit but if I recall correctly it's something you have no way to
determine in advance for a general purpose library. So Debian decided long
long ago to just use -fPIC always.


-- 
greg

On 11/22/13, 12:41 PM, Michael Meskes wrote:
> Checking the Debian logs it appears that all calls use *both* which seems to do
> the right thing. And yes, it appears there is a change in XC that makes it
> break. But still, I would think there has to be a correct set of options.

According to the Debian build logs, postgres-xc compiles the entire
backend with -fPIC.  Not sure what sense that makes.

Michael Meskes <me...@po...> writes:
> On Fri, Nov 22, 2013 at 11:27:45AM -0500, Tom Lane wrote:
>> Furthermore, if we change that convention now, we're going to increase
>> the risk of such mixing failures for other people.

> Sure, but if this a bug we should. I'm not saying it is, I simply don't know.

Well, *if* it's a bug in core PG then we should do something about it,
but at the moment there's no evidence of that.  What seems the most
likely theory here is that the Debian maintainer has broken their package
with an ill-considered patch.  We can't take responsibility for other
people's hacks.

			regards, tom lane

On Fri, Nov 22, 2013 at 11:27:45AM -0500, Tom Lane wrote:
> I think this is probably nonsense.  I spent ten years maintaining Postgres
> for Red Hat, and I never saw any such failure on s390 in their packages.
> If -fpic weren't good enough for shared libraries on s390, how'd any of
> those builds get through their regression tests?

You've got a point here.

> It may well be that *mixing* -fpic and -fPIC is a bad idea, but I'd say
> that points to an error in something XC is doing, because the core
> Postgres build doesn't use -fPIC anywhere for Linux/s390, AFAICS.

I actually only compared to the Debian build which *does* have -fPIC and indeed
it seems it adds -fPIC unconditionally. But then the PostgreSQL package works
flawlessly which obviously points to XC for the problem. I give you that.

> Furthermore, if we change that convention now, we're going to increase
> the risk of such mixing failures for other people.

Sure, but if this a bug we should. I'm not saying it is, I simply don't know.

The thread is starting with my email here
http://lists.debian.org/debian-s390/2013/10/msg00008.html and the reply said:

It uses -fpic instead of -fPIC.

No, I'm not shortening that email reply here. :)

Checking the Debian logs it appears that all calls use *both* which seems to do
the right thing. And yes, it appears there is a change in XC that makes it
break. But still, I would think there has to be a correct set of options.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL

Michael Meskes <me...@po...> writes:
> I spend some time trying to figure out why PostgreSQL builds on
> S390-Linux, but Postgres-XC doesn't. Well at least this holds for the Debian
> packages. So far I haven't figured it out.  However, it appears to me that the
> build should fail for both. I'm not an S390 expert by any means, but I was told
> that S390 requires -fPIC and the build failure in the Debian package of XC came
> from a stray -fpic that was used together with -fPIC. But alas the PostgreSQL
> build has both as well.

I think this is probably nonsense.  I spent ten years maintaining Postgres
for Red Hat, and I never saw any such failure on s390 in their packages.
If -fpic weren't good enough for shared libraries on s390, how'd any of
those builds get through their regression tests?

It may well be that *mixing* -fpic and -fPIC is a bad idea, but I'd say
that points to an error in something XC is doing, because the core
Postgres build doesn't use -fPIC anywhere for Linux/s390, AFAICS.
Furthermore, if we change that convention now, we're going to increase
the risk of such mixing failures for other people.

			regards, tom lane

Hi,

I spend some time trying to figure out why PostgreSQL builds on
S390-Linux, but Postgres-XC doesn't. Well at least this holds for the Debian
packages. So far I haven't figured it out.  However, it appears to me that the
build should fail for both. I'm not an S390 expert by any means, but I was told
that S390 requires -fPIC and the build failure in the Debian package of XC came
from a stray -fpic that was used together with -fPIC. But alas the PostgreSQL
build has both as well.

Anyway, I changed src/makefiles/Makefile.linux to include

ifeq "$(findstring s390,$(host_cpu))" "s390"
CFLAGS_SL = -fPIC
else

before setting CFLAGS_SL = -fpic et voila XC builds just fine on S390. 

So I wonder shouldn't we use -fPIC instead of -fpic for S390 in general?

Michael

-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL

Hi Masataka,
To start debugging, you might want to fire this query with debugger
breaking on errstart() or errfinish() or some error reporting hook where
you can break. The stack stress should give you a lead as to where the
error is coming from. It's high chance that it's coming from
pgxc_locate_grouping_columns(). But anyway, start looking around the
callers of get_sortgroupref_tle() and check their arguments. Best luck.


On Thu, Nov 21, 2013 at 6:51 AM, Masataka Saito <pg...@gm...> wrote:

> Thanks for your information.
> It seems very helpful.
>
> cx=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
> t2.id GROUP BY b;
> ERROR:  XX000: ORDER/GROUP BY expression not found in targetlist
> LOCATION:  get_sortgroupref_tle, tlist.c:251
>
> On Thu, Nov 21, 2013 at 9:53 AM, 鈴木 幸市 <ko...@in...> wrote:
> > Yes, you can do it with SET command as well just for a target statement.
> >
> > Regards;
> > ---
> > Koichi Suzuki
> >
> > 2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール：
> >
> >> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...>
> wrote:
> >>> XC can't build a plan.
> >>>
> >>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
> >>> t2.id GROUP BY b;
> >>> ERROR:  ORDER/GROUP BY expression not found in targetlist
> >> This is a higher-level bug, planner bug just by looking at this code
> >> path src/backend/optimizer/util/tlist.c...
> >>
> >> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more
> >> details about the errors that occurred in server like the file name +
> >> name of this elog/ereport ERROR:
> >> \set VERBOSITY verbose
> >> For dev purposes it is a pretty useful default ;)
> >>
> >> Regards,
> >> --
> >> Michael
> >>
> >>
> ------------------------------------------------------------------------------
> >> Shape the Mobile Experience: Free Subscription
> >> Software experts and developers: Be at the forefront of tech innovation.
> >> Intel(R) Software Adrenaline delivers strategic insight and
> game-changing
> >> conversations that shape the rapidly evolving mobile landscape. Sign up
> now.
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Postgres-xc-developers mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>
> >
>
>
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing
> conversations that shape the rapidly evolving mobile landscape. Sign up
> now.
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Thanks for your information.
It seems very helpful.

cx=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
t2.id GROUP BY b;
ERROR:  XX000: ORDER/GROUP BY expression not found in targetlist
LOCATION:  get_sortgroupref_tle, tlist.c:251

On Thu, Nov 21, 2013 at 9:53 AM, 鈴木 幸市 <ko...@in...> wrote:
> Yes, you can do it with SET command as well just for a target statement.
>
> Regards;
> ---
> Koichi Suzuki
>
> 2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール：
>
>> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> wrote:
>>> XC can't build a plan.
>>>
>>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
>>> t2.id GROUP BY b;
>>> ERROR:  ORDER/GROUP BY expression not found in targetlist
>> This is a higher-level bug, planner bug just by looking at this code
>> path src/backend/optimizer/util/tlist.c...
>>
>> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more
>> details about the errors that occurred in server like the file name +
>> name of this elog/ereport ERROR:
>> \set VERBOSITY verbose
>> For dev purposes it is a pretty useful default ;)
>>
>> Regards,
>> --
>> Michael
>>
>> ------------------------------------------------------------------------------
>> Shape the Mobile Experience: Free Subscription
>> Software experts and developers: Be at the forefront of tech innovation.
>> Intel(R) Software Adrenaline delivers strategic insight and game-changing
>> conversations that shape the rapidly evolving mobile landscape. Sign up now.
>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>

Yes, you can do it with SET command as well just for a target statement.

Regards;
---
Koichi Suzuki

2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール：

> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> wrote:
>> XC can't build a plan.
>> 
>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
>> t2.id GROUP BY b;
>> ERROR:  ORDER/GROUP BY expression not found in targetlist
> This is a higher-level bug, planner bug just by looking at this code
> path src/backend/optimizer/util/tlist.c...
> 
> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more
> details about the errors that occurred in server like the file name +
> name of this elog/ereport ERROR:
> \set VERBOSITY verbose
> For dev purposes it is a pretty useful default ;)
> 
> Regards,
> -- 
> Michael
> 
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing 
> conversations that shape the rapidly evolving mobile landscape. Sign up now. 
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> 

On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> wrote:
> XC can't build a plan.
>
> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
> t2.id GROUP BY b;
> ERROR:  ORDER/GROUP BY expression not found in targetlist
This is a higher-level bug, planner bug just by looking at this code
path src/backend/optimizer/util/tlist.c...

Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more
details about the errors that occurred in server like the file name +
name of this elog/ereport ERROR:
\set VERBOSITY verbose
For dev purposes it is a pretty useful default ;)

Regards,
-- 
Michael

XC can't build a plan.

db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id =
t2.id GROUP BY b;
ERROR:  ORDER/GROUP BY expression not found in targetlist

Regards.

On 11/20/13, Ashutosh Bapat <ash...@en...> wrote:
> My guess is that it's an error coming from the datanode and thus we need
> EXPLAIN output.
>
>
> On Wed, Nov 20, 2013 at 6:53 AM, Koichi Suzuki
> <koi...@gm...>wrote:
>
>> I'm afraid the error occurs before the plan is build.
>> ---
>> Koichi Suzuki
>>
>>
>> 2013/11/19 Ashutosh Bapat <ash...@en...>:
>> > Can you please provide output of EXPLAIN verbose?
>> >
>> >
>> > On Thu, Nov 14, 2013 at 1:44 PM, Masataka Saito <pg...@gm...>
>> wrote:
>> >>
>> >> Hello, all.
>> >>
>> >> I'm testing complex SQLs on Postgres-XC 1.1.
>> >> And I found bad SQL that XC doesn't accept.
>> >>
>> >> The SQL is  from answer 2 of chapter 17 in SQL puzzle book by Joe
>> >> Celko.
>> >> I simplified it for everyone's reproduction.
>> >>
>> >> db=# CREATE TABLE t1(id INT, a INT);
>> >> CREATE TABLE
>> >> db=# CREATE TABLE t2(id INT, b INT);
>> >> CREATE TABLE
>> >> db=# SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = t2.id GROUP BY b;
>> >> ERROR:  ORDER/GROUP BY expression not found in targetlist
>> >>
>> >> This SQL is accepted by PostgreSQL 9.2.4.
>> >>
>> >> Can anyone solve or explain this issue?
>> >>
>> >> Regards.
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> DreamFactory - Open Source REST & JSON Services for HTML5 & Native
>> >> Apps
>> >> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
>> >> Free app hosting. Or install the open source package on any LAMP
>> >> server.
>> >> Sign up and see examples for AngularJS, jQuery, Sencha Touch and
>> >> Native!
>> >>
>> >>
>> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
>> >> _______________________________________________
>> >> Postgres-xc-developers mailing list
>> >> Pos...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>> >
>> >
>> >
>> > --
>> > Best Wishes,
>> > Ashutosh Bapat
>> > EnterpriseDB Corporation
>> > The Postgres Database Company
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Shape the Mobile Experience: Free Subscription
>> > Software experts and developers: Be at the forefront of tech
>> > innovation.
>> > Intel(R) Software Adrenaline delivers strategic insight and
>> > game-changing
>> > conversations that shape the rapidly evolving mobile landscape. Sign up
>> now.
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>

My guess is that it's an error coming from the datanode and thus we need
EXPLAIN output.


On Wed, Nov 20, 2013 at 6:53 AM, Koichi Suzuki <koi...@gm...>wrote:

> I'm afraid the error occurs before the plan is build.
> ---
> Koichi Suzuki
>
>
> 2013/11/19 Ashutosh Bapat <ash...@en...>:
> > Can you please provide output of EXPLAIN verbose?
> >
> >
> > On Thu, Nov 14, 2013 at 1:44 PM, Masataka Saito <pg...@gm...>
> wrote:
> >>
> >> Hello, all.
> >>
> >> I'm testing complex SQLs on Postgres-XC 1.1.
> >> And I found bad SQL that XC doesn't accept.
> >>
> >> The SQL is  from answer 2 of chapter 17 in SQL puzzle book by Joe Celko.
> >> I simplified it for everyone's reproduction.
> >>
> >> db=# CREATE TABLE t1(id INT, a INT);
> >> CREATE TABLE
> >> db=# CREATE TABLE t2(id INT, b INT);
> >> CREATE TABLE
> >> db=# SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = t2.id GROUP BY b;
> >> ERROR:  ORDER/GROUP BY expression not found in targetlist
> >>
> >> This SQL is accepted by PostgreSQL 9.2.4.
> >>
> >> Can anyone solve or explain this issue?
> >>
> >> Regards.
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
> >> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
> >> Free app hosting. Or install the open source package on any LAMP server.
> >> Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Postgres-xc-developers mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
> >
> >
> >
> > --
> > Best Wishes,
> > Ashutosh Bapat
> > EnterpriseDB Corporation
> > The Postgres Database Company
> >
> >
> ------------------------------------------------------------------------------
> > Shape the Mobile Experience: Free Subscription
> > Software experts and developers: Be at the forefront of tech innovation.
> > Intel(R) Software Adrenaline delivers strategic insight and game-changing
> > conversations that shape the rapidly evolving mobile landscape. Sign up
> now.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

I'm afraid the error occurs before the plan is build.
---
Koichi Suzuki


2013/11/19 Ashutosh Bapat <ash...@en...>:
> Can you please provide output of EXPLAIN verbose?
>
>
> On Thu, Nov 14, 2013 at 1:44 PM, Masataka Saito <pg...@gm...> wrote:
>>
>> Hello, all.
>>
>> I'm testing complex SQLs on Postgres-XC 1.1.
>> And I found bad SQL that XC doesn't accept.
>>
>> The SQL is  from answer 2 of chapter 17 in SQL puzzle book by Joe Celko.
>> I simplified it for everyone's reproduction.
>>
>> db=# CREATE TABLE t1(id INT, a INT);
>> CREATE TABLE
>> db=# CREATE TABLE t2(id INT, b INT);
>> CREATE TABLE
>> db=# SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = t2.id GROUP BY b;
>> ERROR:  ORDER/GROUP BY expression not found in targetlist
>>
>> This SQL is accepted by PostgreSQL 9.2.4.
>>
>> Can anyone solve or explain this issue?
>>
>> Regards.
>>
>>
>> ------------------------------------------------------------------------------
>> DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
>> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
>> Free app hosting. Or install the open source package on any LAMP server.
>> Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing
> conversations that shape the rapidly evolving mobile landscape. Sign up now.
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Can you please provide output of EXPLAIN verbose?


On Thu, Nov 14, 2013 at 1:44 PM, Masataka Saito <pg...@gm...> wrote:

> Hello, all.
>
> I'm testing complex SQLs on Postgres-XC 1.1.
> And I found bad SQL that XC doesn't accept.
>
> The SQL is  from answer 2 of chapter 17 in SQL puzzle book by Joe Celko.
> I simplified it for everyone's reproduction.
>
> db=# CREATE TABLE t1(id INT, a INT);
> CREATE TABLE
> db=# CREATE TABLE t2(id INT, b INT);
> CREATE TABLE
> db=# SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = t2.id GROUP BY b;
> ERROR:  ORDER/GROUP BY expression not found in targetlist
>
> This SQL is accepted by PostgreSQL 9.2.4.
>
> Can anyone solve or explain this issue?
>
> Regards.
>
>
> ------------------------------------------------------------------------------
> DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
> Free app hosting. Or install the open source package on any LAMP server.
> Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

I will try to reply to this as soon as I get a chance, but meanwhile,
including this in dev list.


On 16 November 2013 01:50, Mason Sharp <mas...@gm...> wrote:

> Hi Amit,
>
> Can I ask you about this commit?
>
>
>> 0. In initdb.c vacuum-freeze all the databases, so that there will not be
>> any data loss even if it's local xid happens to be in the future w.r.t.
>> GXID.
>>
>
> What case does this handle? A GTM that has a high XID and one decides to
> re-initdb?
>
>
>
>> After the server is brought up, the pg_class.relfrozenxid,
>> pg_database.relfrozenxid, and the xid-wraparound limits should now be
>> adjusted according to the new GXID.
>>
>
> Here, you are just talking about a brand new node/database?
>
>
>> 1. If gxid exceeds "Stop" limit (ShmemVariableCache->xidStopLimit), allow
>> auto-vacuum to adjust the oldest frozen xids, and the various xid wrap
>> limits w.r.t. the new gxid.
>>
>
>
>
>> 2. If gxid is greater than the current xid-wrap limit (i.e. oldest xid +
>> 2^31), don't even allow auto-vacuum to run. DBA would run vacuum-freeze on
>> all the databases using standalone backend.
>>
>
> You suppress autovacuum? Are operations on the database blocked until a
> manual vacuum freeze?
>
>
>
>>  3. Allow standalone backend to run using GXIDs, rather than using local
>> xids. This way, if gxid exceeds wrap limit, the user would use standalone
>> backend to run vacuum-freeze on all databases. Currently, because
>> standalone backend uses local xids, there is no way vacuum-freeze can
>> update the oldest frozen xids. Hence, the need for using global xids.
>> Anyway, it makes sense to start using global xids.
>>
>
> Kind of confused what you mean by standalone backends.
>
>
>> 5. In GetNewTransactionID(), while printing error message about "commands
>> can't run due to xid-wraparound", get_database_name() is called to print
>> the database name. But if the context is auto-vacuum launcher,
>> get_database_name() fails because MyDatabaseId is not set in auto-vacuuum
>> launcher. For this, call get_database_name() only if MyDatabaseId is set.
>>
>> 6. Try to keep ShmemVariableCache->nextXid in sync with gxid. nextXid is
>> used at a lot of places, so if it remains out-of-sync, we get unexpected
>> error/warning messages when running vacuum-freeze on a standalone backend.
>> I have not changed the current behaviour by keeping this sync up only in
>> standalone backend. There must be some reason why we increment nextXid only
>> in certain scenarios (see increment_xid), so did not totally remove that
>> logic.
>>
>
> I am not sure. Seems like we should always use global XIDs and global
> snapshots.
>
>
>>  7. If the server is brought up with xids far in the future w.r.t. GXID,
>> even the relfrozenxids can be in the future w.r.t. to the new GXID. In PG,
>> while updating pg_class.relfrozenxid, it is never set backwards because
>> that is never possible.
>>
>> In XC, that's possible now, so allow it to go backwards. But this will
>> happen only in standalone backend because we disallow commands if gxid is
>> already wrapped around. Similarly, allow pg_database.datfrozenxid to be set
>> in the "past". (vac_update_relstats() and vac_update_datfrozenxid()).
>>
>
> What do you mean by standalone backends here?
>
>
>
>> 8. Keep the current behaviour of *not* truncating the clog if wrap limits
>> are to be set backwards (See vac_truncate_clog()). But once they are set,
>> and all xids are in sync, it will continue normal pg_clog truncation in the
>> next vacuum-freeze cycle.
>>
>> 9. initdb will continue to use local xids. Since a standlalone postgres
>> backend now would use gxid by default, we need a way for initdb to spawn
>> postgres that would use local xids. For this, we have added a new postgres
>> cmd-line argument --localxid. This is to be used only by initdb.
>>
>
> So.... when would --localxid be used? Only by pgxc_ctl when adding new
> nodes? When would a DBA manually use it?
>
> BTW, we have an XC support customer who appeared to run into some data
> errors after running a vacuum freeze, hence I was looking into these
> commits.
>
> Thanks,
>
> Mason
>
>
>
>