postgres-xc-developers Mailing List for Postgres-XC (Page 2)

I've taken configure.in and the M4 files from Postgres 9.3.0 and added in the XC-specific pieces. I hope to have not missed anything. I am testing to ensure I can perform a complete build with no issues before I send the updated files.

With respect to how to handle the include of filio.h and defining PGXC, I added in a pgxc_config.h.in file to handle both. This seems to be in line with how Postgres is handling these types of configs and only requires a simple ifdef for including filio.h.

So I owe you a different patch for proxy_main.c as well.

FYI,

Matt

-----Original Message-----
From: Matt Warner 
Sent: Tuesday, October 08, 2013 1:45 PM
To: 'Michael Paquier'
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: RE: [Postgres-xc-developers] Minor Fixes

Yes, I have autoconf in place and have been putting in time here and there to update the configure.in.

I've also been looking at using autoconf to check for the presence of filio.h and go that route instead of specifying "__SOLARIS__". That would seem to be more portable.

Matt

-----Original Message-----
From: Michael Paquier [mailto:mic...@gm...] 
Sent: Tuesday, October 08, 2013 5:33 AM
To: Matt Warner
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: Re: [Postgres-xc-developers] Minor Fixes

On Tue, Oct 8, 2013 at 9:03 PM, Michael Paquier
<mic...@gm...> wrote:
> 3 - Patch for makesgml
I pushed 3 as it was a trivial change as well, remains 1 and 2. Matt,
are you able to get an environment with autoconf 2.63? I cannot recall
on the top of my head an environment that has it... Ubuntu or CentOS
maintain autoconf archives, no?
-- 
Michael

Yes, I have autoconf in place and have been putting in time here and there to update the configure.in.

I've also been looking at using autoconf to check for the presence of filio.h and go that route instead of specifying "__SOLARIS__". That would seem to be more portable.

Matt

-----Original Message-----
From: Michael Paquier [mailto:mic...@gm...] 
Sent: Tuesday, October 08, 2013 5:33 AM
To: Matt Warner
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: Re: [Postgres-xc-developers] Minor Fixes

On Tue, Oct 8, 2013 at 9:03 PM, Michael Paquier
<mic...@gm...> wrote:
> 3 - Patch for makesgml
I pushed 3 as it was a trivial change as well, remains 1 and 2. Matt,
are you able to get an environment with autoconf 2.63? I cannot recall
on the top of my head an environment that has it... Ubuntu or CentOS
maintain autoconf archives, no?
-- 
Michael

On Tue, Oct 8, 2013 at 9:03 PM, Michael Paquier
<mic...@gm...> wrote:
> 3 - Patch for makesgml
I pushed 3 as it was a trivial change as well, remains 1 and 2. Matt,
are you able to get an environment with autoconf 2.63? I cannot recall
on the top of my head an environment that has it... Ubuntu or CentOS
maintain autoconf archives, no?
-- 
Michael

I just finished working on the patch for GTM-Proxy and directly pushed
it as it was trivial. I also decided to backpatch it as there were
some incorrectness in the code.

The following things are still missing:
1 - Change configure.in and regenerate configure with autoconf
(addition of a flag for Solaris, don't like much this method but I
couldn't come up with smth better)
2 - Patch for pgxcnode.c
3 - Patch for makesgml

I'll work on the patch for makesgml next and post it here for review.
Regards,
-- 
Michael

It looks like having multiple connections to the datanodes from different
coordinators could be a problem if there are writes involved. I am checking
if 1.B.ii below is feasible.

I remember, Michael had implementing some way by which rows can be sent to
and fro between coordinator and the datanode for implementing alter table
distribute by. It might be possible to use the same protocol. Michael, can
you please point out relevant code in as much details as possible?


   B. Population of materialized view (REFRESH MV command)
>       There are two approaches here.
>       i. REFRESH MV command would be sent to each of the coordinators, and
> each of
>       the coordinator would populate its MV table separately. This means
> that each
>       of the coordinators would fire the same query and would get the same
> result,
>       which is waste of resources. In a sane MV implementation we won't
> expect MVs
>       to be refreshed frequently, and thus having this wastage once in a
> while
>       would not be an overhead. Given that this approach needs very small
> efforts,
>       it might be acceptable in version 1. The planner code is sprinkled
> with
>       !IsConnFromCoord()? where we do not create RemoteQueryPaths for
> remote
>       relations. For population of MV, we need to lift this restriction. I
> am not sure how
>       to distinguish normal scans from scans for refreshing MV.
>
>       ii. REFRESH MV on the coordinator where the REFRESH command is
> issued and
>       send the data using COPY or some bulk protocols to the other
> coordinators.
>       This approach needs some extra efforts for constructing the COPY
> command
>       and the corresponding data file to be sent over to the other
> coordinators.
>       Also, it needs some changes to REFRESH MATERIALIZED handling so that
> such
>       data can be sent along with it. We may try this approach in version
> 2.
>
>
-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Thanks Masataka.

Can you create a source-forge ticket for this please? Attach a reproducible
test case with the ticket.



On Thu, Oct 3, 2013 at 2:45 PM, Masataka Saito <pg...@gm...> wrote:

> Hello, all.
>
> I found Postgres-XC returns different error code from PostgreSQL's one
> for repeated rollback, and it makes a pgjdbc's regression test case
> fail.
>
> PostgreSQL returns error code 42704 which is defined as
> ERRCODE_UNDEFINED_OBJECT. It is maybe returned in
> src/backend/access/transam/twophase.c::LockGXact.
> Postgres-XC returns error code XX000 which is defined as
> ERRCODE_INTERNAL_ERROR It is maybe returned in
> src/backend/pgxc/pool/execRemote.c::FinishRemotePreparedTransaction.
>
> Error messages are quite similar. I will show it at the next line.
> > prepared transaction with identifier
> "0_8fHx8fF5/OCyZd9xZmJ2MSPCE1olci037CNQYoXoVCKbi63JcZIoUeJxU7PKl/t4qI4xV1agFPE+9gRrzheGHA==_BAUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="
> does not exist.
>
> Does anyone know ...
>  * Why they have different error codes, and is it reasonable?
>  * To fix the code affects other components?
>
> You can reproduce using pgjdbc's regression test. The source code of
> the test is org.postgresql.test.xa.XADataSourceTest.testRepeatedRolledBack.
> >
> http://sourceforge.net/p/postgres-xc/pgjdbc-xc/ci/master/tree/org/postgresql/test/xa/XADataSourceTest.java#l331
>
> Regards.
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.

Now PostgreSQL comes with event triggers, I think we should consider to
support it at our next major release.   Here's my idea for the first
version implementation.   Any suggestions/ideas are welcome.

1. Event trigger definitions in pg_event_trigger catalog itself will be
propagated to all the nodes.   We can use existing infrastructure for this
(maybe we can exclude datanodes from such propagation though...)

2. Event trigger will be fired only at the originating coordinator.
Trigger function can get XC cluster configuration and determine what they
should do with other nodes if necessary.   It makes event trigger function
design simpler.

Regards;
---
Koichi Suzuki

Thanks Ashutosh for the quick and nice work.   Please find my comment
inline.

Regards;
---
Koichi Suzuki


2013/10/3 Ashutosh Bapat <ash...@en...>

> Hi All,
> I am working on supporting MVs in XC. Here are some notes I have written.
> Please provide your comments. I am leaning towards creating MVs at the
> coordinators.
>
> For any materialized views (called MV henceforth) implementation, we need
> to
> implement following functionality
> 1. Creation of MV (bare creation without population)
> 2. Population of MV
> 3. Querying MV
> 4. Dropping MV
> 5. Alter MV
> 6. Creating Indexes on MV
>
> Following write-up discussed various approaches for implementing MVs in XC
> 1. Create materialised view only on the coordinators
>    While creating the materialised view, the CREATE MATERIALIZED VIEW
> statement is
>    turned into a CTAS query. The new table is created as a materialised
> view table.
>    In XC, CTAS is implemented by rewriting the statement as CREATE TABLE
> ...; INSERT INTO
>    .... Hence we will need to stop the MV's CTAS being rewritten.
>
>    A. Creation of MV
>       At the time of creation of the MV table, we will need to create the
> table
>       locally at the coordinator. The subsequent heap_insert is expected to
>       automatically populate this local table. By propagating Create MV
> statement
>       to each of the coordinator (and not the datanode/s) we will get the
> same
>       effect. More details below.
>       We will need to set the distribution info of MV to NULL to indicate
> that
>       it's a local table.
>       Given that the catalog tables are created locally on each of the
>       coordinators, we should be able to CREATE MV locally on the
> coordinators.
>
>    B. Population of materialized view (REFRESH MV command)
>       There are two approaches here.
>       i. REFRESH MV command would be sent to each of the coordinators, and
> each of
>       the coordinator would populate its MV table separately. This means
> that each
>       of the coordinators would fire the same query and would get the same
> result,
>       which is waste of resources. In a sane MV implementation we won't
> expect MVs
>       to be refreshed frequently, and thus having this wastage once in a
> while
>       would not be an overhead. Given that this approach needs very small
> efforts,
>       it might be acceptable in version 1. The planner code is sprinkled
> with
>       !IsConnFromCoord()? where we do not create RemoteQueryPaths for
> remote
>       relations. For population of MV, we need to lift this restriction. I
> am not sure how
>       to distinguish normal scans from scans for refreshing MV.
>

Especially in DDL handling, this IsConnFromCoord() is widely used from
place to place for specific needs.  So in REFRESH MV command handling, you
can control what to do in such a manner.   utility.c contains such things.
  Because master_pg93_merge is under the work and utility.c may need some
more correction, it may be better to look at REL1_1_STABLE or master.


>
>       ii. REFRESH MV on the coordinator where the REFRESH command is
> issued and
>       send the data using COPY or some bulk protocols to the other
> coordinators.
>       This approach needs some extra efforts for constructing the COPY
> command
>       and the corresponding data file to be sent over to the other
> coordinators.
>       Also, it needs some changes to REFRESH MATERIALIZED handling so that
> such
>       data can be sent along with it. We may try this approach in version
> 2.
>

I agree on this.


>
>    C. Querying MV
>       Nothing more needs to be done here. In most of the places, we have
> treated
>       tables with no distribution info as local tables. Hence this won't be
>       difficult. There might be corner cases, testing will be required for
> such
>       corner cases.
>

OK.  I think this is very important step toward coordinator/datanode
integration.


>
>    D. ALTER MV
>       I don't think there is any change required here. We will need to
> test it
>       to make sure that the things are working as expected. We should
> propagate
>       the ALTER MV command to all the coordinators and not to the
> datanodes.
>

I'm afraid native code for ALTER MV does not allow to run inside 2PC or
transaction block, as ALTER TYPE ... ADD.   If so, we should check this
only at initiated node as done in utility.c and typecmds.c, such as

#ifdef PGXC
        /*

         * We disallow this in transaction blocks, because we can't cope

         * with enum OID values getting into indexes and then having their

         * defining pg_enum entries go away.

         */
        /* Allow this to be run inside transaction block on remote nodes */
        if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
#endif
            PreventTransactionChain(isTopLevel, "ALTER TYPE ... ADD");

>
>    E. Creating indexes on MV
>       Given that we can create indexes on the catalog tables, it should be
>       possible to create indexes on the MV tables as well. Testing will
> need to
>       be done to make sure that this works. CREATE INDEX command will need
> to be
>       propagated to all the coordinators and not to the datanodes.
>

Yes, we need this change.


>
>    Materialized views are a query caching technique. Usually a query would
> be
>    served by a single materialized view or multiple of them. So, having
> them
>    created on the coordinator/s reduces one trip to the datanode. This
> will also
>    allow us to take a step towards coordinator - datanode merge.
>

At present, this means we cannot ship joins with MVIEW.  I hope it is
acceptable as the first implementation.



>
> 2. Create MV definition on both the coordinator and datanode but keep the
> data on the datanode/s
>    While creating the materialised view, the CREATE MATERIALIZED VIEW
> statement is
>    turned into a CTAS query. The new table is created as a materialised
> view table.
>    In XC, CTAS is implemented by rewriting the statement as CREATE TABLE
> ...; INSERT INTO
>    .... We need to filter MVs out of this rewriting.
>
>    A. Creating MV
>       Pass MV definition to the datanode and all the coordinators. This
> would be
>       done in two phases. First we create just the definition on all the
> nodes.
>       In the second phase, the MV table created on the datanode will be
>       populated using method below.
>
>    B. Populating an MV
>       We need to device a special INSERT INTO protocol which would run the
>       associated query on the initiating coordinator and insert the
> results into
>       the MV table at the datanodes. REFRESH MV would be turned into
> INSERT INTO
>       at the initiating coordinator and each other coordinator would get
> REFRESH
>       MV command so that the MV would be set as scanable without causing
> the
>       query to be fired.
>
>    C. Querying MV
>       Like normal tables, we will have to create corresponding
>       REMOTE_QUERY_PATH for MV relations.
>
>    D. ALTER MV
>       Passing down ALTER MV to the datanodes should work. But it needs some
>       testing to make sure that ALTER MV is working fine.
>

Please see the comment above.


>
>    E. Creating indexes on MV
>       This looks straight forward, since we can create indexes on the
> datanodes.
>       Needs some testing.
>
OK.


>
> If we create MV on the datanodes, and do not allow distribution to be
> specified,
> those MVs need to be replicated (since they are stable by definition) by
> default
> as against the default distribution by hash. Given that materialized views
> are
> query caching technique, we expect that a query would be entirely served
> by a
> materialized view or multiple of those. In such case, having materialized
> views
> on datanode/s would create unnecessary hops from coordinator to the
> datanode.
>
> Distributing MV
> ---------------
> If we need to create materialized view only on selected nodes
> (coordinators/datanodes) we will need to
>    a. extend the CREATE/ALTER MV syntax to specify nodes where to create
> the MV.
>    b. disallow any REFRESH on the nodes where the MV does not exist.
>    We may take up this extension in version 2.
>

I think it's okay as the first implementation.   There's two options for
internal.
i) Maintain MVIEW in pgxc_class or other catalog, with full distribution
information,
ii) Not having any pgxc-specific catalogs and hard code version-1 specific
assumption.

This could be determined by required resource.  Anyway, we need option i)
to control the distribution at the next improvement.


>
> What happens to an MV when a node is added or dropped?
>

Ouch! Yes, it is very important point to consider.

Assuming the option ii) above, because all the catalogues are copied when a
node is added, we can just REFRESH it.   When nodes are dropped, maybe we
don't have to do anything.

If we take the option i) above, we need to provide ALTER MVIEW ... ADD NODE
and ALTER MVIEW DELETE NODE to maintain the distribution and we should
redistribute MVIEW to the new set of nodes.   This is similar to ALTER
TABLE approach.

------------------------------------------------------
> We need to test. Having MV at the coordinator helps here, since addition or
> removal of datanode doesn't affect the MV. While adding a coordinator, the
> catalog replication itself would create the MV. Dropping a node wouldn't
> be a
> problem. We need to test the feature.
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>

Hi All,
I am working on supporting MVs in XC. Here are some notes I have written.
Please provide your comments. I am leaning towards creating MVs at the
coordinators.

For any materialized views (called MV henceforth) implementation, we need to
implement following functionality
1. Creation of MV (bare creation without population)
2. Population of MV
3. Querying MV
4. Dropping MV
5. Alter MV
6. Creating Indexes on MV

Following write-up discussed various approaches for implementing MVs in XC
1. Create materialised view only on the coordinators
   While creating the materialised view, the CREATE MATERIALIZED VIEW
statement is
   turned into a CTAS query. The new table is created as a materialised
view table.
   In XC, CTAS is implemented by rewriting the statement as CREATE TABLE
...; INSERT INTO
   .... Hence we will need to stop the MV's CTAS being rewritten.

   A. Creation of MV
      At the time of creation of the MV table, we will need to create the
table
      locally at the coordinator. The subsequent heap_insert is expected to
      automatically populate this local table. By propagating Create MV
statement
      to each of the coordinator (and not the datanode/s) we will get the
same
      effect. More details below.
      We will need to set the distribution info of MV to NULL to indicate
that
      it's a local table.
      Given that the catalog tables are created locally on each of the
      coordinators, we should be able to CREATE MV locally on the
coordinators.

   B. Population of materialized view (REFRESH MV command)
      There are two approaches here.
      i. REFRESH MV command would be sent to each of the coordinators, and
each of
      the coordinator would populate its MV table separately. This means
that each
      of the coordinators would fire the same query and would get the same
result,
      which is waste of resources. In a sane MV implementation we won't
expect MVs
      to be refreshed frequently, and thus having this wastage once in a
while
      would not be an overhead. Given that this approach needs very small
efforts,
      it might be acceptable in version 1. The planner code is sprinkled
with
      !IsConnFromCoord()? where we do not create RemoteQueryPaths for remote
      relations. For population of MV, we need to lift this restriction. I
am not sure how
      to distinguish normal scans from scans for refreshing MV.

      ii. REFRESH MV on the coordinator where the REFRESH command is issued
and
      send the data using COPY or some bulk protocols to the other
coordinators.
      This approach needs some extra efforts for constructing the COPY
command
      and the corresponding data file to be sent over to the other
coordinators.
      Also, it needs some changes to REFRESH MATERIALIZED handling so that
such
      data can be sent along with it. We may try this approach in version 2.

   C. Querying MV
      Nothing more needs to be done here. In most of the places, we have
treated
      tables with no distribution info as local tables. Hence this won't be
      difficult. There might be corner cases, testing will be required for
such
      corner cases.

   D. ALTER MV
      I don't think there is any change required here. We will need to test
it
      to make sure that the things are working as expected. We should
propagate
      the ALTER MV command to all the coordinators and not to the datanodes.

   E. Creating indexes on MV
      Given that we can create indexes on the catalog tables, it should be
      possible to create indexes on the MV tables as well. Testing will
need to
      be done to make sure that this works. CREATE INDEX command will need
to be
      propagated to all the coordinators and not to the datanodes.

   Materialized views are a query caching technique. Usually a query would
be
   served by a single materialized view or multiple of them. So, having them
   created on the coordinator/s reduces one trip to the datanode. This will
also
   allow us to take a step towards coordinator - datanode merge.

2. Create MV definition on both the coordinator and datanode but keep the
data on the datanode/s
   While creating the materialised view, the CREATE MATERIALIZED VIEW
statement is
   turned into a CTAS query. The new table is created as a materialised
view table.
   In XC, CTAS is implemented by rewriting the statement as CREATE TABLE
...; INSERT INTO
   .... We need to filter MVs out of this rewriting.

   A. Creating MV
      Pass MV definition to the datanode and all the coordinators. This
would be
      done in two phases. First we create just the definition on all the
nodes.
      In the second phase, the MV table created on the datanode will be
      populated using method below.

   B. Populating an MV
      We need to device a special INSERT INTO protocol which would run the
      associated query on the initiating coordinator and insert the results
into
      the MV table at the datanodes. REFRESH MV would be turned into INSERT
INTO
      at the initiating coordinator and each other coordinator would get
REFRESH
      MV command so that the MV would be set as scanable without causing the
      query to be fired.

   C. Querying MV
      Like normal tables, we will have to create corresponding
      REMOTE_QUERY_PATH for MV relations.

   D. ALTER MV
      Passing down ALTER MV to the datanodes should work. But it needs some
      testing to make sure that ALTER MV is working fine.

   E. Creating indexes on MV
      This looks straight forward, since we can create indexes on the
datanodes.
      Needs some testing.

If we create MV on the datanodes, and do not allow distribution to be
specified,
those MVs need to be replicated (since they are stable by definition) by
default
as against the default distribution by hash. Given that materialized views
are
query caching technique, we expect that a query would be entirely served by
a
materialized view or multiple of those. In such case, having materialized
views
on datanode/s would create unnecessary hops from coordinator to the
datanode.

Distributing MV
---------------
If we need to create materialized view only on selected nodes
(coordinators/datanodes) we will need to
   a. extend the CREATE/ALTER MV syntax to specify nodes where to create
the MV.
   b. disallow any REFRESH on the nodes where the MV does not exist.
   We may take up this extension in version 2.

What happens to an MV when a node is added or dropped?
------------------------------------------------------
We need to test. Having MV at the coordinator helps here, since addition or
removal of datanode doesn't affect the MV. While adding a coordinator, the
catalog replication itself would create the MV. Dropping a node wouldn't be
a
problem. We need to test the feature.


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Hello, all.

I found Postgres-XC returns different error code from PostgreSQL's one
for repeated rollback, and it makes a pgjdbc's regression test case
fail.

PostgreSQL returns error code 42704 which is defined as
ERRCODE_UNDEFINED_OBJECT. It is maybe returned in
src/backend/access/transam/twophase.c::LockGXact.
Postgres-XC returns error code XX000 which is defined as
ERRCODE_INTERNAL_ERROR It is maybe returned in
src/backend/pgxc/pool/execRemote.c::FinishRemotePreparedTransaction.

Error messages are quite similar. I will show it at the next line.
> prepared transaction with identifier "0_8fHx8fF5/OCyZd9xZmJ2MSPCE1olci037CNQYoXoVCKbi63JcZIoUeJxU7PKl/t4qI4xV1agFPE+9gRrzheGHA==_BAUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==" does not exist.

Does anyone know ...
 * Why they have different error codes, and is it reasonable?
 * To fix the code affects other components?

You can reproduce using pgjdbc's regression test. The source code of
the test is org.postgresql.test.xa.XADataSourceTest.testRepeatedRolledBack.
> http://sourceforge.net/p/postgres-xc/pgjdbc-xc/ci/master/tree/org/postgresql/test/xa/XADataSourceTest.java#l331

Regards.

Looks ok to me.


On Wed, Oct 2, 2013 at 1:25 PM, Koichi Suzuki <koi...@gm...> wrote:

> Here's revised patch.  I think we can apply this both to our master and
> REL1_1_STABLE.  (We didn't apply this change to REL1_0_STABLE).
>
> Regards;
>
> ---
> Koichi Suzuki
>
>
> 2013/10/1 鈴木 幸市 <ko...@in...>
>
>>  Agree to remove #else .. .   Assert is defined under #ifdef
>> ASSERT_ENABLED internally and we don't have to worry about it.  Without
>> this symbol defined, Assert will be expanded to null and there's nothing
>> bothering.
>>
>>  Regards;
>> ---
>> Koichi Suzuki
>>
>>  On 2013/10/01, at 16:43, Ashutosh Bapat <ash...@en...<ashutosh..ba...@en...>>
>> wrote:
>>
>>  We should move the Assert under #ifdef ASSERT_ENABLED as well the
>> definition of old_outEnd. We should remove #else defining old_outEnd to
>> handle->outEnd which is utterly wrong.
>>
>>
>> On Mon, Sep 30, 2013 at 12:15 PM, Koichi Suzuki <koi...@gm...>wrote:
>>
>>> This change was made to eliminate a warning that old_outEnd is not used.
>>>   To improve both, the patch would be as attached.
>>>
>>>  Regards;
>>>
>>>  ---
>>> Koichi Suzuki
>>>
>>>
>>> 2013/9/30 Abbas Butt <abb...@en...>
>>>
>>>>  Hi,
>>>>
>>>> The commit ae29a966d3286d4e85b3adf162d00f9b1e8b8eae made this change
>>>> @@ -1132,7 +1132,11 @@ pgxc_node_send_parse(PGXCNodeHandle * handle,
>>>> const
>>>> char* statement,
>>>>         /* message length */
>>>>         int                     msgLen;
>>>>         int                     cnt_params;
>>>> +#if 0
>>>>         size_t          old_outEnd = handle->outEnd;
>>>> +#else
>>>> +#define old_outEnd handle->outEnd
>>>> +#endif
>>>>
>>>> which is incorrect. The purpose of the declaration and assignment is to
>>>> save the current value of handle->outEnd which is later changed in the
>>>> function.
>>>> This change makes the following assert in function pgxc_node_send_parse
>>>>     Assert(old_outEnd + ntohl(msgLen) + 1 == handle->outEnd);
>>>> fail when the query
>>>> CREATE TABLE junk AS VALUES (1), (2);
>>>> is run.
>>>>
>>>> This patch is required on master as well.
>>>>
>>>> Regards
>>>>
>>>> --
>>>> *Abbas*
>>>>  Architect
>>>>
>>>> Ph: 92.334.5100153
>>>>  Skype ID: gabbasb
>>>> www.enterprisedb.co
>>>> <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>>>> *
>>>> Follow us on Twitter*
>>>> @EnterpriseDB
>>>>
>>>> Visit EnterpriseDB for tutorials, webinars,
>>>> whitepapers<http://www.enterprisedb.com/resources-community>and
>>>> more<http://www.enterprisedb.com/resources-community>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> October Webinars: Code for Performance
>>>> Free Intel webinars can help you accelerate application performance.
>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
>>>> most from
>>>> the latest Intel processors and coprocessors. See abstracts and
>>>> register >
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>>> from
>>> the latest Intel processors and coprocessors. See abstracts and register
>>> >
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EnterpriseDB Corporation
>> The Postgres Database Company
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk_______________________________________________
>>
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

I've shared table definitions, query, plan, and even scrubbed data to Abbas and Michael.

From: Koichi Suzuki [mailto:koi...@gm...]
Sent: Wednesday, October 02, 2013 6:36 PM
To: Matt Warner
Cc: Michael Paquier; Postgres-XC Developers
Subject: Re: [Postgres-xc-developers] Minor Fixes

Thanks Matt and Michael.   Please keep sharing your work on Solaris and please share optimizer bug issue.   It will be nice to share table definition, query and the plan.

Best;
---
Koichi Suzuki

2013/10/3 Matt Warner <MW...@xi...<mailto:MW...@xi...>>
I'm compiling with the Solaris Studio compiler on Solaris Intel right now. If it compiles on Intel, it will compile on SPARC, the only exceptions being assembly code or code that has hard-coded dependencies on byte ordering (big- vs little-endian systems). I don't recall seeing any code like that in XC. That said, I have access to both Intel and SPARC Solaris, so I can perform builds and do some testing.

Solaris supports standard POSIX threads. I've been testing XC on Solaris and haven't seen issues around that, though I have seen what look like optimizer bugs (queries that don't complete when the amount of data changes as opposed to simply taking longer).

Matt

From: Koichi Suzuki [mailto:koi...@gm...<mailto:koi...@gm...>]
Sent: Wednesday, October 02, 2013 6:11 PM
To: Michael Paquier
Cc: Matt Warner; Postgres-XC Developers

Subject: Re: [Postgres-xc-developers] Minor Fixes

Thanks guys for your efforts to test XC on Solaris.   This is very valuable work.   If either of you can keep working with Solaris, we may be able to announce that XC supports Solaris.

Michael, I believe you're testing on Intel architecture.   Is either of you going to test XC against SPARC?

I'm not familiar with Solaris threading.   GTM and GTM_Proxy depend upon Posix thread and I hope it has no problem with Solaris.

Regards;
---
Koichi Suzuki

2013/10/3 Michael Paquier <mic...@gm...<mailto:mic...@gm...>>
On Thu, Oct 3, 2013 at 9:15 AM, Matt Warner <MW...@xi...<mailto:MW...@xi...>> wrote:
> I can spend some time cleaning up configure.in<http://configure.in> if it's not stepping on anyone's toes.
Feel free. I will try to work on your patch for GTM proxy first and do
smth for makesgml then.
--
Michael

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Thanks Matt and Michael.   Please keep sharing your work on Solaris and
please share optimizer bug issue.   It will be nice to share table
definition, query and the plan.

Best;
---
Koichi Suzuki


2013/10/3 Matt Warner <MW...@xi...>

> I’m compiling with the Solaris Studio compiler on Solaris Intel right now.
> If it compiles on Intel, it will compile on SPARC, the only exceptions
> being assembly code or code that has hard-coded dependencies on byte
> ordering (big- vs little-endian systems). I don’t recall seeing any code
> like that in XC. That said, I have access to both Intel and SPARC Solaris,
> so I can perform builds and do some testing.****
>
> ** **
>
> Solaris supports standard POSIX threads. I’ve been testing XC on Solaris
> and haven’t seen issues around that, though I have seen what look like
> optimizer bugs (queries that don’t complete when the amount of data changes
> as opposed to simply taking longer).****
>
> ** **
>
> Matt****
>
> ** **
>
> *From:* Koichi Suzuki [mailto:koi...@gm...]
> *Sent:* Wednesday, October 02, 2013 6:11 PM
> *To:* Michael Paquier
> *Cc:* Matt Warner; Postgres-XC Developers
>
> *Subject:* Re: [Postgres-xc-developers] Minor Fixes****
>
> ** **
>
> Thanks guys for your efforts to test XC on Solaris.   This is very
> valuable work.   If either of you can keep working with Solaris, we may be
> able to announce that XC supports Solaris.****
>
> ** **
>
> Michael, I believe you're testing on Intel architecture.   Is either of
> you going to test XC against SPARC?****
>
> ** **
>
> I'm not familiar with Solaris threading.   GTM and GTM_Proxy depend upon
> Posix thread and I hope it has no problem with Solaris.****
>
> ** **
>
> Regards;****
>
> ---****
>
> Koichi Suzuki****
>
> ** **
>
> 2013/10/3 Michael Paquier <mic...@gm...>****
>
> On Thu, Oct 3, 2013 at 9:15 AM, Matt Warner <MW...@xi...> wrote:
> > I can spend some time cleaning up configure.in if it's not stepping on
> anyone's toes.
> Feel free. I will try to work on your patch for GTM proxy first and do
> smth for makesgml then.****
>
> --
> Michael
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers****
>
> ** **
>

I'm compiling with the Solaris Studio compiler on Solaris Intel right now. If it compiles on Intel, it will compile on SPARC, the only exceptions being assembly code or code that has hard-coded dependencies on byte ordering (big- vs little-endian systems). I don't recall seeing any code like that in XC. That said, I have access to both Intel and SPARC Solaris, so I can perform builds and do some testing.

Solaris supports standard POSIX threads. I've been testing XC on Solaris and haven't seen issues around that, though I have seen what look like optimizer bugs (queries that don't complete when the amount of data changes as opposed to simply taking longer).

Matt

From: Koichi Suzuki [mailto:koi...@gm...]
Sent: Wednesday, October 02, 2013 6:11 PM
To: Michael Paquier
Cc: Matt Warner; Postgres-XC Developers
Subject: Re: [Postgres-xc-developers] Minor Fixes

Thanks guys for your efforts to test XC on Solaris.   This is very valuable work.   If either of you can keep working with Solaris, we may be able to announce that XC supports Solaris.

Michael, I believe you're testing on Intel architecture.   Is either of you going to test XC against SPARC?

I'm not familiar with Solaris threading.   GTM and GTM_Proxy depend upon Posix thread and I hope it has no problem with Solaris.

Regards;
---
Koichi Suzuki

2013/10/3 Michael Paquier <mic...@gm...<mailto:mic...@gm...>>
On Thu, Oct 3, 2013 at 9:15 AM, Matt Warner <MW...@xi...<mailto:MW...@xi...>> wrote:
> I can spend some time cleaning up configure.in<http://configure.in> if it's not stepping on anyone's toes.
Feel free. I will try to work on your patch for GTM proxy first and do
smth for makesgml then.
--
Michael

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

On Thu, Oct 3, 2013 at 10:10 AM, Koichi Suzuki <koi...@gm...> wrote:
> Thanks guys for your efforts to test XC on Solaris.   This is very valuable
> work.   If either of you can keep working with Solaris, we may be able to
> announce that XC supports Solaris.
>
> Michael, I believe you're testing on Intel architecture.   Is either of you
> going to test XC against SPARC?
Nope, just got some Intel machines in my OSS lab.
-- 
Michael

Thanks guys for your efforts to test XC on Solaris.   This is very valuable
work.   If either of you can keep working with Solaris, we may be able to
announce that XC supports Solaris.

Michael, I believe you're testing on Intel architecture.   Is either of you
going to test XC against SPARC?

I'm not familiar with Solaris threading.   GTM and GTM_Proxy depend upon
Posix thread and I hope it has no problem with Solaris.

Regards;
---
Koichi Suzuki


2013/10/3 Michael Paquier <mic...@gm...>

> On Thu, Oct 3, 2013 at 9:15 AM, Matt Warner <MW...@xi...> wrote:
> > I can spend some time cleaning up configure.in if it's not stepping on
> anyone's toes.
> Feel free. I will try to work on your patch for GTM proxy first and do
> smth for makesgml then.
> --
> Michael
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Because this is not only for merge resolution.   The code was added before we cut 1_1_STABLE, which will cause assertion exception too.   This should be a part of release 1.1.1.

Regards;
---
Koichi Suzuki

On 2013/10/03, at 0:01, Ahsan Hadi <ahs...@en...<mailto:ahs...@en...>>
 wrote:




On Wed, Oct 2, 2013 at 12:55 PM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote:
Here's revised patch.  I think we can apply this both to our master and REL1_1_STABLE.  (We didn't apply this change to REL1_0_STABLE).

Why do we need to apply this patch to 1_1 if the commit that is causing this is part of 9.3 merge?


Regards;

---
Koichi Suzuki


2013/10/1 鈴木 幸市 <ko...@in...<mailto:ko...@in...>>
Agree to remove #else .. .   Assert is defined under #ifdef ASSERT_ENABLED internally and we don't have to worry about it.  Without this symbol defined, Assert will be expanded to null and there's nothing bothering.

Regards;
---
Koichi Suzuki

On 2013/10/01, at 16:43, Ashutosh Bapat <ash...@en...<mailto:ashutosh..ba...@en...>> wrote:

We should move the Assert under #ifdef ASSERT_ENABLED as well the definition of old_outEnd. We should remove #else defining old_outEnd to handle->outEnd which is utterly wrong.


On Mon, Sep 30, 2013 at 12:15 PM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote:
This change was made to eliminate a warning that old_outEnd is not used.   To improve both, the patch would be as attached.

Regards;

---
Koichi Suzuki


2013/9/30 Abbas Butt <abb...@en...<mailto:abb...@en...>>
Hi,

The commit ae29a966d3286d4e85b3adf162d00f9b1e8b8eae made this change
@@ -1132,7 +1132,11 @@ pgxc_node_send_parse(PGXCNodeHandle * handle, const
char* statement,
        /* message length */
        int                     msgLen;
        int                     cnt_params;
+#if 0
        size_t          old_outEnd = handle->outEnd;
+#else
+#define old_outEnd handle->outEnd
+#endif

which is incorrect. The purpose of the declaration and assignment is to
save the current value of handle->outEnd which is later changed in the
function.
This change makes the following assert in function pgxc_node_send_parse
    Assert(old_outEnd + ntohl(msgLen) + 1 == handle->outEnd);
fail when the query
CREATE TABLE junk AS VALUES (1), (2);
is run.

This patch is required on master as well.

Regards

--
*Abbas*
 Architect

Ph: 92.334.5100153<tel:92.334.5100153>
 Skype ID: gabbasb
www.enterprisedb.co<http://www.enterprisedb.co/>
<http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
*
Follow us on Twitter*
@EnterpriseDB

Visit EnterpriseDB for tutorials, webinars,
whitepapers<http://www.enterprisedb.com/resources-community>and
more<http://www.enterprisedb.com/resources-community>

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers



------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers




--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk_______________________________________________

Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers



------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers




--
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874
Mobile: +92-333-5162114

Website: www.enterprisedb.com<http://www.enterprisedb.com/>
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message..
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

On Thu, Oct 3, 2013 at 9:15 AM, Matt Warner <MW...@xi...> wrote:
> I can spend some time cleaning up configure.in if it's not stepping on anyone's toes.
Feel free. I will try to work on your patch for GTM proxy first and do
smth for makesgml then.
-- 
Michael

I can spend some time cleaning up configure.in if it's not stepping on anyone's toes. 

> On Oct 2, 2013, at 5:07 PM, "Michael Paquier" <mic...@gm...> wrote:
> 
>> On Thu, Oct 3, 2013 at 9:04 AM, Matt Warner <MW...@xi...> wrote:
>> Just to follow up, the change to configure.in is trivial to include "__SOLARIS__", but it looks to me like in the master branch that someone else is already hacking the configure file and not updating configure.in.
>> 
>> Examples:
>> Someone has set CFLAGS="-DPGXC $CFLAGS" in configure but not in configure.in
>> PACKAGE_XC_VERSION, PGXC_MAJORVERSION, and PACKAGE_XC_VERSION are not defined in configure.in
>> PACKAGE_BUGREPORT is set to pgs...@po... in configure.in
>> 
>> Not trying to harp, just wondering how best to get this change incorporated?
> Woah, I completely forgot about that... All those changes should be
> incorporated in configure.in, definitely. That's one more thing to fix
> on top of makesgml.
> Regards,
> -- 
> Michael

On Thu, Oct 3, 2013 at 9:04 AM, Matt Warner <MW...@xi...> wrote:
> Just to follow up, the change to configure.in is trivial to include "__SOLARIS__", but it looks to me like in the master branch that someone else is already hacking the configure file and not updating configure.in.
>
> Examples:
> Someone has set CFLAGS="-DPGXC $CFLAGS" in configure but not in configure.in
> PACKAGE_XC_VERSION, PGXC_MAJORVERSION, and PACKAGE_XC_VERSION are not defined in configure.in
> PACKAGE_BUGREPORT is set to pgs...@po... in configure.in
>
> Not trying to harp, just wondering how best to get this change incorporated?
Woah, I completely forgot about that... All those changes should be
incorporated in configure.in, definitely. That's one more thing to fix
on top of makesgml.
Regards,
-- 
Michael

Just to follow up, the change to configure.in is trivial to include "__SOLARIS__", but it looks to me like in the master branch that someone else is already hacking the configure file and not updating configure.in.

Examples:
Someone has set CFLAGS="-DPGXC $CFLAGS" in configure but not in configure.in
PACKAGE_XC_VERSION, PGXC_MAJORVERSION, and PACKAGE_XC_VERSION are not defined in configure.in
PACKAGE_BUGREPORT is set to pgs...@po... in configure.in

Not trying to harp, just wondering how best to get this change incorporated.

?

Matt

-----Original Message-----
From: Matt Warner 
Sent: Wednesday, October 02, 2013 4:43 PM
To: 'Michael Paquier'
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: RE: [Postgres-xc-developers] Minor Fixes

Hi,

First off, thank you for taking the time to review the patches!

Responses to the items you noted:

1. Solaris 10 is still a very active OS: Oracle shows it's end of support is not until January 2021. I think that means we should not necessarily expect everyone's build environment to be S11. If we can update maksgml to avoid strndup, then problem solved either way. Solaris has excellent backward binary compatibility, meaning what we compile on S9 or S10 will run on S11, but the reverse might or might not be true.

2. I actually did look at updating configure.in at one point in the past but when I ran autoconf, it produced a configure file that looked nothing like what was supplied with the version in the repository. I don't recall at this point which version I was looking at. I see from more recent builds that everything looks OK now, so I accept the -1 as my bad for not doing this the right way. I'm more than happy to update configure.in instead of configure. I can modify it to set something like "__SOLARIS_10__", although in your note 4 you indicated you didn't like the use of the CFLAG.

3. Glad to help!

4. I used the CFLAG because I was unsuccessful in writing a macro to do this. If you'd prefer a macro to the configure and CFLAG route, please let me know and I'll pursue it further. Otherwise, I'll plan to update configure.in

5. Thanks!

Matt

-----Original Message-----
From: Michael Paquier [mailto:mic...@gm...] 
Sent: Wednesday, October 02, 2013 6:45 AM
To: Matt Warner
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: Re: [Postgres-xc-developers] Minor Fixes

Hi,

I have set up a Solaris 11 instance in my lab... So here is some
feedback about those patches:
1) makesgml patch: strndup is available in Solaris 11, not in Solaris
<= 10. IMO this patch is not necessary for 2 reasons:
- makesgml should not use strndup.
- Solaris 10 was out in 2005... That's getting old.
So we should patch makesgml and remove the calls to strndup inside it
instead of creating a fake version of it as you do.
2) configure patch. Not necessary, and -1 for adding a flag like that.
Isn't it possible to use something like __sparc__ or __SOLARIS__
instead? You need also to keep in mind that configure is generated
with autoconfigure, so you would actually need to patch configure.in.
3) Makefile patch, actually this is not only a bug of Solaris, so I
went ahead and committed it. That's a good catch.
4) pgxcnode patch. This is indeed necessary for the compilation on
Solaris, FIONREAD is declared in filio.h, but I don't like the use of
a CFLAGS as you did in configure.
5) GTM proxy patch. It looks like you spotted a couple of other bugs!
There is a strange mixture of things returned from functions where
they shouldn't, and this is not only a problem with Solaris. I'll
rework it a bit and commit it later after checking that it works in my
lab.

Thanks for the report and really sorry for the delay in responding!
Regards,
-- 
Michael

Hi,

First off, thank you for taking the time to review the patches!

Responses to the items you noted:

1. Solaris 10 is still a very active OS: Oracle shows it's end of support is not until January 2021. I think that means we should not necessarily expect everyone's build environment to be S11. If we can update maksgml to avoid strndup, then problem solved either way. Solaris has excellent backward binary compatibility, meaning what we compile on S9 or S10 will run on S11, but the reverse might or might not be true.

2. I actually did look at updating configure.in at one point in the past but when I ran autoconf, it produced a configure file that looked nothing like what was supplied with the version in the repository. I don't recall at this point which version I was looking at. I see from more recent builds that everything looks OK now, so I accept the -1 as my bad for not doing this the right way. I'm more than happy to update configure.in instead of configure. I can modify it to set something like "__SOLARIS_10__", although in your note 4 you indicated you didn't like the use of the CFLAG.

3. Glad to help!

4. I used the CFLAG because I was unsuccessful in writing a macro to do this. If you'd prefer a macro to the configure and CFLAG route, please let me know and I'll pursue it further. Otherwise, I'll plan to update configure.in

5. Thanks!

Matt

-----Original Message-----
From: Michael Paquier [mailto:mic...@gm...] 
Sent: Wednesday, October 02, 2013 6:45 AM
To: Matt Warner
Cc: 鈴木 幸市; Postgres-XC Developers; Koichi Suzuki
Subject: Re: [Postgres-xc-developers] Minor Fixes

Hi,

I have set up a Solaris 11 instance in my lab... So here is some
feedback about those patches:
1) makesgml patch: strndup is available in Solaris 11, not in Solaris
<= 10. IMO this patch is not necessary for 2 reasons:
- makesgml should not use strndup.
- Solaris 10 was out in 2005... That's getting old.
So we should patch makesgml and remove the calls to strndup inside it
instead of creating a fake version of it as you do.
2) configure patch. Not necessary, and -1 for adding a flag like that.
Isn't it possible to use something like __sparc__ or __SOLARIS__
instead? You need also to keep in mind that configure is generated
with autoconfigure, so you would actually need to patch configure.in.
3) Makefile patch, actually this is not only a bug of Solaris, so I
went ahead and committed it. That's a good catch.
4) pgxcnode patch. This is indeed necessary for the compilation on
Solaris, FIONREAD is declared in filio.h, but I don't like the use of
a CFLAGS as you did in configure.
5) GTM proxy patch. It looks like you spotted a couple of other bugs!
There is a strange mixture of things returned from functions where
they shouldn't, and this is not only a problem with Solaris. I'll
rework it a bit and commit it later after checking that it works in my
lab.

Thanks for the report and really sorry for the delay in responding!
Regards,
-- 
Michael

On Wed, Oct 2, 2013 at 12:55 PM, Koichi Suzuki <koi...@gm...>wrote:

> Here's revised patch.  I think we can apply this both to our master and
> REL1_1_STABLE.  (We didn't apply this change to REL1_0_STABLE).
>

Why do we need to apply this patch to 1_1 if the commit that is causing
this is part of 9.3 merge?


>
> Regards;
>
> ---
> Koichi Suzuki
>
>
> 2013/10/1 鈴木 幸市 <ko...@in...>
>
>>  Agree to remove #else .. .   Assert is defined under #ifdef
>> ASSERT_ENABLED internally and we don't have to worry about it.  Without
>> this symbol defined, Assert will be expanded to null and there's nothing
>> bothering.
>>
>>  Regards;
>> ---
>> Koichi Suzuki
>>
>>  On 2013/10/01, at 16:43, Ashutosh Bapat <ash...@en...<ashutosh..ba...@en...>>
>> wrote:
>>
>>  We should move the Assert under #ifdef ASSERT_ENABLED as well the
>> definition of old_outEnd. We should remove #else defining old_outEnd to
>> handle->outEnd which is utterly wrong.
>>
>>
>> On Mon, Sep 30, 2013 at 12:15 PM, Koichi Suzuki <koi...@gm...>wrote:
>>
>>> This change was made to eliminate a warning that old_outEnd is not used.
>>>   To improve both, the patch would be as attached.
>>>
>>>  Regards;
>>>
>>>  ---
>>> Koichi Suzuki
>>>
>>>
>>> 2013/9/30 Abbas Butt <abb...@en...>
>>>
>>>>  Hi,
>>>>
>>>> The commit ae29a966d3286d4e85b3adf162d00f9b1e8b8eae made this change
>>>> @@ -1132,7 +1132,11 @@ pgxc_node_send_parse(PGXCNodeHandle * handle,
>>>> const
>>>> char* statement,
>>>>         /* message length */
>>>>         int                     msgLen;
>>>>         int                     cnt_params;
>>>> +#if 0
>>>>         size_t          old_outEnd = handle->outEnd;
>>>> +#else
>>>> +#define old_outEnd handle->outEnd
>>>> +#endif
>>>>
>>>> which is incorrect. The purpose of the declaration and assignment is to
>>>> save the current value of handle->outEnd which is later changed in the
>>>> function.
>>>> This change makes the following assert in function pgxc_node_send_parse
>>>>     Assert(old_outEnd + ntohl(msgLen) + 1 == handle->outEnd);
>>>> fail when the query
>>>> CREATE TABLE junk AS VALUES (1), (2);
>>>> is run.
>>>>
>>>> This patch is required on master as well.
>>>>
>>>> Regards
>>>>
>>>> --
>>>> *Abbas*
>>>>  Architect
>>>>
>>>> Ph: 92.334.5100153
>>>>  Skype ID: gabbasb
>>>> www.enterprisedb.co
>>>> <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>>>> *
>>>> Follow us on Twitter*
>>>> @EnterpriseDB
>>>>
>>>> Visit EnterpriseDB for tutorials, webinars,
>>>> whitepapers<http://www.enterprisedb.com/resources-community>and
>>>> more<http://www.enterprisedb.com/resources-community>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> October Webinars: Code for Performance
>>>> Free Intel webinars can help you accelerate application performance.
>>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
>>>> most from
>>>> the latest Intel processors and coprocessors. See abstracts and
>>>> register >
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>>> from
>>> the latest Intel processors and coprocessors. See abstracts and register
>>> >
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EnterpriseDB Corporation
>> The Postgres Database Company
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk_______________________________________________
>>
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.

Hi,

I have set up a Solaris 11 instance in my lab... So here is some
feedback about those patches:
1) makesgml patch: strndup is available in Solaris 11, not in Solaris
<= 10. IMO this patch is not necessary for 2 reasons:
- makesgml should not use strndup.
- Solaris 10 was out in 2005... That's getting old.
So we should patch makesgml and remove the calls to strndup inside it
instead of creating a fake version of it as you do.
2) configure patch. Not necessary, and -1 for adding a flag like that.
Isn't it possible to use something like __sparc__ or __SOLARIS__
instead? You need also to keep in mind that configure is generated
with autoconfigure, so you would actually need to patch configure.in.
3) Makefile patch, actually this is not only a bug of Solaris, so I
went ahead and committed it. That's a good catch.
4) pgxcnode patch. This is indeed necessary for the compilation on
Solaris, FIONREAD is declared in filio.h, but I don't like the use of
a CFLAGS as you did in configure.
5) GTM proxy patch. It looks like you spotted a couple of other bugs!
There is a strange mixture of things returned from functions where
they shouldn't, and this is not only a problem with Solaris. I'll
rework it a bit and commit it later after checking that it works in my
lab.

Thanks for the report and really sorry for the delay in responding!
Regards,
-- 
Michael

On Wed, Oct 2, 2013 at 3:45 PM, Koichi Suzuki <koi...@gm...> wrote:
> Okay, I will wait for your test result to commit it.
You shouldn't worry about that too...
-- 
Michael