postgres-xc-developers Mailing List for Postgres-XC (Page 5)

Hi,

patch attached.

Hi,

patch attached.

Now Postgres-XC 1.0.4 and Postgres-XC 1.1.1 are available.   They come
with various fixes/improvements as seen in each release note.

For 1.1.1,
Release note is available at
http://postgres-xc.sourceforge.net/docs/1_1_1/release-xc-1-1-1.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.1/
Document is available at http://postgres-xc.sourceforge.net/docs/1_1_1/

For 1.0.4
Release note is available at
http://postgres-xc.sourceforge.net/docs/1_0_4/release-xc-1-0-4.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.0/
Document is available at http://postgres-xc.sourceforge.net/docs/1_0_4/

Enjoy;
---
Koichi Suzuki

that's the good news.


On Thu, Apr 3, 2014 at 11:54 AM, Koichi Suzuki <koi...@gm...>wrote:

> Postgres-XC development group is pleased to announce that Postgres-XC
> 1.2.1 is out.
>
> This includes many important features from PostgreSQL 9.3 such as
> materialized views, LATERAL, event triggers and automatic updatable
> views, as well as improved distributed query optimizer.   It also
> comes with many fixes and improvement as found in the release note.
>
> Release note is available at
> http://postgres-xc.sourceforge.net/docs/1_2_beta/release-xc-1-2.html
> Source tarball is available at
> https://sourceforge.net/projects/postgres-xc/files/Version_1.2/
> Document is available at http://postgres-xc.sourceforge.net/docs/1_2_1/
>
> Enjoy.
> ---
> Koichi Suzuki
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
Thanks,
Juned Khan
iNextrix Technologies Pvt Ltd.
www.inextrix.com

Postgres-XC development group is pleased to announce that Postgres-XC
1.2.1 is out.

This includes many important features from PostgreSQL 9.3 such as
materialized views, LATERAL, event triggers and automatic updatable
views, as well as improved distributed query optimizer.   It also
comes with many fixes and improvement as found in the release note.

Release note is available at
http://postgres-xc.sourceforge.net/docs/1_2_beta/release-xc-1-2.html
Source tarball is available at
https://sourceforge.net/projects/postgres-xc/files/Version_1.2/
Document is available at http://postgres-xc.sourceforge.net/docs/1_2_1/

Enjoy.
---
Koichi Suzuki

Thank you very much for the patch.   It looks nice.   I tested it
against master and REL1_2_STABLE and found it works fine with the
barrier.   This will be a part of 1.2.1 and current master.

Thank you again.   I'd like to see more of such patches from you.

Regards;
---
Koichi Suzuki


2014-04-01 16:17 GMT+09:00 Wang Diancheng <dia...@gm...>:
> Hi,
>
> pg_xlogdump in current contrib directory could not be compiled, this is
> the patch.
>
> --
> Best Regards,
> Wang Diancheng
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Hi,

pg_xlogdump in current contrib directory could not be compiled, this is
the patch.

-- 
Best Regards,
Wang Diancheng

Yes, it seems to be okay.   Because many people encounter configuration problems, I advice to use pgxc_ctl instead if you are configuring XC cluster supported by pgxc_ctl.

It will be a good idea to set preferred node if your coordinator is configured at the same server as one of the datanodes.
Also, it will be a good idea to select one datanode as primary node to maintain replicated table update/delete.

pgxc_ctl does these automatically.

Regards;
---
Koichi Suzuki

2014/03/26 4:14、Chi-Young Ku <ch...@qu...<mailto:ch...@qu...>> のメール：

Hi,

    I would like to configure a 1-Coor-1-GTM-2-Data Postgres XC instance in a 4-node AWS cluster
using the following steps:

        1. I downloaded Postgres XC source code on each AWS node.

        2. I configured, built and installed Postgres XC on each node.

        3. I initialized the following components:

            node 1: initgtm -Z gtm -D gtm
     node 2: initdb -D datanode1 --nodename dn1 # Initialize Datanode 1
     node 3: initdb -D datanode2 --nodename dn2 # Initialize Datanode 2
     node 4: initdb -D coord1 --nodename co1 # Initialize Coordinator 1

        4. I edited the postgressql.conf file on node 1 to specify the listen_addresses as the address from ifconfig command.

        5. I edited the postgressql.conf file on node 2, 3 and 4 to specify the gtm_host as the listen_address of the GTM.

        6. I started the components:

                node 1: gtm -D gtm
                node 2: postgres -X -D datanode1 -i & # Start Datanode 1
                node 3: postgres -X -D datanode2 -i & # Start Datanode 2
                node 4: postgres -C -D coord1 -i & # Start Coordinator 1

        7. I logged on to the coordinator to create the data nodes:

            On node 4:

            psql postgres

            CREATE NODE dn1 WITH (TYPE='datanode', host=<ip address of node2>);
            CREATE NODE dn2 WITH (TYPE='datanode', host=<ip address of node 3>);
            select pgxc_pool_reload();

    Is the above procedure corrrect?

    Thanks in advance.

Chi


----- Forwarded Message -----
From: Masataka Saito <pg...@gm...<mailto:pg...@gm...>>
To: Abbas Butt <abb...@en...<mailto:abb...@en...>>
Cc: pgxc-hackers mailing list <pos...@li...<mailto:pos...@li...>>
Sent: Tuesday, March 18, 2014 6:31 AM
Subject: Re: [Postgres-xc-developers] Replicated update and delete

Hi,

This patch still have unconcerned path that causes regression.

Here's a script to fail.
----------------------------------------------------------------------------------------------------------------
CREATE TABLE a (
  id INTEGER,
val INTEGER,
PRIMARY KEY (id, val)
)
DISTRIBUTE BY REPLICATION;

INSERT INTO a VALUES(1, 6);

PREPARE ps AS DELETE FROM a WHERE id in (SELECT id FROM a GROUP BY id
HAVING SUM(val) > 0);

EXECUTE ps;
----------------------------------------------------------------------------------------------------------------

Here's an explained plan.
----------------------------------------------------------------------------------------------------------------
                                                QUERY PLAN
------------------------------------------------------------------------------------------------------------
Delete on public.a  (cost=0.12..14.38 rows=5 width=42)
  Primary node/s: datanode1
  Node/s: datanode2
  Remote query: DELETE FROM ONLY public.a WHERE ((a.id = $1) AND (a.val = $2))
  ->  Hash Join  (cost=0.12..14.38 rows=5 width=42)
        Output: a.id, a.val, a.ctid, "ANY_subquery".*
        Hash Cond: ("ANY_subquery".id = a.id)
        ->  Subquery Scan on "ANY_subquery"  (cost=0.00..10.00
rows=1000 width=32)
              Output: "ANY_subquery".*, "ANY_subquery".id
              ->  Data Node Scan on "__REMOTE_GROUP_QUERY__"
(cost=0.00..0.00 rows=1000 width=8)
                    Output: id
                    Node/s: datanode1
                    Remote query: SELECT id FROM ONLY public.a WHERE
true GROUP BY 1 HAVING (sum(val) > 0)
        ->  Hash  (cost=0.00..0.00 rows=1000 width=14)
              Output: a.id, a.val, a.ctid
              ->  Data Node Scan on a "_REMOTE_TABLE_QUERY_"
(cost=0.00..0.00 rows=1000 width=14)
                    Output: a.id, a.val, a.ctid
                    Node/s: datanode1
                    Remote query: SELECT id, val, ctid FROM ONLY
public.a WHERE true
----------------------------------------------------------------------------------------------------------------

The query sends a subquery "DELETE FROM ONLY public.a WHERE ((a.id =
$1) AND (a.val = $2))" to datanodes.
Then the subquery parameters should be 2 integers but a coordinator
tells datanodes that the subquery has 1 tid parameter bound to ctid of
the row to be deleted.
I think this is a remnant from the days of using ctid.

Does anyone have an idea to fix this issue?

Regards.

On 25 February 2014 13:33, Abbas Butt <abb...@en...<mailto:abb...@en...>> wrote:
> Hi,
> Attached please find revised and updated patch that performs updates/deletes
> to replicated tables based on either primary key or unique index under the
> following conditions
> 1. The replicated table has either a primary key or a unique index defined.
> 2. The query is not changing the primary key itself.
> Otherwise ctid is used, like we were using previously.
>
> Getting a clean regression with the patch was not easy. Lot of issues had to
> be fixed especially with deletes.
> Regression and DBT-1 now work cleanly after that patch.
> Test cases are added in the regression to make sure the added functionality
> is working fine.
>
> The attached patch does not include the cursors approach as of now.
> The patch to use cursors in case when primary key cannot be used is not
> complete yet. The changes in the planner are done, but the changes in
> executor are still to do. The current executor does not work with a fetch
> and update plan, and needs some re factoring.
>
> Best Regards
>
>
>
>
> On Mon, Feb 24, 2014 at 10:12 PM, Ahsan Hadi <ahs...@en...<mailto:ahs...@en...>>
> wrote:
>>
>> Abbas,
>> Please give us an update on progress?
>>
>>
>> On Thu, Feb 20, 2014 at 3:38 PM, Abbas Butt <abb...@en...<mailto:abb...@en...>>
>> wrote:
>>>
>>>
>>>
>>>
>>> On Thu, Feb 20, 2014 at 3:25 PM, Ahsan Hadi <ahs...@en...<mailto:ahs...@en...>>
>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Feb 19, 2014 at 3:56 PM, Abbas Butt
>>>> <abb...@en...<mailto:abb...@en...>> wrote:
>>>>>
>>>>> Lets do this
>>>>> 1. If the replicated table has a unique key defined , then use Mason's
>>>>> patch after testing and refinement if required.
>>>>> 2. If the replicated table has no key, then we can use cursors idea
>>>>> proposed by Ashutosh.
>>>>>
>>>>> Does every body agree?
>>>>
>>>>
>>>> Yes.
>>>>
>>>> Can you provide a patch for the above before monday?
>>>
>>>
>>> I will try my best to.
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 19, 2014 at 12:15 PM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>>
>>>>> wrote:
>>>>>>
>>>>>> 2014-02-19 16:00 GMT+09:00 Ahsan Hadi <ahs...@en...<mailto:ahs...@en...>>:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Tue, Feb 18, 2014 at 11:26 PM, Amit Khandekar
>>>>>> > <ami...@en...<mailto:ami...@en...>> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> On 13 February 2014 11:54, Ashutosh Bapat
>>>>>> >> <ash...@en...<mailto:ash...@en...>> wrote:
>>>>>> >>>
>>>>>> >>> One more solution would be to use cursors for replicated tables.
>>>>>> >>> The idea
>>>>>> >>> is to open cursors on all the copies of the table and append the
>>>>>> >>> query with
>>>>>> >>> an ORDER BY clause on all the columns. Thus we are sure that the
>>>>>> >>> current of
>>>>>> >>> each of these cursors point to same row on all the copies. While
>>>>>> >>> fetching a
>>>>>> >>> row from a replicated table, we fetch from all the cursors and
>>>>>> >>> choose only
>>>>>> >>> one row for the data processing. While updating or deleting we
>>>>>> >>> send UPDATE
>>>>>> >>> or DELETE with WHERE CURRENT OF. The down side of this approach is
>>>>>> >>> that, if
>>>>>> >>> there are coordinator quals, we will end up locking more rows than
>>>>>> >>> necessary, increasing the probability of the deadlock but at least
>>>>>> >>> there
>>>>>> >>> won't be a necessary restriction of having primary or unique key
>>>>>> >>> and we
>>>>>> >>> won't break backward compatibility.
>>>>>> >>>
>>>>>> >>> If there two identical rows, we might mix the update from
>>>>>> >>> different
>>>>>> >>> nodes, but then who knew which of them were corresponded across
>>>>>> >>> the nodes to
>>>>>> >>> start with.
>>>>>> >>>
>>>>>> >>
>>>>>> >> Locking all rows doesn't look good especially because we are
>>>>>> >> looking for a
>>>>>> >> permanent long term solution. If we can come up with some other
>>>>>> >> solution
>>>>>> >> that avoids this, we better avoid this compromise. For a replicated
>>>>>> >> table
>>>>>> >> with 10000 rows, all concurrent updates will be serialized even if
>>>>>> >> they are
>>>>>> >> updating a different row.
>>>>>> >>
>>>>>> >> Other thing is datanode performance impact for ORDER BY all
>>>>>> >> columns,
>>>>>> >> especially with many large size columns. I had also mentioned about
>>>>>> >> ORDER BY
>>>>>> >> in approach A. I am not sure whether there is some kind of
>>>>>> >> optimization in
>>>>>> >> the sort, such as: if we find unique rows with the first n columns,
>>>>>> >> it does
>>>>>> >> not compare the rest of the columns.
>>>>>> >>
>>>>>> >> I think declaring cursors is a cool idea in general for DMLs but it
>>>>>> >> requires refactoring of DML planning, and also it requires ORDER
>>>>>> >> BY. There
>>>>>> >> is a concurrent update issue #398 for which we do require a
>>>>>> >> refactor of DML
>>>>>> >> handling. While doing that it will be clearer whether declaring
>>>>>> >> cursor is
>>>>>> >> really beneficial or if it's not feasible. For ORDER BY, again, for
>>>>>> >> long
>>>>>> >> term, we should have a primary key or an internal unique key so
>>>>>> >> that rows
>>>>>> >> can be ordered on that single column as against all columns. So
>>>>>> >> again, we
>>>>>> >> still are better of with a new system column.
>>>>>> >>
>>>>>> >>
>>>>>> >> As regards to approach C, if we find a way to uniquely generate a
>>>>>> >> new row
>>>>>> >> id independently, then the task of generating rowid will be pretty
>>>>>> >> lightweight. We won't require any other table to store it or
>>>>>> >> generate it.
>>>>>> >> The coordinator will generate it at each insert (both fqs and
>>>>>> >> non-fqs), or
>>>>>> >> may be datanodes themselves find a way to generate a new rowid
>>>>>> >> which is
>>>>>> >> always the same regardless of the datanode. A combination of gxid,
>>>>>> >> timestamp
>>>>>> >> and cmd id can be used to construct a unique rowid at the
>>>>>> >> coordinator.
>>>>>> >>
>>>>>> >> I think one action plan can be :
>>>>>> >> 1. Use Mason's patch and tweak it so that it needs very little
>>>>>> >> modification later on if and when we add the system rowid column.
>>>>>> >> 2. Check in the patch but let it not error out if the primary key
>>>>>> >> is not
>>>>>> >> there. This way we would at least make the replicated tables with
>>>>>> >> primary
>>>>>> >> keys work without data issues, but continue to work as it is now
>>>>>> >> for tables
>>>>>> >> without primary key.
>>>>>> >> 3. Lastly support the new system row id implementation, and do an
>>>>>> >> incremental change in Mason's checked in changes to use this id
>>>>>> >> instead of
>>>>>> >> primary key.
>>>>>> >>
>>>>>> >
>>>>>> > Sounds good but as i understand 3 is not targeted for 1.2?
>>>>>>
>>>>>> I agree on this too.  System column will benefit not only this, but
>>>>>> also WCO, more variety of the cursor, and Triggers.  I agree this is
>>>>>> targeted to 1.3 or later.
>>>>>>
>>>>>> Regards;
>>>>>> ---
>>>>>> Koichi Suzuki
>>>>>>
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >>>
>>>>>> >>> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki
>>>>>> >>> <koi...@gm...<mailto:koi...@gm...>>
>>>>>> >>> wrote:
>>>>>> >>>>
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I tested the patch and found that primary key is mandatory.  We
>>>>>> >>>> need
>>>>>> >>>> to modify regression test considerably to give each replicated
>>>>>> >>>> table
>>>>>> >>>> primary keys.
>>>>>> >>>>
>>>>>> >>>> I think this patch helps but I'm not afraid this is good,
>>>>>> >>>> especially
>>>>>> >>>> when we try to take XC features back to PG.
>>>>>> >>>>
>>>>>> >>>> Did you post another patch to use all column values if primary
>>>>>> >>>> key is
>>>>>> >>>> not available?
>>>>>> >>>>
>>>>>> >>>> I think better way is as follows:
>>>>>> >>>>
>>>>>> >>>> 1) If primary key is defined, use it,
>>>>>> >>>> 2) If not, create a primary key as system column, the size should
>>>>>> >>>> be
>>>>>> >>>> 64bit.
>>>>>> >>>> 3) If primary key is added to a replicated table, remove system
>>>>>> >>>> primary
>>>>>> >>>> key.
>>>>>> >>>>
>>>>>> >>>> The value of primary key can be obtained as follows:
>>>>>> >>>>
>>>>>> >>>> 1) add new column to pgxc_class catalog to represent maximum
>>>>>> >>>> value of
>>>>>> >>>> the system primary key,
>>>>>> >>>> 2) when first "insert" is done to the primary node, system
>>>>>> >>>> primary key
>>>>>> >>>> value is taken from 1) and 1) is updated.  The value is returned
>>>>>> >>>> to
>>>>>> >>>> the coordinator to be propagated to other nodes.
>>>>>> >>>> 3) when subsequent "insert" is being done, system primary key
>>>>>> >>>> value is
>>>>>> >>>> added to the column value.  In this case, each datanode updates
>>>>>> >>>> 1)
>>>>>> >>>> column value if it is larger than the current maximum value.
>>>>>> >>>>
>>>>>> >>>> 3) is important to change primary node to another.  This is
>>>>>> >>>> needed to
>>>>>> >>>> carry over the primary node to another.
>>>>>> >>>>
>>>>>> >>>> ALTER TABLE should take care of them.
>>>>>> >>>>
>>>>>> >>>> Other issues are:
>>>>>> >>>>
>>>>>> >>>> 4) pg_dump/pg_dumpall should not include this system column
>>>>>> >>>> value,
>>>>>> >>>> 5) cluster may need to handle this too to repack system primary
>>>>>> >>>> key
>>>>>> >>>> value (not now but at least in 1.3 or later).
>>>>>> >>>>
>>>>>> >>>> Regards;
>>>>>> >>>> ---
>>>>>> >>>> Koichi Suzuki
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...<mailto:ms...@tr...>>:
>>>>>> >>>> > Please see attached patch that tries to address the issue of XC
>>>>>> >>>> > using
>>>>>> >>>> > CTID
>>>>>> >>>> > for replicated updates and deletes when it is evaluated at a
>>>>>> >>>> > coordinator
>>>>>> >>>> > instead of being pushed down.
>>>>>> >>>> >
>>>>>> >>>> > The problem here is that CTID could be referring to a different
>>>>>> >>>> > tuple
>>>>>> >>>> > altogether on a different data node, which is what happened for
>>>>>> >>>> > one of
>>>>>> >>>> > our
>>>>>> >>>> > Postgres-XC support customers, leading to data issues.
>>>>>> >>>> >
>>>>>> >>>> > Instead, the patch looks for a primary key or unique index
>>>>>> >>>> > (with the
>>>>>> >>>> > primary
>>>>>> >>>> > key preferred) and uses those values instead of CTID.
>>>>>> >>>> >
>>>>>> >>>> > The patch could be improved further.  Extra parameters are set
>>>>>> >>>> > even if
>>>>>> >>>> > not
>>>>>> >>>> > used in the execution of the prepared statement sent down to
>>>>>> >>>> > the data
>>>>>> >>>> > nodes.
>>>>>> >>>> >
>>>>>> >>>> > Regards,
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > --
>>>>>> >>>> > Mason Sharp
>>>>>> >>>> >
>>>>>> >>>> > TransLattice - http://www.translattice.com<http://www.translattice.com/>
>>>>>> >>>> > Distributed and Clustered Database Solutions
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > ------------------------------------------------------------------------------
>>>>>> >>>> > November Webinars for C, C++, Fortran Developers
>>>>>> >>>> > Accelerate application performance with scalable programming
>>>>>> >>>> > models.
>>>>>> >>>> > Explore
>>>>>> >>>> > techniques for threading, error checking, porting, and tuning.
>>>>>> >>>> > Get the
>>>>>> >>>> > most
>>>>>> >>>> > from the latest Intel processors and coprocessors. See
>>>>>> >>>> > abstracts and
>>>>>> >>>> > register
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
>>>>>> >>>> > _______________________________________________
>>>>>> >>>> > Postgres-xc-developers mailing list
>>>>>> >>>> > Pos...@li...<mailto:Pos...@li...>
>>>>>> >>>> >
>>>>>> >>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>> >
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> ------------------------------------------------------------------------------
>>>>>> >>>> Android apps run on BlackBerry 10
>>>>>> >>>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>>> Get your Android app in front of a whole new audience.  Start
>>>>>> >>>> now.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>>> _______________________________________________
>>>>>> >>>> Postgres-xc-developers mailing list
>>>>>> >>>> Pos...@li...<mailto:Pos...@li...>
>>>>>> >>>>
>>>>>> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Best Wishes,
>>>>>> >>> Ashutosh Bapat
>>>>>> >>> EnterpriseDB Corporation
>>>>>> >>> The Postgres Database Company
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> ------------------------------------------------------------------------------
>>>>>> >>> Android apps run on BlackBerry 10
>>>>>> >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>> Get your Android app in front of a whole new audience.  Start now.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>> _______________________________________________
>>>>>> >>> Postgres-xc-developers mailing list
>>>>>> >>> Pos...@li...<mailto:Pos...@li...>
>>>>>> >>>
>>>>>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> ------------------------------------------------------------------------------
>>>>>> >> Managing the Performance of Cloud-Based Applications
>>>>>> >> Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> >> Pitfalls.
>>>>>> >> Read the Whitepaper.
>>>>>> >>
>>>>>> >>
>>>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> >>
>>>>>> >> _______________________________________________
>>>>>> >> Postgres-xc-developers mailing list
>>>>>> >> Pos...@li...<mailto:Pos...@li...>
>>>>>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Ahsan Hadi
>>>>>> > Snr Director Product Development
>>>>>> > EnterpriseDB Corporation
>>>>>> > The Enterprise Postgres Company
>>>>>> >
>>>>>> > Phone: +92-51-8358874
>>>>>> > Mobile: +92-333-5162114
>>>>>> >
>>>>>> > Website: www.enterprisedb.com<http://www.enterprisedb.com>
>>>>>> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>>>> > Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>>> >
>>>>>> > This e-mail message (and any attachment) is intended for the use of
>>>>>> > the
>>>>>> > individual or entity to whom it is addressed. This message contains
>>>>>> > information from EnterpriseDB Corporation that may be privileged,
>>>>>> > confidential, or exempt from disclosure under applicable law. If you
>>>>>> > are not
>>>>>> > the intended recipient or authorized to receive this for the
>>>>>> > intended
>>>>>> > recipient, any use, dissemination, distribution, retention,
>>>>>> > archiving, or
>>>>>> > copying of this communication is strictly prohibited. If you have
>>>>>> > received
>>>>>> > this e-mail in error, please notify the sender immediately by reply
>>>>>> > e-mail
>>>>>> > and delete this message.
>>>>>> >
>>>>>> >
>>>>>> > ------------------------------------------------------------------------------
>>>>>> > Managing the Performance of Cloud-Based Applications
>>>>>> > Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> > Pitfalls.
>>>>>> > Read the Whitepaper.
>>>>>> >
>>>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> > _______________________________________________
>>>>>> > Postgres-xc-developers mailing list
>>>>>> > Pos...@li...<mailto:Pos...@li...>
>>>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Managing the Performance of Cloud-Based Applications
>>>>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>>>>> Read the Whitepaper.
>>>>>>
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> _______________________________________________
>>>>>> Postgres-xc-developers mailing list
>>>>>> Pos...@li...<mailto:Pos...@li...>
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Abbas
>>>>> Architect
>>>>>
>>>>> Ph: 92.334.5100153
>>>>> Skype ID: gabbasb
>>>>> www.enterprisedb.com<http://www.enterprisedb.com>
>>>>>
>>>>> Follow us on Twitter
>>>>> @EnterpriseDB
>>>>>
>>>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ahsan Hadi
>>>> Snr Director Product Development
>>>> EnterpriseDB Corporation
>>>> The Enterprise Postgres Company
>>>>
>>>> Phone: +92-51-8358874
>>>> Mobile: +92-333-5162114
>>>>
>>>> Website: www.enterprisedb.com<http://www.enterprisedb.com>
>>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>
>>>> This e-mail message (and any attachment) is intended for the use of the
>>>> individual or entity to whom it is addressed. This message contains
>>>> information from EnterpriseDB Corporation that may be privileged,
>>>> confidential, or exempt from disclosure under applicable law. If you are not
>>>> the intended recipient or authorized to receive this for the intended
>>>> recipient, any use, dissemination, distribution, retention, archiving, or
>>>> copying of this communication is strictly prohibited. If you have received
>>>> this e-mail in error, please notify the sender immediately by reply e-mail
>>>> and delete this message.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Abbas
>>> Architect
>>>
>>> Ph: 92.334.5100153
>>> Skype ID: gabbasb
>>> www.enterprisedb.com<http://www.enterprisedb.com>
>>>
>>> Follow us on Twitter
>>> @EnterpriseDB
>>>
>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>
>>
>>
>>
>> --
>> Ahsan Hadi
>> Snr Director Product Development
>> EnterpriseDB Corporation
>> The Enterprise Postgres Company
>>
>> Phone: +92-51-8358874
>> Mobile: +92-333-5162114
>>
>> Website: www.enterprisedb.com<http://www.enterprisedb.com>
>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>
>> This e-mail message (and any attachment) is intended for the use of the
>> individual or entity to whom it is addressed. This message contains
>> information from EnterpriseDB Corporation that may be privileged,
>> confidential, or exempt from disclosure under applicable law. If you are not
>> the intended recipient or authorized to receive this for the intended
>> recipient, any use, dissemination, distribution, retention, archiving, or
>> copying of this communication is strictly prohibited. If you have received
>> this e-mail in error, please notify the sender immediately by reply e-mail
>> and delete this message.
>
>
>
>
> --
> --
> Abbas
> Architect
>
> Ph: 92.334.5100153
> Skype ID: gabbasb
> www.enterprisedb.com<http://www.enterprisedb.com>
>
> Follow us on Twitter
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...<mailto:Pos...@li...>
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Hi,

    I would like to configure a 1-Coor-1-GTM-2-Data Postgres XC instance in a 4-node AWS cluster
using the following steps:

        1. I downloaded Postgres XC source code on each AWS node.

        2. I configured, built and installed Postgres XC on each node.

        3. I initialized the following components:

            node 1: initgtm -Z gtm -D gtm
     node 2: initdb -D datanode1 --nodename dn1 # Initialize Datanode 1
     node 3: initdb -D datanode2 --nodename dn2 # Initialize Datanode 2
     node 4: initdb -D coord1 --nodename co1 # Initialize Coordinator 1

        4. I edited the postgressql.conf file on node 1 to specify the listen_addresses as the address from ifconfig command.

        5. I edited the postgressql.conf file on node 2, 3 and 4 to specify the gtm_host as the listen_address of the GTM.

        6. I started the components:

                node 1: gtm -D gtm 
                node 2: postgres -X -D datanode1 -i & # Start Datanode 1
                node 3: postgres -X -D datanode2 -i & # Start Datanode 2
                node 4: postgres -C -D coord1 -i & # Start Coordinator 1

        7. I logged on to the coordinator to create the data nodes:

            On node 4:

            psql postgres

            CREATE NODE dn1 WITH (TYPE='datanode', host=<ip address of node2>);
            CREATE NODE dn2 WITH (TYPE='datanode', host=<ip address of node 3>);
            select pgxc_pool_reload();

    Is the above procedure corrrect?

    Thanks in advance.

Chi



----- Forwarded Message -----
From: Masataka Saito <pg...@gm...>
To: Abbas Butt <abb...@en...> 
Cc: pgxc-hackers mailing list <pos...@li...> 
Sent: Tuesday, March 18, 2014 6:31 AM
Subject: Re: [Postgres-xc-developers] Replicated update and delete
 

Hi,

This patch still have unconcerned path that causes regression.

Here's a script to fail.
----------------------------------------------------------------------------------------------------------------
CREATE TABLE a (
  id INTEGER,
val INTEGER,
PRIMARY KEY (id, val)
)
DISTRIBUTE BY REPLICATION;

INSERT INTO a VALUES(1, 6);

PREPARE ps AS DELETE FROM a WHERE id in (SELECT id FROM a GROUP BY id
HAVING SUM(val) > 0);

EXECUTE ps;
----------------------------------------------------------------------------------------------------------------

Here's an explained plan.
----------------------------------------------------------------------------------------------------------------
                                                QUERY PLAN
------------------------------------------------------------------------------------------------------------
Delete on public.a  (cost=0.12..14.38 rows=5 width=42)
   Primary node/s: datanode1
   Node/s: datanode2
   Remote query: DELETE FROM ONLY public.a WHERE ((a.id = $1) AND (a.val = $2))
   ->  Hash Join  (cost=0.12..14.38 rows=5 width=42)
         Output: a.id, a.val, a.ctid, "ANY_subquery".*
         Hash Cond: ("ANY_subquery".id = a.id)
         ->  Subquery Scan on "ANY_subquery"  (cost=0.00..10.00
rows=1000 width=32)
               Output: "ANY_subquery".*, "ANY_subquery".id
               ->  Data Node Scan on "__REMOTE_GROUP_QUERY__"
(cost=0.00..0.00 rows=1000 width=8)
                     Output: id
                     Node/s: datanode1
                     Remote query: SELECT id FROM ONLY public.a WHERE
true GROUP BY 1 HAVING (sum(val) > 0)
         ->  Hash  (cost=0.00..0.00 rows=1000 width=14)
               Output: a.id, a.val, a.ctid
               ->  Data Node Scan on a "_REMOTE_TABLE_QUERY_"
(cost=0.00..0.00 rows=1000 width=14)
                     Output: a.id, a.val, a.ctid
                     Node/s: datanode1
                     Remote query: SELECT id, val, ctid FROM ONLY
public.a WHERE true
----------------------------------------------------------------------------------------------------------------

The query sends a subquery "DELETE FROM ONLY public.a WHERE ((a.id =
$1) AND (a.val = $2))" to datanodes.
Then the subquery parameters should be 2 integers but a coordinator
tells datanodes that the subquery has 1 tid parameter bound to ctid of
the row to be deleted.
I think this is a remnant from the days of using ctid.

Does anyone have an idea to fix this issue?

Regards.

On 25 February 2014 13:33, Abbas Butt <abb...@en...> wrote:
> Hi,
> Attached please find revised and updated patch that performs updates/deletes
> to replicated tables based on either primary key or unique index under the
> following conditions
> 1. The replicated table has either a primary key or a unique index defined.
> 2. The query is not changing the primary key itself.
> Otherwise ctid is used, like we were using previously.
>
> Getting a clean regression with the patch was not easy. Lot of issues had to
> be fixed especially with deletes.
> Regression and DBT-1 now work cleanly after that patch.
> Test cases are added in the regression to make sure the added functionality
> is working fine.
>
> The attached patch does not include the cursors approach as of now.
> The patch to use cursors in case when primary key cannot be used is not
> complete yet. The changes in the planner are done, but the changes in
> executor are still to do. The current executor does not work with a fetch
> and update plan, and needs some re factoring.
>
> Best Regards
>
>
>
>
> On Mon, Feb 24, 2014 at 10:12 PM, Ahsan Hadi <ahs...@en...>
> wrote:
>>
>> Abbas,
>> Please give us an update on progress?
>>
>>
>> On Thu, Feb 20, 2014 at 3:38 PM, Abbas Butt <abb...@en...>
>> wrote:
>>>
>>>
>>>
>>>
>>> On Thu, Feb 20, 2014 at 3:25 PM, Ahsan Hadi <ahs...@en...>
>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Feb 19, 2014 at 3:56 PM, Abbas Butt
>>>> <abb...@en...> wrote:
>>>>>
>>>>> Lets do this
>>>>> 1. If the replicated table has a unique key defined , then use Mason's
>>>>> patch after testing and refinement if required.
>>>>> 2. If the replicated table has no key, then we can use cursors idea
>>>>> proposed by Ashutosh.
>>>>>
>>>>> Does every body agree?
>>>>
>>>>
>>>> Yes.
>>>>
>>>> Can you provide a patch for the above before monday?
>>>
>>>
>>> I will try my best to.
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 19, 2014 at 12:15 PM, Koichi Suzuki <koi...@gm...>
>>>>> wrote:
>>>>>>
>>>>>> 2014-02-19 16:00 GMT+09:00 Ahsan Hadi <ahs...@en...>:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Tue, Feb 18, 2014 at 11:26 PM, Amit Khandekar
>>>>>> > <ami...@en...> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> On 13 February 2014 11:54, Ashutosh Bapat
>>>>>> >> <ash...@en...> wrote:
>>>>>> >>>
>>>>>> >>> One more solution would be to use cursors for replicated tables.
>>>>>> >>> The idea
>>>>>> >>> is to open cursors on all the copies of the table and append the
>>>>>> >>> query with
>>>>>> >>> an ORDER BY clause on all the columns. Thus we are sure that the
>>>>>> >>> current of
>>>>>> >>> each of these cursors point to same row on all the copies. While
>>>>>> >>> fetching a
>>>>>> >>> row from a replicated table, we fetch from all the cursors and
>>>>>> >>> choose only
>>>>>> >>> one row for the data processing. While updating or deleting we
>>>>>> >>> send UPDATE
>>>>>> >>> or DELETE with WHERE CURRENT OF. The down side of this approach is
>>>>>> >>> that, if
>>>>>> >>> there are coordinator quals, we will end up locking more rows than
>>>>>> >>> necessary, increasing the probability of the deadlock but at least
>>>>>> >>> there
>>>>>> >>> won't be a necessary restriction of having primary or unique key
>>>>>> >>> and we
>>>>>> >>> won't break backward compatibility.
>>>>>> >>>
>>>>>> >>> If there two identical rows, we might mix the update from
>>>>>> >>> different
>>>>>> >>> nodes, but then who knew which of them were corresponded across
>>>>>> >>> the nodes to
>>>>>> >>> start with.
>>>>>> >>>
>>>>>> >>
>>>>>> >> Locking all rows doesn't look good especially because we are
>>>>>> >> looking for a
>>>>>> >> permanent long term solution. If we can come up with some other
>>>>>> >> solution
>>>>>> >> that avoids this, we better avoid this compromise. For a replicated
>>>>>> >> table
>>>>>> >> with 10000 rows, all concurrent updates will be serialized even if
>>>>>> >> they are
>>>>>> >> updating a different row.
>>>>>> >>
>>>>>> >> Other thing is datanode performance impact for ORDER BY all
>>>>>> >> columns,
>>>>>> >> especially with many large size columns. I had also mentioned about
>>>>>> >> ORDER BY
>>>>>> >> in approach A. I am not sure whether there is some kind of
>>>>>> >> optimization in
>>>>>> >> the sort, such as: if we find unique rows with the first n columns,
>>>>>> >> it does
>>>>>> >> not compare the rest of the columns.
>>>>>> >>
>>>>>> >> I think declaring cursors is a cool idea in general for DMLs but it
>>>>>> >> requires refactoring of DML planning, and also it requires ORDER
>>>>>> >> BY. There
>>>>>> >> is a concurrent update issue #398 for which we do require a
>>>>>> >> refactor of DML
>>>>>> >> handling. While doing that it will be clearer whether declaring
>>>>>> >> cursor is
>>>>>> >> really beneficial or if it's not feasible. For ORDER BY, again, for
>>>>>> >> long
>>>>>> >> term, we should have a primary key or an internal unique key so
>>>>>> >> that rows
>>>>>> >> can be ordered on that single column as against all columns. So
>>>>>> >> again, we
>>>>>> >> still are better of with a new system column.
>>>>>> >>
>>>>>> >>
>>>>>> >> As regards to approach C, if we find a way to uniquely generate a
>>>>>> >> new row
>>>>>> >> id independently, then the task of generating rowid will be pretty
>>>>>> >> lightweight. We won't require any other table to store it or
>>>>>> >> generate it.
>>>>>> >> The coordinator will generate it at each insert (both fqs and
>>>>>> >> non-fqs), or
>>>>>> >> may be datanodes themselves find a way to generate a new rowid
>>>>>> >> which is
>>>>>> >> always the same regardless of the datanode. A combination of gxid,
>>>>>> >> timestamp
>>>>>> >> and cmd id can be used to construct a unique rowid at the
>>>>>> >> coordinator.
>>>>>> >>
>>>>>> >> I think one action plan can be :
>>>>>> >> 1. Use Mason's patch and tweak it so that it needs very little
>>>>>> >> modification later on if and when we add the system rowid column.
>>>>>> >> 2. Check in the patch but let it not error out if the primary key
>>>>>> >> is not
>>>>>> >> there. This way we would at least make the replicated tables with
>>>>>> >> primary
>>>>>> >> keys work without data issues, but continue to work as it is now
>>>>>> >> for tables
>>>>>> >> without primary key.
>>>>>> >> 3. Lastly support the new system row id implementation, and do an
>>>>>> >> incremental change in Mason's checked in changes to use this id
>>>>>> >> instead of
>>>>>> >> primary key.
>>>>>> >>
>>>>>> >
>>>>>> > Sounds good but as i understand 3 is not targeted for 1.2?
>>>>>>
>>>>>> I agree on this too.   System column will benefit not only this, but
>>>>>> also WCO, more variety of the cursor, and Triggers.   I agree this is
>>>>>> targeted to 1.3 or later.
>>>>>>
>>>>>> Regards;
>>>>>> ---
>>>>>> Koichi Suzuki
>>>>>>
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >>>
>>>>>> >>> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki
>>>>>> >>> <koi...@gm...>
>>>>>> >>> wrote:
>>>>>> >>>>
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I tested the patch and found that primary key is mandatory.   We
>>>>>> >>>> need
>>>>>> >>>> to modify regression test considerably to give each replicated
>>>>>> >>>> table
>>>>>> >>>> primary keys.
>>>>>> >>>>
>>>>>> >>>> I think this patch helps but I'm not afraid this is good,
>>>>>> >>>> especially
>>>>>> >>>> when we try to take XC features back to PG.
>>>>>> >>>>
>>>>>> >>>> Did you post another patch to use all column values if primary
>>>>>> >>>> key is
>>>>>> >>>> not available?
>>>>>> >>>>
>>>>>> >>>> I think better way is as follows:
>>>>>> >>>>
>>>>>> >>>> 1) If primary key is defined, use it,
>>>>>> >>>> 2) If not, create a primary key as system column, the size should
>>>>>> >>>> be
>>>>>> >>>> 64bit.
>>>>>> >>>> 3) If primary key is added to a replicated table, remove system
>>>>>> >>>> primary
>>>>>> >>>> key.
>>>>>> >>>>
>>>>>> >>>> The value of primary key can be obtained as follows:
>>>>>> >>>>
>>>>>> >>>> 1) add new column to pgxc_class catalog to represent maximum
>>>>>> >>>> value of
>>>>>> >>>> the system primary key,
>>>>>> >>>> 2) when first "insert" is done to the primary node, system
>>>>>> >>>> primary key
>>>>>> >>>> value is taken from 1) and 1) is updated.  The value is returned
>>>>>> >>>> to
>>>>>> >>>> the coordinator to be propagated to other nodes.
>>>>>> >>>> 3) when subsequent "insert" is being done, system primary key
>>>>>> >>>> value is
>>>>>> >>>> added to the column value.   In this case, each datanode updates
>>>>>> >>>> 1)
>>>>>> >>>> column value if it is larger than the current maximum value.
>>>>>> >>>>
>>>>>> >>>> 3) is important to change primary node to another.   This is
>>>>>> >>>> needed to
>>>>>> >>>> carry over the primary node to another.
>>>>>> >>>>
>>>>>> >>>> ALTER TABLE should take care of them.
>>>>>> >>>>
>>>>>> >>>> Other issues are:
>>>>>> >>>>
>>>>>> >>>> 4) pg_dump/pg_dumpall should not include this system column
>>>>>> >>>> value,
>>>>>> >>>> 5) cluster may need to handle this too to repack system primary
>>>>>> >>>> key
>>>>>> >>>> value (not now but at least in 1.3 or later).
>>>>>> >>>>
>>>>>> >>>> Regards;
>>>>>> >>>> ---
>>>>>> >>>> Koichi Suzuki
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>:
>>>>>> >>>> > Please see attached patch that tries to address the issue of XC
>>>>>> >>>> > using
>>>>>> >>>> > CTID
>>>>>> >>>> > for replicated updates and deletes when it is evaluated at a
>>>>>> >>>> > coordinator
>>>>>> >>>> > instead of being pushed down.
>>>>>> >>>> >
>>>>>> >>>> > The problem here is that CTID could be referring to a different
>>>>>> >>>> > tuple
>>>>>> >>>> > altogether on a different data node, which is what happened for
>>>>>> >>>> > one of
>>>>>> >>>> > our
>>>>>> >>>> > Postgres-XC support customers, leading to data issues.
>>>>>> >>>> >
>>>>>> >>>> > Instead, the patch looks for a primary key or unique index
>>>>>> >>>> > (with the
>>>>>> >>>> > primary
>>>>>> >>>> > key preferred) and uses those values instead of CTID.
>>>>>> >>>> >
>>>>>> >>>> > The patch could be improved further.  Extra parameters are set
>>>>>> >>>> > even if
>>>>>> >>>> > not
>>>>>> >>>> > used in the execution of the prepared statement sent down to
>>>>>> >>>> > the data
>>>>>> >>>> > nodes.
>>>>>> >>>> >
>>>>>> >>>> > Regards,
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > --
>>>>>> >>>> > Mason Sharp
>>>>>> >>>> >
>>>>>> >>>> > TransLattice - http://www.translattice.com
>>>>>> >>>> > Distributed and Clustered Database Solutions
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > ------------------------------------------------------------------------------
>>>>>> >>>> > November Webinars for C, C++, Fortran Developers
>>>>>> >>>> > Accelerate application performance with scalable programming
>>>>>> >>>> > models.
>>>>>> >>>> > Explore
>>>>>> >>>> > techniques for threading, error checking, porting, and tuning.
>>>>>> >>>> > Get the
>>>>>> >>>> > most
>>>>>> >>>> > from the latest Intel processors and coprocessors. See
>>>>>> >>>> > abstracts and
>>>>>> >>>> > register
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
>>>>>> >>>> > _______________________________________________
>>>>>> >>>> > Postgres-xc-developers mailing list
>>>>>> >>>> > Pos...@li...
>>>>>> >>>> >
>>>>>> >>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>> >
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> ------------------------------------------------------------------------------
>>>>>> >>>> Android apps run on BlackBerry 10
>>>>>> >>>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>>> Get your Android app in front of a whole new audience.  Start
>>>>>> >>>> now.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>>> _______________________________________________
>>>>>> >>>> Postgres-xc-developers mailing list
>>>>>> >>>> Pos...@li...
>>>>>> >>>>
>>>>>> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Best Wishes,
>>>>>> >>> Ashutosh Bapat
>>>>>> >>> EnterpriseDB Corporation
>>>>>> >>> The Postgres Database Company
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> ------------------------------------------------------------------------------
>>>>>> >>> Android apps run on BlackBerry 10
>>>>>> >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>> Get your Android app in front of a whole new audience.  Start now.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>> _______________________________________________
>>>>>> >>> Postgres-xc-developers mailing list
>>>>>> >>> Pos...@li...
>>>>>> >>>
>>>>>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> ------------------------------------------------------------------------------
>>>>>> >> Managing the Performance of Cloud-Based Applications
>>>>>> >> Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> >> Pitfalls.
>>>>>> >> Read the Whitepaper.
>>>>>> >>
>>>>>> >>
>>>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> >>
>>>>>> >> _______________________________________________
>>>>>> >> Postgres-xc-developers mailing list
>>>>>> >> Pos...@li...
>>>>>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Ahsan Hadi
>>>>>> > Snr Director Product Development
>>>>>> > EnterpriseDB Corporation
>>>>>> > The Enterprise Postgres Company
>>>>>> >
>>>>>> > Phone: +92-51-8358874
>>>>>> > Mobile: +92-333-5162114
>>>>>> >
>>>>>> > Website: www.enterprisedb.com
>>>>>> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>>>> > Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>>> >
>>>>>> > This e-mail message (and any attachment) is intended for the use of
>>>>>> > the
>>>>>> > individual or entity to whom it is addressed. This message contains
>>>>>> > information from EnterpriseDB Corporation that may be privileged,
>>>>>> > confidential, or exempt from disclosure under applicable law. If you
>>>>>> > are not
>>>>>> > the intended recipient or authorized to receive this for the
>>>>>> > intended
>>>>>> > recipient, any use, dissemination, distribution, retention,
>>>>>> > archiving, or
>>>>>> > copying of this communication is strictly prohibited. If you have
>>>>>> > received
>>>>>> > this e-mail in error, please notify the sender immediately by reply
>>>>>> > e-mail
>>>>>> > and delete this message.
>>>>>> >
>>>>>> >
>>>>>> > ------------------------------------------------------------------------------
>>>>>> > Managing the Performance of Cloud-Based Applications
>>>>>> > Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> > Pitfalls.
>>>>>> > Read the Whitepaper.
>>>>>> >
>>>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> > _______________________________________________
>>>>>> > Postgres-xc-developers mailing list
>>>>>> > Pos...@li...
>>>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Managing the Performance of Cloud-Based Applications
>>>>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>>>>> Read the Whitepaper.
>>>>>>
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> _______________________________________________
>>>>>> Postgres-xc-developers mailing list
>>>>>> Pos...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Abbas
>>>>> Architect
>>>>>
>>>>> Ph: 92.334.5100153
>>>>> Skype ID: gabbasb
>>>>> www.enterprisedb.com
>>>>>
>>>>> Follow us on Twitter
>>>>> @EnterpriseDB
>>>>>
>>>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ahsan Hadi
>>>> Snr Director Product Development
>>>> EnterpriseDB Corporation
>>>> The Enterprise Postgres Company
>>>>
>>>> Phone: +92-51-8358874
>>>> Mobile: +92-333-5162114
>>>>
>>>> Website: www.enterprisedb.com
>>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>
>>>> This e-mail message (and any attachment) is intended for the use of the
>>>> individual or entity to whom it is addressed. This message contains
>>>> information from EnterpriseDB Corporation that may be privileged,
>>>> confidential, or exempt from disclosure under applicable law. If you are not
>>>> the intended recipient or authorized to receive this for the intended
>>>> recipient, any use, dissemination, distribution, retention, archiving, or
>>>> copying of this communication is strictly prohibited. If you have received
>>>> this e-mail in error, please notify the sender immediately by reply e-mail
>>>> and delete this message.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Abbas
>>> Architect
>>>
>>> Ph: 92.334.5100153
>>> Skype ID: gabbasb
>>> www.enterprisedb.com
>>>
>>> Follow us on Twitter
>>> @EnterpriseDB
>>>
>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>
>>
>>
>>
>> --
>> Ahsan Hadi
>> Snr Director Product Development
>> EnterpriseDB Corporation
>> The Enterprise Postgres Company
>>
>> Phone: +92-51-8358874
>> Mobile: +92-333-5162114
>>
>> Website: www.enterprisedb.com
>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>
>> This e-mail message (and any attachment) is intended for the use of the
>> individual or entity to whom it is addressed. This message contains
>> information from EnterpriseDB Corporation that may be privileged,
>> confidential, or exempt from disclosure under applicable law. If you are not
>> the intended recipient or authorized to receive this for the intended
>> recipient, any use, dissemination, distribution, retention, archiving, or
>> copying of this communication is strictly prohibited. If you have received
>> this e-mail in error, please notify the sender immediately by reply e-mail
>> and delete this message.
>
>
>
>
> --
> --
> Abbas
> Architect
>
> Ph: 92.334.5100153
> Skype ID: gabbasb
> www.enterprisedb.com
>
> Follow us on Twitter
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Hi,

This patch still have unconcerned path that causes regression.

Here's a script to fail.
----------------------------------------------------------------------------------------------------------------
CREATE TABLE a (
  id INTEGER,
val INTEGER,
PRIMARY KEY (id, val)
)
DISTRIBUTE BY REPLICATION;

INSERT INTO a VALUES(1, 6);

PREPARE ps AS DELETE FROM a WHERE id in (SELECT id FROM a GROUP BY id
HAVING SUM(val) > 0);

EXECUTE ps;
----------------------------------------------------------------------------------------------------------------

Here's an explained plan.
----------------------------------------------------------------------------------------------------------------
                                                QUERY PLAN
------------------------------------------------------------------------------------------------------------
 Delete on public.a  (cost=0.12..14.38 rows=5 width=42)
   Primary node/s: datanode1
   Node/s: datanode2
   Remote query: DELETE FROM ONLY public.a WHERE ((a.id = $1) AND (a.val = $2))
   ->  Hash Join  (cost=0.12..14.38 rows=5 width=42)
         Output: a.id, a.val, a.ctid, "ANY_subquery".*
         Hash Cond: ("ANY_subquery".id = a.id)
         ->  Subquery Scan on "ANY_subquery"  (cost=0.00..10.00
rows=1000 width=32)
               Output: "ANY_subquery".*, "ANY_subquery".id
               ->  Data Node Scan on "__REMOTE_GROUP_QUERY__"
(cost=0.00..0.00 rows=1000 width=8)
                     Output: id
                     Node/s: datanode1
                     Remote query: SELECT id FROM ONLY public.a WHERE
true GROUP BY 1 HAVING (sum(val) > 0)
         ->  Hash  (cost=0.00..0.00 rows=1000 width=14)
               Output: a.id, a.val, a.ctid
               ->  Data Node Scan on a "_REMOTE_TABLE_QUERY_"
(cost=0.00..0.00 rows=1000 width=14)
                     Output: a.id, a.val, a.ctid
                     Node/s: datanode1
                     Remote query: SELECT id, val, ctid FROM ONLY
public.a WHERE true
----------------------------------------------------------------------------------------------------------------

The query sends a subquery "DELETE FROM ONLY public.a WHERE ((a.id =
$1) AND (a.val = $2))" to datanodes.
Then the subquery parameters should be 2 integers but a coordinator
tells datanodes that the subquery has 1 tid parameter bound to ctid of
the row to be deleted.
I think this is a remnant from the days of using ctid.

Does anyone have an idea to fix this issue?

Regards.

On 25 February 2014 13:33, Abbas Butt <abb...@en...> wrote:
> Hi,
> Attached please find revised and updated patch that performs updates/deletes
> to replicated tables based on either primary key or unique index under the
> following conditions
> 1. The replicated table has either a primary key or a unique index defined.
> 2. The query is not changing the primary key itself.
> Otherwise ctid is used, like we were using previously.
>
> Getting a clean regression with the patch was not easy. Lot of issues had to
> be fixed especially with deletes.
> Regression and DBT-1 now work cleanly after that patch.
> Test cases are added in the regression to make sure the added functionality
> is working fine.
>
> The attached patch does not include the cursors approach as of now.
> The patch to use cursors in case when primary key cannot be used is not
> complete yet. The changes in the planner are done, but the changes in
> executor are still to do. The current executor does not work with a fetch
> and update plan, and needs some re factoring.
>
> Best Regards
>
>
>
>
> On Mon, Feb 24, 2014 at 10:12 PM, Ahsan Hadi <ahs...@en...>
> wrote:
>>
>> Abbas,
>> Please give us an update on progress?
>>
>>
>> On Thu, Feb 20, 2014 at 3:38 PM, Abbas Butt <abb...@en...>
>> wrote:
>>>
>>>
>>>
>>>
>>> On Thu, Feb 20, 2014 at 3:25 PM, Ahsan Hadi <ahs...@en...>
>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Feb 19, 2014 at 3:56 PM, Abbas Butt
>>>> <abb...@en...> wrote:
>>>>>
>>>>> Lets do this
>>>>> 1. If the replicated table has a unique key defined , then use Mason's
>>>>> patch after testing and refinement if required.
>>>>> 2. If the replicated table has no key, then we can use cursors idea
>>>>> proposed by Ashutosh.
>>>>>
>>>>> Does every body agree?
>>>>
>>>>
>>>> Yes.
>>>>
>>>> Can you provide a patch for the above before monday?
>>>
>>>
>>> I will try my best to.
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 19, 2014 at 12:15 PM, Koichi Suzuki <koi...@gm...>
>>>>> wrote:
>>>>>>
>>>>>> 2014-02-19 16:00 GMT+09:00 Ahsan Hadi <ahs...@en...>:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Tue, Feb 18, 2014 at 11:26 PM, Amit Khandekar
>>>>>> > <ami...@en...> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> On 13 February 2014 11:54, Ashutosh Bapat
>>>>>> >> <ash...@en...> wrote:
>>>>>> >>>
>>>>>> >>> One more solution would be to use cursors for replicated tables.
>>>>>> >>> The idea
>>>>>> >>> is to open cursors on all the copies of the table and append the
>>>>>> >>> query with
>>>>>> >>> an ORDER BY clause on all the columns. Thus we are sure that the
>>>>>> >>> current of
>>>>>> >>> each of these cursors point to same row on all the copies. While
>>>>>> >>> fetching a
>>>>>> >>> row from a replicated table, we fetch from all the cursors and
>>>>>> >>> choose only
>>>>>> >>> one row for the data processing. While updating or deleting we
>>>>>> >>> send UPDATE
>>>>>> >>> or DELETE with WHERE CURRENT OF. The down side of this approach is
>>>>>> >>> that, if
>>>>>> >>> there are coordinator quals, we will end up locking more rows than
>>>>>> >>> necessary, increasing the probability of the deadlock but at least
>>>>>> >>> there
>>>>>> >>> won't be a necessary restriction of having primary or unique key
>>>>>> >>> and we
>>>>>> >>> won't break backward compatibility.
>>>>>> >>>
>>>>>> >>> If there two identical rows, we might mix the update from
>>>>>> >>> different
>>>>>> >>> nodes, but then who knew which of them were corresponded across
>>>>>> >>> the nodes to
>>>>>> >>> start with.
>>>>>> >>>
>>>>>> >>
>>>>>> >> Locking all rows doesn't look good especially because we are
>>>>>> >> looking for a
>>>>>> >> permanent long term solution. If we can come up with some other
>>>>>> >> solution
>>>>>> >> that avoids this, we better avoid this compromise. For a replicated
>>>>>> >> table
>>>>>> >> with 10000 rows, all concurrent updates will be serialized even if
>>>>>> >> they are
>>>>>> >> updating a different row.
>>>>>> >>
>>>>>> >> Other thing is datanode performance impact for ORDER BY all
>>>>>> >> columns,
>>>>>> >> especially with many large size columns. I had also mentioned about
>>>>>> >> ORDER BY
>>>>>> >> in approach A. I am not sure whether there is some kind of
>>>>>> >> optimization in
>>>>>> >> the sort, such as: if we find unique rows with the first n columns,
>>>>>> >> it does
>>>>>> >> not compare the rest of the columns.
>>>>>> >>
>>>>>> >> I think declaring cursors is a cool idea in general for DMLs but it
>>>>>> >> requires refactoring of DML planning, and also it requires ORDER
>>>>>> >> BY. There
>>>>>> >> is a concurrent update issue #398 for which we do require a
>>>>>> >> refactor of DML
>>>>>> >> handling. While doing that it will be clearer whether declaring
>>>>>> >> cursor is
>>>>>> >> really beneficial or if it's not feasible. For ORDER BY, again, for
>>>>>> >> long
>>>>>> >> term, we should have a primary key or an internal unique key so
>>>>>> >> that rows
>>>>>> >> can be ordered on that single column as against all columns. So
>>>>>> >> again, we
>>>>>> >> still are better of with a new system column.
>>>>>> >>
>>>>>> >>
>>>>>> >> As regards to approach C, if we find a way to uniquely generate a
>>>>>> >> new row
>>>>>> >> id independently, then the task of generating rowid will be pretty
>>>>>> >> lightweight. We won't require any other table to store it or
>>>>>> >> generate it.
>>>>>> >> The coordinator will generate it at each insert (both fqs and
>>>>>> >> non-fqs), or
>>>>>> >> may be datanodes themselves find a way to generate a new rowid
>>>>>> >> which is
>>>>>> >> always the same regardless of the datanode. A combination of gxid,
>>>>>> >> timestamp
>>>>>> >> and cmd id can be used to construct a unique rowid at the
>>>>>> >> coordinator.
>>>>>> >>
>>>>>> >> I think one action plan can be :
>>>>>> >> 1. Use Mason's patch and tweak it so that it needs very little
>>>>>> >> modification later on if and when we add the system rowid column.
>>>>>> >> 2. Check in the patch but let it not error out if the primary key
>>>>>> >> is not
>>>>>> >> there. This way we would at least make the replicated tables with
>>>>>> >> primary
>>>>>> >> keys work without data issues, but continue to work as it is now
>>>>>> >> for tables
>>>>>> >> without primary key.
>>>>>> >> 3. Lastly support the new system row id implementation, and do an
>>>>>> >> incremental change in Mason's checked in changes to use this id
>>>>>> >> instead of
>>>>>> >> primary key.
>>>>>> >>
>>>>>> >
>>>>>> > Sounds good but as i understand 3 is not targeted for 1.2?
>>>>>>
>>>>>> I agree on this too.   System column will benefit not only this, but
>>>>>> also WCO, more variety of the cursor, and Triggers.   I agree this is
>>>>>> targeted to 1.3 or later.
>>>>>>
>>>>>> Regards;
>>>>>> ---
>>>>>> Koichi Suzuki
>>>>>>
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >>>
>>>>>> >>> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki
>>>>>> >>> <koi...@gm...>
>>>>>> >>> wrote:
>>>>>> >>>>
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I tested the patch and found that primary key is mandatory.   We
>>>>>> >>>> need
>>>>>> >>>> to modify regression test considerably to give each replicated
>>>>>> >>>> table
>>>>>> >>>> primary keys.
>>>>>> >>>>
>>>>>> >>>> I think this patch helps but I'm not afraid this is good,
>>>>>> >>>> especially
>>>>>> >>>> when we try to take XC features back to PG.
>>>>>> >>>>
>>>>>> >>>> Did you post another patch to use all column values if primary
>>>>>> >>>> key is
>>>>>> >>>> not available?
>>>>>> >>>>
>>>>>> >>>> I think better way is as follows:
>>>>>> >>>>
>>>>>> >>>> 1) If primary key is defined, use it,
>>>>>> >>>> 2) If not, create a primary key as system column, the size should
>>>>>> >>>> be
>>>>>> >>>> 64bit.
>>>>>> >>>> 3) If primary key is added to a replicated table, remove system
>>>>>> >>>> primary
>>>>>> >>>> key.
>>>>>> >>>>
>>>>>> >>>> The value of primary key can be obtained as follows:
>>>>>> >>>>
>>>>>> >>>> 1) add new column to pgxc_class catalog to represent maximum
>>>>>> >>>> value of
>>>>>> >>>> the system primary key,
>>>>>> >>>> 2) when first "insert" is done to the primary node, system
>>>>>> >>>> primary key
>>>>>> >>>> value is taken from 1) and 1) is updated.  The value is returned
>>>>>> >>>> to
>>>>>> >>>> the coordinator to be propagated to other nodes.
>>>>>> >>>> 3) when subsequent "insert" is being done, system primary key
>>>>>> >>>> value is
>>>>>> >>>> added to the column value.   In this case, each datanode updates
>>>>>> >>>> 1)
>>>>>> >>>> column value if it is larger than the current maximum value.
>>>>>> >>>>
>>>>>> >>>> 3) is important to change primary node to another.   This is
>>>>>> >>>> needed to
>>>>>> >>>> carry over the primary node to another.
>>>>>> >>>>
>>>>>> >>>> ALTER TABLE should take care of them.
>>>>>> >>>>
>>>>>> >>>> Other issues are:
>>>>>> >>>>
>>>>>> >>>> 4) pg_dump/pg_dumpall should not include this system column
>>>>>> >>>> value,
>>>>>> >>>> 5) cluster may need to handle this too to repack system primary
>>>>>> >>>> key
>>>>>> >>>> value (not now but at least in 1.3 or later).
>>>>>> >>>>
>>>>>> >>>> Regards;
>>>>>> >>>> ---
>>>>>> >>>> Koichi Suzuki
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>:
>>>>>> >>>> > Please see attached patch that tries to address the issue of XC
>>>>>> >>>> > using
>>>>>> >>>> > CTID
>>>>>> >>>> > for replicated updates and deletes when it is evaluated at a
>>>>>> >>>> > coordinator
>>>>>> >>>> > instead of being pushed down.
>>>>>> >>>> >
>>>>>> >>>> > The problem here is that CTID could be referring to a different
>>>>>> >>>> > tuple
>>>>>> >>>> > altogether on a different data node, which is what happened for
>>>>>> >>>> > one of
>>>>>> >>>> > our
>>>>>> >>>> > Postgres-XC support customers, leading to data issues.
>>>>>> >>>> >
>>>>>> >>>> > Instead, the patch looks for a primary key or unique index
>>>>>> >>>> > (with the
>>>>>> >>>> > primary
>>>>>> >>>> > key preferred) and uses those values instead of CTID.
>>>>>> >>>> >
>>>>>> >>>> > The patch could be improved further.  Extra parameters are set
>>>>>> >>>> > even if
>>>>>> >>>> > not
>>>>>> >>>> > used in the execution of the prepared statement sent down to
>>>>>> >>>> > the data
>>>>>> >>>> > nodes.
>>>>>> >>>> >
>>>>>> >>>> > Regards,
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > --
>>>>>> >>>> > Mason Sharp
>>>>>> >>>> >
>>>>>> >>>> > TransLattice - http://www.translattice.com
>>>>>> >>>> > Distributed and Clustered Database Solutions
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > ------------------------------------------------------------------------------
>>>>>> >>>> > November Webinars for C, C++, Fortran Developers
>>>>>> >>>> > Accelerate application performance with scalable programming
>>>>>> >>>> > models.
>>>>>> >>>> > Explore
>>>>>> >>>> > techniques for threading, error checking, porting, and tuning.
>>>>>> >>>> > Get the
>>>>>> >>>> > most
>>>>>> >>>> > from the latest Intel processors and coprocessors. See
>>>>>> >>>> > abstracts and
>>>>>> >>>> > register
>>>>>> >>>> >
>>>>>> >>>> >
>>>>>> >>>> > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
>>>>>> >>>> > _______________________________________________
>>>>>> >>>> > Postgres-xc-developers mailing list
>>>>>> >>>> > Pos...@li...
>>>>>> >>>> >
>>>>>> >>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>> >
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> ------------------------------------------------------------------------------
>>>>>> >>>> Android apps run on BlackBerry 10
>>>>>> >>>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>>> Get your Android app in front of a whole new audience.  Start
>>>>>> >>>> now.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>>> _______________________________________________
>>>>>> >>>> Postgres-xc-developers mailing list
>>>>>> >>>> Pos...@li...
>>>>>> >>>>
>>>>>> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Best Wishes,
>>>>>> >>> Ashutosh Bapat
>>>>>> >>> EnterpriseDB Corporation
>>>>>> >>> The Postgres Database Company
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> ------------------------------------------------------------------------------
>>>>>> >>> Android apps run on BlackBerry 10
>>>>>> >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
>>>>>> >>> Now with support for Jelly Bean, Bluetooth, Mapview and more.
>>>>>> >>> Get your Android app in front of a whole new audience.  Start now.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
>>>>>> >>> _______________________________________________
>>>>>> >>> Postgres-xc-developers mailing list
>>>>>> >>> Pos...@li...
>>>>>> >>>
>>>>>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> ------------------------------------------------------------------------------
>>>>>> >> Managing the Performance of Cloud-Based Applications
>>>>>> >> Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> >> Pitfalls.
>>>>>> >> Read the Whitepaper.
>>>>>> >>
>>>>>> >>
>>>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> >>
>>>>>> >> _______________________________________________
>>>>>> >> Postgres-xc-developers mailing list
>>>>>> >> Pos...@li...
>>>>>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Ahsan Hadi
>>>>>> > Snr Director Product Development
>>>>>> > EnterpriseDB Corporation
>>>>>> > The Enterprise Postgres Company
>>>>>> >
>>>>>> > Phone: +92-51-8358874
>>>>>> > Mobile: +92-333-5162114
>>>>>> >
>>>>>> > Website: www.enterprisedb.com
>>>>>> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>>>> > Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>>> >
>>>>>> > This e-mail message (and any attachment) is intended for the use of
>>>>>> > the
>>>>>> > individual or entity to whom it is addressed. This message contains
>>>>>> > information from EnterpriseDB Corporation that may be privileged,
>>>>>> > confidential, or exempt from disclosure under applicable law. If you
>>>>>> > are not
>>>>>> > the intended recipient or authorized to receive this for the
>>>>>> > intended
>>>>>> > recipient, any use, dissemination, distribution, retention,
>>>>>> > archiving, or
>>>>>> > copying of this communication is strictly prohibited. If you have
>>>>>> > received
>>>>>> > this e-mail in error, please notify the sender immediately by reply
>>>>>> > e-mail
>>>>>> > and delete this message.
>>>>>> >
>>>>>> >
>>>>>> > ------------------------------------------------------------------------------
>>>>>> > Managing the Performance of Cloud-Based Applications
>>>>>> > Take advantage of what the Cloud has to offer - Avoid Common
>>>>>> > Pitfalls.
>>>>>> > Read the Whitepaper.
>>>>>> >
>>>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> > _______________________________________________
>>>>>> > Postgres-xc-developers mailing list
>>>>>> > Pos...@li...
>>>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>> >
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Managing the Performance of Cloud-Based Applications
>>>>>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>>>>>> Read the Whitepaper.
>>>>>>
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
>>>>>> _______________________________________________
>>>>>> Postgres-xc-developers mailing list
>>>>>> Pos...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Abbas
>>>>> Architect
>>>>>
>>>>> Ph: 92.334.5100153
>>>>> Skype ID: gabbasb
>>>>> www.enterprisedb.com
>>>>>
>>>>> Follow us on Twitter
>>>>> @EnterpriseDB
>>>>>
>>>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ahsan Hadi
>>>> Snr Director Product Development
>>>> EnterpriseDB Corporation
>>>> The Enterprise Postgres Company
>>>>
>>>> Phone: +92-51-8358874
>>>> Mobile: +92-333-5162114
>>>>
>>>> Website: www.enterprisedb.com
>>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>>
>>>> This e-mail message (and any attachment) is intended for the use of the
>>>> individual or entity to whom it is addressed. This message contains
>>>> information from EnterpriseDB Corporation that may be privileged,
>>>> confidential, or exempt from disclosure under applicable law. If you are not
>>>> the intended recipient or authorized to receive this for the intended
>>>> recipient, any use, dissemination, distribution, retention, archiving, or
>>>> copying of this communication is strictly prohibited. If you have received
>>>> this e-mail in error, please notify the sender immediately by reply e-mail
>>>> and delete this message.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Abbas
>>> Architect
>>>
>>> Ph: 92.334.5100153
>>> Skype ID: gabbasb
>>> www.enterprisedb.com
>>>
>>> Follow us on Twitter
>>> @EnterpriseDB
>>>
>>> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>>
>>
>>
>>
>> --
>> Ahsan Hadi
>> Snr Director Product Development
>> EnterpriseDB Corporation
>> The Enterprise Postgres Company
>>
>> Phone: +92-51-8358874
>> Mobile: +92-333-5162114
>>
>> Website: www.enterprisedb.com
>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>
>> This e-mail message (and any attachment) is intended for the use of the
>> individual or entity to whom it is addressed. This message contains
>> information from EnterpriseDB Corporation that may be privileged,
>> confidential, or exempt from disclosure under applicable law. If you are not
>> the intended recipient or authorized to receive this for the intended
>> recipient, any use, dissemination, distribution, retention, archiving, or
>> copying of this communication is strictly prohibited. If you have received
>> this e-mail in error, please notify the sender immediately by reply e-mail
>> and delete this message.
>
>
>
>
> --
> --
> Abbas
> Architect
>
> Ph: 92.334.5100153
> Skype ID: gabbasb
> www.enterprisedb.com
>
> Follow us on Twitter
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers and more
>
> ------------------------------------------------------------------------------
> Flow-based real-time traffic analytics software. Cisco certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

On Mon, Mar 10, 2014 at 6:32 AM, Michael Paquier
<mic...@gm...>wrote:

> On Mon, Mar 10, 2014 at 6:24 PM, Pavan Deolasee
> <pav...@gm...> wrote:
> > IMHO correctness takes precedence over performance, especially now that
> we
> > are moving from being an experimental product to being production ready
> +1.
>

+1



-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

On Mon, Mar 10, 2014 at 2:54 PM, Pavan Deolasee <pav...@gm...>wrote:

>
>
>
>
> On 10-Mar-2014, at 2:08 pm, Ashutosh Bapat <
> ash...@en...> wrote:
>
> Here is another reason why we shouldn't add requirement for primary key in
> replicated tables. Thanks Abbas for pointing it out.
>
> If the UPDATE/DELETE statement gets FQSed, we don't need a primary key for
> replicated table and if it doesn't, then we need a primary key. It will be
> difficult to explain users why some DMLs on replicated tables error out and
> why some of them don't.
>
>
> We should  error out irrespective of whether the query can be FQSed or not.
>
>
That's not what the patch is doing. Hence, my objection. Anyway, there is
no reason for an FQSable DML not to be FQSed since there is no data
corruption happening in that case.

> Again we come back to the backward compatibility.
>
>
> Again, it's a bug
>
>
My objection is for the hasty fixes. We should fix the bug in a clean way
without having any bad impacts.

> Before anyone suggests, disabling FQS for replicated tables, is not a
> solution here. It will affect performance badly.
>
>
> IMHO correctness takes precedence over performance, especially now that we
> are moving from being an experimental product to being production ready
>
>
Well, try doing that, and people wouldn't use Postgres-XC :)


> FWIW I am ok with adding a non documented guc for now
>
>
Thanks,
> Pavan
>
>
>
>>
>>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On Mon, Mar 10, 2014 at 6:24 PM, Pavan Deolasee
<pav...@gm...> wrote:
> IMHO correctness takes precedence over performance, especially now that we
> are moving from being an experimental product to being production ready
+1.

> FWIW I am ok with adding a non documented guc for now
But -1 for that. It is better to be transparent to the user, if you do
not document it even developers might forget about it and what it
does.
-- 
Michael

> On 10-Mar-2014, at 2:08 pm, Ashutosh Bapat <ash...@en...> wrote:
> 
> Here is another reason why we shouldn't add requirement for primary key in replicated tables. Thanks Abbas for pointing it out.
> 
> If the UPDATE/DELETE statement gets FQSed, we don't need a primary key for replicated table and if it doesn't, then we need a primary key. It will be difficult to explain users why some DMLs on replicated tables error out and why some of them don't.

We should  error out irrespective of whether the query can be FQSed or not.

> Again we come back to the backward compatibility.

Again, it's a bug

> Before anyone suggests, disabling FQS for replicated tables, is not a solution here. It will affect performance badly.

IMHO correctness takes precedence over performance, especially now that we are moving from being an experimental product to being production ready

FWIW I am ok with adding a non documented guc for now 

Thanks,
Pavan
>  
>> 

On Sat, Mar 8, 2014 at 7:52 PM, Abbas Butt <abb...@en...>wrote:

> Hi,
> Attached please find patch to add a GUC called require_replicated_table_pkey
> which is true by default.
> The purpose of the GUC is to error out for non-FQS UPDATE / DELETE to a
> replicated table.
>

Here is another reason why we shouldn't add requirement for primary key in
replicated tables. Thanks Abbas for pointing it out.

If the UPDATE/DELETE statement gets FQSed, we don't need a primary key for
replicated table and if it doesn't, then we need a primary key. It will be
difficult to explain users why some DMLs on replicated tables error out and
why some of them don't. Again we come back to the backward compatibility.
Before anyone suggests, disabling FQS for replicated tables, is not a
solution here. It will affect performance badly.


> By setting the GUC to false the error can be by passed and this is done to
> run regression tests.
> make check runs fine after the patch.
> Test cases are added to check the added functionality.
> Best Regards
>
>
>
> On Thu, Mar 6, 2014 at 9:00 AM, Pavan Deolasee <pav...@gm...>wrote:
>
>>
>>
>> On 06-Mar-2014, at 7:57 am, Abbas Butt <abb...@en...>
>> wrote:
>>
>>
>>
>>
>> On Wed, Mar 5, 2014 at 10:18 PM, Pavan Deolasee <pav...@gm...
>> > wrote:
>>
>>> On Wed, Mar 5, 2014 at 2:32 PM, Koichi Suzuki <koi...@gm...>wrote:
>>>
>>>> I'd like Abbas to answer why it is practical at this point ...
>>>>
>>>>
>>> Sure. I would wait for him to explain.
>>>
>>
>> It was done as a quick fix for backward compatibility,
>>
>>
>> IMHO it's a bug that needs to be fixed and not a compatibility that needs
>> to be backward supported. Let's just throw an error for now until we have
>> time and resources to fix the bug.
>>
>> Thanks,
>> Pavan
>>
>>
>
>
> --
> --
> *Abbas*
>  Architect
>
> Ph: 92.334.5100153
>  Skype ID: gabbasb
> www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
>
> *Follow us on Twitter*
> @EnterpriseDB
>
> Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community>
>
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to
> Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and
> the
> freedom to use Git, Perforce or both. Make the move to Perforce.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Hi,
Attached please find patch to add a GUC called require_replicated_table_pkey
which is true by default.
The purpose of the GUC is to error out for non-FQS UPDATE / DELETE to a
replicated table.
By setting the GUC to false the error can be by passed and this is done to
run regression tests.
make check runs fine after the patch.
Test cases are added to check the added functionality.
Best Regards



On Thu, Mar 6, 2014 at 9:00 AM, Pavan Deolasee <pav...@gm...>wrote:

>
>
> On 06-Mar-2014, at 7:57 am, Abbas Butt <abb...@en...>
> wrote:
>
>
>
>
> On Wed, Mar 5, 2014 at 10:18 PM, Pavan Deolasee <pav...@gm...>wrote:
>
>> On Wed, Mar 5, 2014 at 2:32 PM, Koichi Suzuki <koi...@gm...>wrote:
>>
>>> I'd like Abbas to answer why it is practical at this point ...
>>>
>>>
>> Sure. I would wait for him to explain.
>>
>
> It was done as a quick fix for backward compatibility,
>
>
> IMHO it's a bug that needs to be fixed and not a compatibility that needs
> to be backward supported. Let's just throw an error for now until we have
> time and resources to fix the bug.
>
> Thanks,
> Pavan
>
>


-- 
-- 
*Abbas*
 Architect

Ph: 92.334.5100153
 Skype ID: gabbasb
www.enterprisedb.co
<http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>

*Follow us on Twitter*
@EnterpriseDB

Visit EnterpriseDB for tutorials, webinars,
whitepapers<http://www.enterprisedb.com/resources-community>and
more<http://www.enterprisedb.com/resources-community>

XC did not have specific restrictions to this.    I've tested
btree-gin and btree-gist.   They look working.

Please let us know if you have any problem.

Regards;
---
Koichi Suzuki


2014-03-07 20:42 GMT+00:00 Chi-Young Ku <ch...@qu...>:
> Hi,
>
>     I am new to PostGres-XC.   I am wondering if XC supports
> user defined index type.
>
>     Thanks in advance.
>
> Chi
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to
> Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and
> the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Hi,

    I am new to PostGres-XC.   I am wondering if XC supports
user defined index type. 


    Thanks in advance.

Chi

Suzuki-san,

Thank you for your reply.

I've checked pgbench in REL1_0_STABLE.
At a glance the lines from REL1_0_STABLE as follow,
the part that I fixed for REL1_1_STABLE seems
not to include the problem I fixed before.

pgbench.c L.256-274.
----
#ifdef PGXC
static char *tpc_b_bid = {
	"\\set nbranches :scale\n"
	"\\set ntellers 10 * :scale\n"
	"\\set naccounts 100000 * :scale\n"
	"\\setrandom aid 1 :naccounts\n"
	"\\setrandom bid 1 :nbranches\n"
	"\\setrandom tid 1 :ntellers\n"
	"\\setrandom delta -5000 5000\n"
	"BEGIN;\n"
	"UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid =
:aid;\n"
	"SELECT abalance FROM pgbench_accounts WHERE aid = :aid\n"
	"UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid =
:tid;\n"
	"UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid =
:bid;\n"
	"INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES
(:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);\n"
	"END;\n"
};
#endif
---

This is almost same as that of the fixed version
of pgbench for REL1_1_STABLE.

Best Regards,

Tetsuo Sakata.

(2014/03/06 16:14), Koichi Suzuki wrote:
> Thank you Sakata-san;
> 
> Yes, the patch was successful with REL1_1_STABLE.    With some offset,
> it was also successful with REL1_2_STABLE and master.
> But failed with REL1_0_STABLE branch.
> 
> Because of the nature of the fix, I think this should also go to 1.0.x.
> 
> Could you further look into it?   Also hope you test this fix with
> REL1_0_STABLE as well.
> 
> Best;
> ---
> Koichi Suzuki
> 
> 
> 2014-03-06 15:28 GMT+09:00 坂田　哲夫 <sak...@la...>:
>> Suzuki-san,
>>
>> Sorry for the delay, and thank you for your comments.
>>
>> I've made the patch I sent before for the XC 1.1.0 not REL_1_1_STABLE.
>> So I recreated a patch for REL_1_1_STABLE.
>>
>> Please check the attached patch.
>>
>> Thanks in advance,
>>
>> Tetsuo Sakata.
>>
>>
>>
>>
>> (2014/03/05 17:36), Koichi Suzuki wrote:
>>> Sakata-san;
>>>
>>> I've tested the patch and found it does not apply to current
>>> master/REL1_2_STABLE branch.   I also tested against REL1_1_STABLE and
>>> the result was the same.   Could you check your patch against the
>>> current repo?
>>>
>>> I wonder why it happens because pgbench is very stable.    Maybe it
>>> needs some tweak to deal with the current PG repo as well.
>>>
>>> Could you also consider to use the format you get with "git diff"?
>>>
>>> Thank you very much in advance.
>>> ---
>>> Koichi Suzuki
>>>
>>>
>>> 2014-03-05 15:23 GMT+09:00 坂田　哲夫 <sak...@la...>:
>>>> Hi, folks,
>>>>
>>>> I found a pgbench bug that pgbench fails to update tables such as
>>>> pgbench_accounts and pgbench_tellers.
>>>>
>>>> For example, original pgbench for XC tries to update pgbench_accounts'
>>>> specifying aid and bid values which are selected independently.
>>>> By the pgbench's nature, bid should have a particular value for a
>>>> given aid value. So the update statement fails picking up target
>>>> tuple and updating the table in most cases.
>>>>
>>>> The attached patch, which should be applied to XC 1.1s
>>>> pgbench, eliminates wrongly added bid specifications
>>>> and corrects index and distribute settings related.
>>>>
>>>> Best regards,
>>>>
>>>> Tetsuo Sakata.
>>>>
>>>> --
>>>> sakata.tetsuo _at_ lab.ntt.co.jp
>>>> SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>>>> With Perforce, you get hassle-free workflows. Merge that actually works.
>>>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>>>> freedom to use Git, Perforce or both. Make the move to Perforce.
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>>> With Perforce, you get hassle-free workflows. Merge that actually works.
>>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>>> freedom to use Git, Perforce or both. Make the move to Perforce.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>>
>> --
>> sakata.tetsuo _at_ lab.ntt.co.jp
>> SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.
>>
>> ------------------------------------------------------------------------------
>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>> With Perforce, you get hassle-free workflows. Merge that actually works.
>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>> freedom to use Git, Perforce or both. Make the move to Perforce.
>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
> 
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> 

-- 
sakata.tetsuo _at_ lab.ntt.co.jp ☆メアド変わりました oss→lab☆
SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.

I see.   Then maybe we should make this time as GUC and customizable for the environment?

Regards;
---
Koichi Suzuki

2014/03/06 16:47、Masataka Saito <pg...@gm...> のメール：

> I ALSO revised the daily building scripts today.
> 
> Past scripts tested the regression on a virtual disk which is created
> on a file in the host filesystem. I think it is slower than USB disk
> which is accessed as block device bypassing filesystem in the host OS.
> 
> Regards.
> 
> On 6 March 2014 15:48, 鈴木 幸市 <ko...@in...> wrote:
>> The fix has already been checked-in.
>> ---
>> Koichi Suzuki
>> 
>> 2014/03/06 15:36、Masataka Saito <pg...@gm...> のメール：
>> 
>>> I revised buildfarm scripts to compile and test on USB disk which is
>>> accessed by the buildfarm vm via virtio as block device bypassing
>>> filesystem in the host OS. I tested my patch on the same condition at
>>> the buildfarm vm and I've never seen the random failure.
>>> 
>>> I hope the fix solves the problem.
>>> 
>>> Regards.
>>> 
>>> ------------------------------------------------------------------------------
>>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>>> With Perforce, you get hassle-free workflows. Merge that actually works..
>>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>>> freedom to use Git, Perforce or both. Make the move to Perforce.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> 
>> 
> 

I ALSO revised the daily building scripts today.

Past scripts tested the regression on a virtual disk which is created
on a file in the host filesystem. I think it is slower than USB disk
which is accessed as block device bypassing filesystem in the host OS.

Regards.

On 6 March 2014 15:48, 鈴木 幸市 <ko...@in...> wrote:
> The fix has already been checked-in.
> ---
> Koichi Suzuki
>
> 2014/03/06 15:36、Masataka Saito <pg...@gm...> のメール：
>
>> I revised buildfarm scripts to compile and test on USB disk which is
>> accessed by the buildfarm vm via virtio as block device bypassing
>> filesystem in the host OS. I tested my patch on the same condition at
>> the buildfarm vm and I've never seen the random failure.
>>
>> I hope the fix solves the problem.
>>
>> Regards.
>>
>> ------------------------------------------------------------------------------
>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>> With Perforce, you get hassle-free workflows. Merge that actually works.
>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>> freedom to use Git, Perforce or both. Make the move to Perforce.
>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>

Thank you Sakata-san;

Yes, the patch was successful with REL1_1_STABLE.    With some offset,
it was also successful with REL1_2_STABLE and master.
But failed with REL1_0_STABLE branch.

Because of the nature of the fix, I think this should also go to 1.0.x.

Could you further look into it?   Also hope you test this fix with
REL1_0_STABLE as well.

Best;
---
Koichi Suzuki


2014-03-06 15:28 GMT+09:00 坂田　哲夫 <sak...@la...>:
> Suzuki-san,
>
> Sorry for the delay, and thank you for your comments.
>
> I've made the patch I sent before for the XC 1.1.0 not REL_1_1_STABLE.
> So I recreated a patch for REL_1_1_STABLE.
>
> Please check the attached patch.
>
> Thanks in advance,
>
> Tetsuo Sakata.
>
>
>
>
> (2014/03/05 17:36), Koichi Suzuki wrote:
>> Sakata-san;
>>
>> I've tested the patch and found it does not apply to current
>> master/REL1_2_STABLE branch.   I also tested against REL1_1_STABLE and
>> the result was the same.   Could you check your patch against the
>> current repo?
>>
>> I wonder why it happens because pgbench is very stable.    Maybe it
>> needs some tweak to deal with the current PG repo as well.
>>
>> Could you also consider to use the format you get with "git diff"?
>>
>> Thank you very much in advance.
>> ---
>> Koichi Suzuki
>>
>>
>> 2014-03-05 15:23 GMT+09:00 坂田　哲夫 <sak...@la...>:
>>> Hi, folks,
>>>
>>> I found a pgbench bug that pgbench fails to update tables such as
>>> pgbench_accounts and pgbench_tellers.
>>>
>>> For example, original pgbench for XC tries to update pgbench_accounts'
>>> specifying aid and bid values which are selected independently.
>>> By the pgbench's nature, bid should have a particular value for a
>>> given aid value. So the update statement fails picking up target
>>> tuple and updating the table in most cases.
>>>
>>> The attached patch, which should be applied to XC 1.1s
>>> pgbench, eliminates wrongly added bid specifications
>>> and corrects index and distribute settings related.
>>>
>>> Best regards,
>>>
>>> Tetsuo Sakata.
>>>
>>> --
>>> sakata.tetsuo _at_ lab.ntt.co.jp
>>> SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.
>>>
>>> ------------------------------------------------------------------------------
>>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>>> With Perforce, you get hassle-free workflows. Merge that actually works.
>>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>>> freedom to use Git, Perforce or both. Make the move to Perforce.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>> ------------------------------------------------------------------------------
>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>> With Perforce, you get hassle-free workflows. Merge that actually works.
>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>> freedom to use Git, Perforce or both. Make the move to Perforce.
>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>
> --
> sakata.tetsuo _at_ lab.ntt.co.jp
> SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

The fix has already been checked-in.
---
Koichi Suzuki

2014/03/06 15:36、Masataka Saito <pg...@gm...> のメール：

> I revised buildfarm scripts to compile and test on USB disk which is
> accessed by the buildfarm vm via virtio as block device bypassing
> filesystem in the host OS. I tested my patch on the same condition at
> the buildfarm vm and I've never seen the random failure.
> 
> I hope the fix solves the problem.
> 
> Regards.
> 
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works. 
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> 

I revised buildfarm scripts to compile and test on USB disk which is
accessed by the buildfarm vm via virtio as block device bypassing
filesystem in the host OS. I tested my patch on the same condition at
the buildfarm vm and I've never seen the random failure.

I hope the fix solves the problem.

Regards.

Suzuki-san,

Sorry for the delay, and thank you for your comments.

I've made the patch I sent before for the XC 1.1.0 not REL_1_1_STABLE.
So I recreated a patch for REL_1_1_STABLE.

Please check the attached patch.

Thanks in advance,

Tetsuo Sakata.




(2014/03/05 17:36), Koichi Suzuki wrote:
> Sakata-san;
> 
> I've tested the patch and found it does not apply to current
> master/REL1_2_STABLE branch.   I also tested against REL1_1_STABLE and
> the result was the same.   Could you check your patch against the
> current repo?
> 
> I wonder why it happens because pgbench is very stable.    Maybe it
> needs some tweak to deal with the current PG repo as well.
> 
> Could you also consider to use the format you get with "git diff"?
> 
> Thank you very much in advance.
> ---
> Koichi Suzuki
> 
> 
> 2014-03-05 15:23 GMT+09:00 坂田　哲夫 <sak...@la...>:
>> Hi, folks,
>>
>> I found a pgbench bug that pgbench fails to update tables such as
>> pgbench_accounts and pgbench_tellers.
>>
>> For example, original pgbench for XC tries to update pgbench_accounts'
>> specifying aid and bid values which are selected independently.
>> By the pgbench's nature, bid should have a particular value for a
>> given aid value. So the update statement fails picking up target
>> tuple and updating the table in most cases.
>>
>> The attached patch, which should be applied to XC 1.1s
>> pgbench, eliminates wrongly added bid specifications
>> and corrects index and distribute settings related.
>>
>> Best regards,
>>
>> Tetsuo Sakata.
>>
>> --
>> sakata.tetsuo _at_ lab.ntt.co.jp
>> SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.
>>
>> ------------------------------------------------------------------------------
>> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
>> With Perforce, you get hassle-free workflows. Merge that actually works.
>> Faster operations. Version large binaries.  Built-in WAN optimization and the
>> freedom to use Git, Perforce or both. Make the move to Perforce.
>> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
> 
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> 


-- 
sakata.tetsuo _at_ lab.ntt.co.jp
SAKATA, Tetsuo. Shinagawa Tokyo JAPAN.