postgres-xc-developers Mailing List for Postgres-XC (Page 2)

On Thu, Sep 6, 2012 at 8:41 PM, Michael Paquier
<mic...@gm...>wrote:

>
>
> On Thu, Sep 6, 2012 at 6:34 PM, Nikhil Sontakke <ni...@st...>wrote:
>
>> > 1) Get the current value of sequence by using currval inside pg_dump
>> and not
>> > the cached info on each local node. This way you just need to modify
>> pg_dump
>>
>> The currval function is session specific and currently it errors out
>> if it's invoked before calling a nextval in the same session. An
>> undocumented variation of currval which does not error out might help.
>> But then we will need to do an initdb.
>>
>> We could expose GTM ip, port to pg_dump and have it connect to the GTM
>> for this info, but that seems uglier.
>>
> Also, an argument against that: we cannot expose GTM to client
applications, including pg_dump.
This would impact security inside cluster.
-- 
Michael Paquier
http://michael.otacoo.com

On Thu, Sep 6, 2012 at 6:34 PM, Nikhil Sontakke <ni...@st...> wrote:

> > 1) Get the current value of sequence by using currval inside pg_dump and
> not
> > the cached info on each local node. This way you just need to modify
> pg_dump
>
> The currval function is session specific and currently it errors out
> if it's invoked before calling a nextval in the same session. An
> undocumented variation of currval which does not error out might help.
> But then we will need to do an initdb.
>
> We could expose GTM ip, port to pg_dump and have it connect to the GTM
> for this info, but that seems uglier.
>
I still vote for the pg_dump solution. And also update the documentation
that the current value of sequence cached on each Coordinator may vary.
-- 
Michael Paquier
http://michael.otacoo.com

> 1) Get the current value of sequence by using currval inside pg_dump and not
> the cached info on each local node. This way you just need to modify pg_dump

The currval function is session specific and currently it errors out
if it's invoked before calling a nextval in the same session. An
undocumented variation of currval which does not error out might help.
But then we will need to do an initdb.

We could expose GTM ip, port to pg_dump and have it connect to the GTM
for this info, but that seems uglier.

Regards,
Nikhils
-- 
StormDB - http://www.stormdb.com
The Database Cloud
Postgres-XC Support and Service

I've got that TRIGGER will be developed in three steps.     I tried to locate where AFTER trigger issue is from the patch but could not find.   I'm afraid I checked the patch too quickly and made misread.   I apologize for the mistake.
---
Koichi

On Tue, 4 Sep 2012 10:39:54 +0900
Michael Paquier <mic...@gm...> wrote:

> On Tue, Sep 4, 2012 at 9:44 AM, Koichi Suzuki <ko...@in...>wrote:
> 
> > I'm wondering if it is sufficient for the next major release not to
> > support "FOR EACH ROW" trigger, which is the default of PostgreSQL.   I
> > think FOR EACH ROW is what application considers to use at first.
> >  Example of CREATE TRIGGER in PG reference deals with FOR EACH ROW.
> >
> I think you didn't get me... This is the *first* and not *last* patch of
> the trigger series.
> My strategy is to support triggers step-by-step. The structure of
> PostgreSQL is made such as constraints and triggers are using the same
> things, and we need more basic infrastructure on that before really getting
> into the row-trigger support.
> 
> So, I already sent emails about that but here is once again what I am
> talking abou :)
> 1) Support for statement triggers: addition of trigger shippability
> algorithm, XC test cases and allow all types of statement triggers (done)
> 2) Extend support of constraints and triggers to allow locally enforced
> constraints to be treated as shippable triggers (working on)
> 3) Provide a mechanism to fetch and update tuples on remote nodes inside
> DML executor: this will allow support for row-based triggers. The method
> used is not yet completely decided: I designed an idea of my own, but
> Ashutosh suggested to use
> 
> 
> > Not supporting "AFTER" optin could also be a severe restriction.
> >  Applications may want to see "AFTER" row image for different actions.
> >
> Once again I never said that there will be restrictions. But I just think
> that providing all the support in 1 patch might be too heavy considering
> that it is possible to do that in an understandable step-by-step way.
> -- 
> Michael Paquier
> http://michael.otacoo.com

On Tue, Sep 4, 2012 at 9:44 AM, Koichi Suzuki <ko...@in...>wrote:

> I'm wondering if it is sufficient for the next major release not to
> support "FOR EACH ROW" trigger, which is the default of PostgreSQL.   I
> think FOR EACH ROW is what application considers to use at first.
>  Example of CREATE TRIGGER in PG reference deals with FOR EACH ROW.
>
I think you didn't get me... This is the *first* and not *last* patch of
the trigger series.
My strategy is to support triggers step-by-step. The structure of
PostgreSQL is made such as constraints and triggers are using the same
things, and we need more basic infrastructure on that before really getting
into the row-trigger support.

So, I already sent emails about that but here is once again what I am
talking abou :)
1) Support for statement triggers: addition of trigger shippability
algorithm, XC test cases and allow all types of statement triggers (done)
2) Extend support of constraints and triggers to allow locally enforced
constraints to be treated as shippable triggers (working on)
3) Provide a mechanism to fetch and update tuples on remote nodes inside
DML executor: this will allow support for row-based triggers. The method
used is not yet completely decided: I designed an idea of my own, but
Ashutosh suggested to use


> Not supporting "AFTER" optin could also be a severe restriction.
>  Applications may want to see "AFTER" row image for different actions.
>
Once again I never said that there will be restrictions. But I just think
that providing all the support in 1 patch might be too heavy considering
that it is possible to do that in an understandable step-by-step way.
-- 
Michael Paquier
http://michael.otacoo.com

I'm wondering if it is sufficient for the next major release not to support "FOR EACH ROW" trigger, which is the default of PostgreSQL.   I think FOR EACH ROW is what application considers to use at first.    Example of CREATE TRIGGER in PG reference deals with FOR EACH ROW.

Not supporting "AFTER" optin could also be a severe restriction.    Applications may want to see "AFTER" row image for different actions.

Further input on this?
---
Koichi

On Mon, 3 Sep 2012 18:04:26 +0900
Michael Paquier <mic...@gm...> wrote:

> Hi all,
> 
> Please find attached a patch that support triggers with FOR EACH STATEMENT.
> This patch contains all the infrastructure to evaluate trigger shippability
> and control trigger firing on local or remote node at execution.
> All the statement -based triggers are working here: INSERT, DELETE, UPDATE,
> TRUNCATE.
> Some tests have also been included to test this feature.
> 
> Vanilla PostgreSQL uses the infrastructure of triggers to define
> constraints like DEFERED, foreign keys, etc.
> And, in Postgres-XC we allow firing such constraints , which are row
> triggers, on remote nodes if this constraint can be enforced locally. For
> example, take the case of a replicated table with INSERT, we can safely do
> a constraint check on this table on Datanodes.
> The current XC mechanism is controlled by 2 things:
> - checkLocalFKConstraints of parse_utilcmd.c which checks if a constraint
> can be safely evaluated on remote nodes. If it is not the case, the
> constraint creation is blocked.
> - completely block the reference integrity check of triggers on
> Coordinators in RI_FKey_check of ri_triggers.c
> Those mechanisms have a good complementarity, but we need to move forward
> to something more complex if we want a real row-based implementation for
> triggers.
> 
> When implementing row-based triggers, what we need to do is:
> - Remove the condition in ri_triggers.c blocking reference integrity check
> on Coordinators. If the trigger is not shippable to remote nodes, we need
> to evaluate it on Coordinators.
> - Instead of forbiding the creation of foreign constraints that need to be
> evaluated on local Coordinator, addition of a flag in Constraint that will
> be calculated by a function replacing checkLocalFKConstraints. This flag
> will be pushed down to the row-based trigger at trigger creation and
> control if the trigger can be launched safely on remote nodes or on local
> Coordinator. A trigger cannot be launched to both remote nodes and local
> Coordinator.
> 
> Such a mechanism is needed on top of the basic mechanism to fetch/update
> tuples from/to remote nodes inside the DML executor when firing a trigger
> on Coordinator.
> Btw, the current patch version is based on the basic infrastructure, blocks
> the creation of row-base triggers when defined by CREATE TRIGGER, and
> insures the launch of other triggers that are defined by CREATE TABLE.
> 
> I will try first to provide a patch that brings in a better control of
> row-based trigger firing for foreign constraints on Coordinator and
> Datanodes.
> This patch will block the firing of triggers on Coordinator when given and
> this condition will be re-enabled once the fetching mechanism inside
> executor is done.
> 
> I am still having 1 test case which is failing for TRUNCATE AFTER trigger:
> CREATE TABLE trunc_trigger_test (f1 int, f2 text, f3 text); -- not working
> CREATE TABLE trunc_trigger_log (tgop text, tglevel text, tgwhen text,
>         tgargv text, tgtable name, rowcount bigint);
> CREATE FUNCTION trunctrigger() RETURNS trigger as $$
> declare c bigint;
> begin
>     execute 'select count(*) from ' || quote_ident(tg_table_name) into c;
>     insert into trunc_trigger_log values
>       (TG_OP, TG_LEVEL, TG_WHEN, TG_ARGV[0], tg_table_name, c);
>     return null;
> end;
> $$ LANGUAGE plpgsql;
> -- same test with an after trigger
> INSERT INTO trunc_trigger_test VALUES(1, 'foo', 'bar'), (2, 'baz', 'quux');
> CREATE TRIGGER tt
> AFTER TRUNCATE ON trunc_trigger_test
> FOR EACH STATEMENT
> EXECUTE PROCEDURE trunctrigger('after trigger truncate');
> SELECT count(*) as "Row count in test table" FROM trunc_trigger_test;
> SELECT * FROM trunc_trigger_log;
> TRUNCATE trunc_trigger_test;
> SELECT count(*) as "Row count in test table" FROM trunc_trigger_test;
> SELECT * FROM trunc_trigger_log; -- Num of row inserted should be 0, this
> happens AFTER the TRUNCATE
> 
> The problem here is that the last query gives this result:
> SELECT * FROM trunc_trigger_log;
>    tgop   |  tglevel  | tgwhen |         tgargv         |
> tgtable       | rowcount
> ----------+-----------+--------+------------------------+--------------------+----------
>  TRUNCATE | STATEMENT | AFTER  | after trigger truncate |
> trunc_trigger_test |        2
> (1 row)
> instead of that:
> SELECT * FROM trunc_trigger_log;
>    tgop   |  tglevel  | tgwhen |         tgargv         |
> tgtable       | rowcount
> ----------+-----------+--------+------------------------+--------------------+----------
>  TRUNCATE | STATEMENT | AFTER  | after trigger truncate |
> trunc_trigger_test |        0
> (1 row)
> So the row count obtained is not correct, I believe this is a problem with
> command ID. So I tried to enforce sending the command ID from Coordinator
> to Datanodes but couldn't get any results.
> I also noticed that when  trunc_trigger_test is a replicated table, the
> procedure query gets pushed down and there are no problems.
> Abbas, do you have some ideas? I recall that you worked on command IDs.
> 
> Thanks,
> 
> On Thu, Aug 30, 2012 at 6:29 PM, Koichi Suzuki <ko...@in...>wrote:
> 
> > On Thu, 30 Aug 2012 17:18:48 +0900
> > Michael Paquier <mic...@gm...> wrote:
> >
> > > On Thu, Aug 30, 2012 at 5:11 PM, Koichi Suzuki <ko...@in...
> > >wrote:
> > >
> > > > Hi, Michael;
> > > >
> > > > Thanks for the detailed research and consideration.   I think they
> > mostly
> > > > work fine.   I have a couple of questions.
> > > > 1) What happens if trigger is not shippable and the target is
> > replicated
> > > > table?  (I'm not sure if ther's such case
> > > > actually though).
> > > >
> > > For row-based triggers?
> >
> > Yes.
> >
> > >
> > > 2) In the case of replicated tables, trigger functions may alter the NEW
> > > > tuples.   Are there any means to enforce
> > > > the integrity?
> > > >
> > > Do you mean ship the check on constraints to remote nodes?
> >
> > I don't insist specific means.  If it works efficiently, it's okay.
> >
> > >
> > > > Or should trigger functions be responsible to maintain it?  (I think
> > both
> > > > make sense.)
> > > >
> > > Do you mean that constraints need to be checked on Cordinator? controlled
> > > by Coordinator?
> >
> > Whatever means to use, I think someone should enforce the integrity, not
> > necessarily the core.
> >
> > Replicated table needs specific (and careful) handling to maintain all the
> > replica consists identical rows.   I understand it's not always easy to do
> > in the core.    I'm wondering if we can enforce it when trigger functions
> > alter the NEW row using volatile functions.    I also think it's acceptable
> > to ask trigger functions to be responsible to maintain it.
> >
> > I hope they are in your scope.
> >
> > Regards;
> > > --
> > > Michael Paquier
> > > http://michael.otacoo.com
> >
> 
> 
> 
> -- 
> Michael Paquier
> http://michael.otacoo.com