postgres-xc-developers Mailing List for Postgres-XC

On Sat, Jul 14, 2012 at 1:01 AM, Nikhil Sontakke <ni...@st...>wrote:

> > But, one thing that is essential is catalog updates. Are you suggesting
> that
> > the catalog updates too should be done using some SQL?
>
> Surely, we can do:
>
> BEGIN;
>
> CREATE NODE; /* Hmm, I think we should have a CREATE CLUSTER NODE version*/
>
> ALL the redistribution SQL here;
>
> COMMIT;
>
> The new node info will be visible to the rest of the SQL commands that
> follow this ways.
>
> Michael, I don't have a strong view against adding stuff in the core,
> but since the first cut for redistribution seems to be a bunch of SQL
> grouped together, I thought this might be worth investigating too.
>
Honestly I thought about such solutions before beginning my stuff, but
arrived at the conclusion that we need a native way to manage
redistribution. In this case, you fall under the following limitations:
- How to manage node-level granularity for redistribution operations? In
XC, by design, a SQL is launched globally on the cluster. At the exception
of EXECUTE DIRECT but it only allows SELECT commands.
- When and how to manage the catalog update? Even if it is possible to
update catalog with DMLs, we need native APIs to be able to modify catalog
entries.
- CREATE NODE is used for the addition and deletion of nodes. Data
redistribution does not concern changing the configuration of the cluster.
You got a certain number of Datanodes, Coordinators, and you want to change
the nodes where data of a table is located inside this given cluster. Your
approach makes redistribution dependent on cluster configuration and it is
honestly not welcome to add such degrees of dependencies that may be a
burden in the future if we change once again the way cluster is configured.
- For certain redistribution operations, we do NOT always need a
transaction block! Just take the example of a replicated table changed to a
distributed table. You just need to send a DELETE query to remote nodes to
remove only the tuples that do not satisfy a hash condition. This is one of
those things I am working on now.

To my mind, each argument here makes necessary this feature in core. All
combined even strengthen my arguments.


> Regards,
> Nikhils
>
> > Something I am afraid is not possible with an external utility is
> control of
> > redistribution at node level. For example, an external contrib module or
> > utility will launch SQL queries to xc that have to be treated as global.
> > However, redistribution needs to take care of cases like for example the
> > reduction of nodes for replicated tables. In this case you just need to
> > delete the data from removed nodes. Another easy example is the case of
> an
> > increase of nodes for replicated tables. You need to pick up data on
> > coordinator and then send it only to the new nodes. Those simple examples
> > need a core management to minimize the work of redistribution inside
> > cluster.
> >
> > On 2012/07/13, at 15:09, Ashutosh Bapat <ash...@en...
> >
> > wrote:
> >
> > Even, I am wondering if that would be better.
> >
> > But, one thing that is essential is catalog updates. Are you suggesting
> that
> > the catalog updates too should be done using some SQL?
> >
> > Can you please expand more on your idea, may be providing some examples,
> > pseudo-code etc.?
> >
> > On Fri, Jul 13, 2012 at 10:36 AM, Nikhil Sontakke <ni...@st...>
> > wrote:
> >>
> >> Just a thought.
> >>
> >> If we have a utility which spews out all of these statements to
> >> redistribute a table across node modifications, then we can just wrap
> >> them inside a transaction block and just run that?
> >>
> >> Wont it save all of the core changes?
> >>
> >> Regards,
> >> Nikhils
> >>
> >> On Fri, Jul 13, 2012 at 12:29 AM, Michael Paquier
> >> <mic...@gm...> wrote:
> >> > Hi all,
> >> >
> >> > Please find attached an updated patch adding redistribution
> >> > optimizations
> >> > for replicated tables.
> >> > If the node subset of a replicated table is reduced, the necessary
> nodes
> >> > are
> >> > simply truncated.
> >> > If it is increased, a COPY TO is done to fetch the data, and COPY FROM
> >> > is
> >> > done only on the necessary nodes.
> >> > New regression tests have been added to test that.
> >> >
> >> > Regards,
> >> >
> >> >
> >> > On Thu, Jul 12, 2012 at 5:30 PM, Michael Paquier
> >> > <mic...@gm...>
> >> > wrote:
> >> >>
> >> >> OK, here is the mammoth patch: 3000 lines including docs,
> >> >> implementation
> >> >> and  regressions.
> >> >> The code has been realigned with current master.
> >> >> This patch introduces the latest thing I am working on: the
> >> >> redistribution
> >> >> command tree planning and execution.
> >> >>
> >> >> As I explained before, a redistribution consists of a series of
> >> >> commands
> >> >> (TRUNCATE, REINDEX, DELETE, COPY FROM, COPY TO) that need to be
> >> >> determined
> >> >> depending on the new and old locator information of the relation.
> Each
> >> >> action can be done on a subset of nodes.
> >> >> This patch introduces the basic infrastructure of the command tree
> >> >> build
> >> >> and execution.
> >> >> For the time being, redistribution uses only what is called the
> default
> >> >> command tree consisting of:
> >> >> 1) COPY TO
> >> >> 2) TRUNCATE
> >> >> 3) COPY FROM
> >> >> 4) REINDEX
> >> >> But this structure can be easily completed with more complicated
> >> >> operations.
> >> >> In this patch there is still a small thing missing which is the
> >> >> possibility to launch a COPY FROM on a subset of nodes, particularly
> >> >> useful
> >> >> when redistribution consists of a replicated table whose set of nodes
> >> >> is
> >> >> increased.
> >> >> Compared to the last versions, the impact of redistribution in
> >> >> tablecmds.c
> >> >> is limited.
> >> >>
> >> >> Regards,
> >> >>
> >> >> --
> >> >> Michael Paquier
> >> >> http://michael.otacoo.com
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Michael Paquier
> >> > http://michael.otacoo.com
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Live Security Virtual Conference
> >> > Exclusive live event will cover all the ways today's security and
> >> > threat landscape has changed and how IT managers can respond.
> >> > Discussions
> >> > will include endpoint security, mobile security and the latest in
> >> > malware
> >> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >> > _______________________________________________
> >> > Postgres-xc-developers mailing list
> >> > Pos...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >> >
> >>
> >>
> >>
> >> --
> >> StormDB - http://www.stormdb.com
> >> The Database Cloud
> >
> >
> >
> >
> > --
> > Best Wishes,
> > Ashutosh Bapat
> > EntepriseDB Corporation
> > The Enterprise Postgres Company
> >
>
>
>
> --
> StormDB - http://www.stormdb.com
> The Database Cloud
>



-- 
Michael Paquier
http://michael.otacoo.com

> But, one thing that is essential is catalog updates. Are you suggesting that
> the catalog updates too should be done using some SQL?

Surely, we can do:

BEGIN;

CREATE NODE; /* Hmm, I think we should have a CREATE CLUSTER NODE version*/

ALL the redistribution SQL here;

COMMIT;

The new node info will be visible to the rest of the SQL commands that
follow this ways.

Michael, I don't have a strong view against adding stuff in the core,
but since the first cut for redistribution seems to be a bunch of SQL
grouped together, I thought this might be worth investigating too.

Regards,
Nikhils

> Something I am afraid is not possible with an external utility is control of
> redistribution at node level. For example, an external contrib module or
> utility will launch SQL queries to xc that have to be treated as global.
> However, redistribution needs to take care of cases like for example the
> reduction of nodes for replicated tables. In this case you just need to
> delete the data from removed nodes. Another easy example is the case of an
> increase of nodes for replicated tables. You need to pick up data on
> coordinator and then send it only to the new nodes. Those simple examples
> need a core management to minimize the work of redistribution inside
> cluster.
>
> On 2012/07/13, at 15:09, Ashutosh Bapat <ash...@en...>
> wrote:
>
> Even, I am wondering if that would be better.
>
> But, one thing that is essential is catalog updates. Are you suggesting that
> the catalog updates too should be done using some SQL?
>
> Can you please expand more on your idea, may be providing some examples,
> pseudo-code etc.?
>
> On Fri, Jul 13, 2012 at 10:36 AM, Nikhil Sontakke <ni...@st...>
> wrote:
>>
>> Just a thought.
>>
>> If we have a utility which spews out all of these statements to
>> redistribute a table across node modifications, then we can just wrap
>> them inside a transaction block and just run that?
>>
>> Wont it save all of the core changes?
>>
>> Regards,
>> Nikhils
>>
>> On Fri, Jul 13, 2012 at 12:29 AM, Michael Paquier
>> <mic...@gm...> wrote:
>> > Hi all,
>> >
>> > Please find attached an updated patch adding redistribution
>> > optimizations
>> > for replicated tables.
>> > If the node subset of a replicated table is reduced, the necessary nodes
>> > are
>> > simply truncated.
>> > If it is increased, a COPY TO is done to fetch the data, and COPY FROM
>> > is
>> > done only on the necessary nodes.
>> > New regression tests have been added to test that.
>> >
>> > Regards,
>> >
>> >
>> > On Thu, Jul 12, 2012 at 5:30 PM, Michael Paquier
>> > <mic...@gm...>
>> > wrote:
>> >>
>> >> OK, here is the mammoth patch: 3000 lines including docs,
>> >> implementation
>> >> and  regressions.
>> >> The code has been realigned with current master.
>> >> This patch introduces the latest thing I am working on: the
>> >> redistribution
>> >> command tree planning and execution.
>> >>
>> >> As I explained before, a redistribution consists of a series of
>> >> commands
>> >> (TRUNCATE, REINDEX, DELETE, COPY FROM, COPY TO) that need to be
>> >> determined
>> >> depending on the new and old locator information of the relation. Each
>> >> action can be done on a subset of nodes.
>> >> This patch introduces the basic infrastructure of the command tree
>> >> build
>> >> and execution.
>> >> For the time being, redistribution uses only what is called the default
>> >> command tree consisting of:
>> >> 1) COPY TO
>> >> 2) TRUNCATE
>> >> 3) COPY FROM
>> >> 4) REINDEX
>> >> But this structure can be easily completed with more complicated
>> >> operations.
>> >> In this patch there is still a small thing missing which is the
>> >> possibility to launch a COPY FROM on a subset of nodes, particularly
>> >> useful
>> >> when redistribution consists of a replicated table whose set of nodes
>> >> is
>> >> increased.
>> >> Compared to the last versions, the impact of redistribution in
>> >> tablecmds.c
>> >> is limited.
>> >>
>> >> Regards,
>> >>
>> >> --
>> >> Michael Paquier
>> >> http://michael.otacoo.com
>> >
>> >
>> >
>> >
>> > --
>> > Michael Paquier
>> > http://michael.otacoo.com
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Live Security Virtual Conference
>> > Exclusive live event will cover all the ways today's security and
>> > threat landscape has changed and how IT managers can respond.
>> > Discussions
>> > will include endpoint security, mobile security and the latest in
>> > malware
>> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>>
>>
>>
>> --
>> StormDB - http://www.stormdb.com
>> The Database Cloud
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>



-- 
StormDB - http://www.stormdb.com
The Database Cloud

On Fri, Jul 13, 2012 at 12:50 AM, Michael Paquier <mic...@gm...
> wrote:

>
>
> On Fri, Jul 13, 2012 at 11:45 AM, nop <no...@in...> wrote:
>
>> Hi,
>>
>> Currently HEAD won't build because of a merge conflict. The attached
>> patch resolves that.
>>
>> Cheers,
>> Andrew
>>
> Thanks this has been fixed:
> http://github.com/postgres-xc/postgres-xc/commit/5655bc5
> I didn't reused your patch, as you did not take into account the options
> "-undefined suppress -flat_namespace" in LINK.shared that are used when
> compiling code on MacOS.
> Regards.
> --
> Michael Paquier
> http://michael.otacoo.com
>

Ah. OS X. Thanks for fixing it.

Ashutosh/Michael, thanks for your comments. More comments inline.

I have gone through each of the statements to see how we can tackle each
one, and have uploaded the details here:

https://docs.google.com/document/d/1KtPU8jrztHtGf_hzerCVacSoUPY36MK7aEpI6oqFQJc/edit

To see if we can apply the approach, I have tried out some ad-hoc
code-changes with sample statements namely create-tablespace, and
alter-database-set-tablespace, and basic things seem to work.


On 4 July 2012 09:51, Ashutosh Bapat <ash...@en...>wrote:

> Amit,
> If a complete automated solution is not possible, you can as well think
> about reporting an error and expecting human intervention.
>

Yes, in many cases, the final resort is to report a message showing which
particular node has failed. THis is after trying out a few things and then
giving up.


> On Tue, Jul 3, 2012 at 10:16 AM, Michael Paquier <
> mic...@gm...> wrote:
>
>>
>> On Fri, Jun 29, 2012 at 8:34 PM, Amit Khandekar <
>> ami...@en...> wrote:
>>
>>> For utility statements in general, the coordinator propagates SQL
>>> statements to all the required nodes, and most of these statements get run
>>> on the datanodes inside a transaction block. So, when the statement fails
>>> on at least one of the nodes, the statement gets rollbacked on all the
>>> nodes due to the two-phase commit taking place, and therefore the cluster
>>> rollbacks to a consistent state.  But there are some statements which
>>> cannot be run inside a transaction block. Here are some important ones:
>>> CREATE/DROP DATABASE
>>> CREATE/DROP TABLESPACE
>>> ALTER DATABASE SET TABLESPACE
>>> ALTER TYPE ADD ... (for enum types)
>>> CREATE INDEX CONCURRENTLY
>>> REINDEX DATABASE
>>> DISCARD ALL
>>>
>>> So such statements run on datanodes in auto-commit mode, and so create
>>> problems if they succeed on some nodes and abort on other nodes.  For e.g.
>>> : CREATE DATABASE. If a datanode d1 returns with error, and any other
>>> datanode d2 has already returned back to coordinator with success, the
>>> coordinator can't undo the commit of d2 because this is already committed.
>>> Or if the coordinator itself crashes after datanodes commit but before the
>>> coordinator commits, then again we have the same problem. The database
>>> cannot be recreated from coordinator, since it is already created on some
>>> of the other nodes. In such a cluster state, administrator needs to connect
>>> to datanodes and do the needed cleanup.
>>>
>>> The committed statements can be followed by statements that undo the
>>> operation, for e.g. DROP DATABASE for a CREATE DATABASE. But here again
>>> this statement can fail for some reason. Also, typically for such
>>> statements, their UNDO counterparts themselves cannot be run inside a
>>> transaction block as well. So this is not a guaranteed way to bring back
>>> the cluster to a consistent state.
>>>
>>> To find out how we can get around this issue, let's see why these
>>> statements require to be run outside a transaction block in the first
>>> place. There are two reasons why:
>>>
>>> 1. Typically such statements modify OS files and directories which
>>> cannot be rollbacked.
>>>
>>> For DMLs, the rollback does not have to be explicitly undone. MVCC takes
>>> care of it. But for OS file operations, there is no automatic way. So such
>>> operations cannot be rollbacked. So in a transaction block, if a
>>> create-database is followed by 10 other SQL statements before commit, and
>>> one of the statements throws an error, ultimately the database won't be
>>> created but there will be database files taking up disk space, and this has
>>> happened just because the user has written the script wrongly.
>>>
>>> So by restricting such statement to be run outside a transaction block,
>>> an unrelated error won't cause garbage files to be created.
>>>
>>> The statement itself does get committed eventually as usual. And it can
>>> also get rolled back in the end. But maximum care has been taken in the
>>> statement function (for e.g. createdb) such that the chances of an error
>>> occurring *after* the files are created is least. For this, such a code
>>> segment is inside PG_ENSURE_ERROR_CLEANUP() with some error_callback
>>> function (createdb_failure_callback) which tries to clean up the files
>>> created.
>>>
>>> So the end result is that this window between files-created and
>>> error-occurred is minimized, not that such statements will never create
>>> such cleanup issues if run outside transaction block.
>>>
>>> Possible solution:
>>>
>>> So regarding Postgres-XC, if we let such statements to be run inside
>>> transaction block but only on remote nodes, what are the consequences? This
>>> will of course prevent the issue of the statement committed on one node and
>>> not the other. Also, the end user will still be prevented from running the
>>> statement inside the transaction. Moreover, for such statement, say
>>> create-database, the database will be created on all nodes or none, even if
>>> one of the nodes return error. The only issue is, if the create-database is
>>> aborted, it will leave disk space wasted on nodes where it has succeeded.
>>> But this will be caused because of some configuration issues like disk
>>> space, network down etc. The issue of other unrelated operations in the
>>> same transaction causing rollback of create-database will not occur anyways
>>> because we still don't allow it in a transaction block for the end-user.
>>>
>>> So the end result is we have solved the inconsistent cluster issue,
>>> leaving some chances of disk cleanup issue, although not due to
>>> user-queries getting aborted. So may be when such statements error out, we
>>> display a notice that files need to be cleaned up.
>>>
>> Could it be possible to store somewhere in the PGDATA folder of the node
>> involved the files that need to be cleaned up? We could use for this
>> purpose some binary encoding or something. Ultimately this would finish
>> just by being a list of files inside PGDATA to be cleaned up.
>> We could then create a system function that unlinks all the files whose
>> name have been stored on local node. As such a system function does not
>> interact with other databases it could be immutable in order to allow a
>> clean up from coordinator with EXECUTE DIRECT.
>>
>
In an unfortnate even that some files remained to be cleaned up, I think it
is better to not clean them up ourselved, because we are not sure whether
we have aborted because of some half-cooked files, or these files were
already there related to some other object. If we delete such pre-existing
files, it will leave other objects corrupt.


>>
>>> We can go further ahead to reduce this window. We split the
>>> create-database operation. We begin a transaction block, and then let
>>> datanodes create the non-file operations first, like inserting pg_database
>>> row, etc, by running them using a new function call. Don't commit it yet.
>>> Then fire the last part: file system operations, this too using another
>>> function call. And then finally commit. This file operation will be under
>>> PG_ENSURE_ERROR_CLEANUP(). Due to synchronizing these individual tasks, we
>>> reduce the window further.
>>>
>> We need to be careful here with the impact of our code on PostgreSQL
>> code. It would be a pain to have a complecated implementation here for
>> future merges.
>>
>
Yes, as you can see  in the doc I uploaded, I have come to the conclusion
that this splitting is not very much required. The only function where we
need to do this is movedb() (i.e. create-db-set-tbspace) where it has
internal commits.


>>
>>> 2. Some statements do internal commits.
>>>
>>> For e.g. movedb() calls TransactionCommit() after copying the files, and
>>> then removes the original files, so that if it crashes while removing the
>>> files, the database with the new tablespace is already committed and
>>> intact, so we just leave some old files.
>>>
>>> Such statements doing internal commits cannot be rolled back if run
>>> inside transaction block, because they already do some commits. For such
>>> statements, the above solution does not work. We need to find a separate
>>> way for these specific statements. Few of such statements include:
>>> ALTER DATABASE SET TABLESPACE
>>> CLUSTER
>>> CREATE INDEX CONCURRENTLY
>>>
>>> One similar solution is to split the individual tasks that get
>>> internally committed using different functions for each task, and run the
>>> individual functions on all the nodes synchronously. So the 2nd task does
>>> not start until the first one gets committed on all the nodes. Whether it
>>> is feasible to split the task is a question, and it depends on the
>>> particular command.
>>>
>> We would need a locking system for each task and each task step like what
>> is done for barrier.
>> Or a new communication protocol, once again like barriers. Those are once
>> again just ideas on the top of my mind.
>>
>
Check the document where it explains how we can split the
alter-db-set-tbspace commits.


>
>>
>>>
>>> As of now, I am not sure whether we can do some common changes in the
>>> way transactions are implemented to find a common solution which does not
>>> require changes for individual commands. But I will investigate more.
>>>
>> Thanks.
>> --
>> Michael Paquier
>> http://michael.otacoo.com
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>

Hi All,
I am trying to run make check on XC and getting following error.
[ashutosh@ubuntu regress]make check
make -C ../../../src/port all
make[1]: Entering directory `/work/xc_cat_update/coderoot/src/port'
make -C ../backend submake-errcodes
make[2]: Entering directory `/work/xc_cat_update/coderoot/src/backend'
make[2]: Nothing to be done for `submake-errcodes'.
make[2]: Leaving directory `/work/xc_cat_update/coderoot/src/backend'
make[1]: Leaving directory `/work/xc_cat_update/coderoot/src/port'
rm -rf ./testtablespace
mkdir ./testtablespace
../../../src/test/regress/pg_regress --inputdir=.
--temp-install=./tmp_check --top-builddir=../../..   --dlpath=.
--schedule=./parallel_schedule
============== removing existing temp installation    ==============
============== creating temporary installation        ==============
============== initializing database system           ==============
============== starting postmaster                    ==============
============== starting GTM process                   ==============
running on port 57432, pooler port 57437 with PID 18593 for Coordinator 1
running on port 57433, pooler port 57438 with PID 18594 for Coordinator 2
running on port 57434 with PID 18604 for Datanode 1
running on port 57435 with PID 18614 for Datanode 2
running on port 57436 with PID 18592 for GTM
============== setting connection information         ==============
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/tmp/.s.PGSQL.57432"?
command failed:
"/work/xc_cat_update/coderoot/src/test/regress/./tmp_check/install//work/xc_cat_update//Build/bin/psql"
-X -p 57432 -c "CREATE NODE dn1 WITH (HOST = 'localhost', type =
'datanode', PORT = 57434);" "postgres"
pg_ctl: PID file
"/work/xc_cat_update/coderoot/src/test/regress/./tmp_check/data_co1/postmaster.pid"
does not exist
Is server running?

pg_regress: could not stop postmaster: exit code was 256
make: *** [check] Error 2

Anybody has a clue, what's going wrong?

-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Something I am afraid is not possible with an external utility is control of redistribution at node level. For example, an external contrib module or utility will launch SQL queries to xc that have to be treated as global. However, redistribution needs to take care of cases like for example the reduction of nodes for replicated tables. In this case you just need to delete the data from removed nodes. Another easy example is the case of an increase of nodes for replicated tables. You need to pick up data on coordinator and then send it only to the new nodes. Those simple examples need a core management to minimize the work of redistribution inside cluster.

On 2012/07/13, at 15:09, Ashutosh Bapat <ash...@en...> wrote:

> Even, I am wondering if that would be better.
> 
> But, one thing that is essential is catalog updates. Are you suggesting that the catalog updates too should be done using some SQL?
> 
> Can you please expand more on your idea, may be providing some examples, pseudo-code etc.?
> 
> On Fri, Jul 13, 2012 at 10:36 AM, Nikhil Sontakke <ni...@st...> wrote:
> Just a thought.
> 
> If we have a utility which spews out all of these statements to
> redistribute a table across node modifications, then we can just wrap
> them inside a transaction block and just run that?
> 
> Wont it save all of the core changes?
> 
> Regards,
> Nikhils
> 
> On Fri, Jul 13, 2012 at 12:29 AM, Michael Paquier
> <mic...@gm...> wrote:
> > Hi all,
> >
> > Please find attached an updated patch adding redistribution optimizations
> > for replicated tables.
> > If the node subset of a replicated table is reduced, the necessary nodes are
> > simply truncated.
> > If it is increased, a COPY TO is done to fetch the data, and COPY FROM is
> > done only on the necessary nodes.
> > New regression tests have been added to test that.
> >
> > Regards,
> >
> >
> > On Thu, Jul 12, 2012 at 5:30 PM, Michael Paquier <mic...@gm...>
> > wrote:
> >>
> >> OK, here is the mammoth patch: 3000 lines including docs, implementation
> >> and  regressions.
> >> The code has been realigned with current master.
> >> This patch introduces the latest thing I am working on: the redistribution
> >> command tree planning and execution.
> >>
> >> As I explained before, a redistribution consists of a series of commands
> >> (TRUNCATE, REINDEX, DELETE, COPY FROM, COPY TO) that need to be determined
> >> depending on the new and old locator information of the relation. Each
> >> action can be done on a subset of nodes.
> >> This patch introduces the basic infrastructure of the command tree build
> >> and execution.
> >> For the time being, redistribution uses only what is called the default
> >> command tree consisting of:
> >> 1) COPY TO
> >> 2) TRUNCATE
> >> 3) COPY FROM
> >> 4) REINDEX
> >> But this structure can be easily completed with more complicated
> >> operations.
> >> In this patch there is still a small thing missing which is the
> >> possibility to launch a COPY FROM on a subset of nodes, particularly useful
> >> when redistribution consists of a replicated table whose set of nodes is
> >> increased.
> >> Compared to the last versions, the impact of redistribution in tablecmds.c
> >> is limited.
> >>
> >> Regards,
> >>
> >> --
> >> Michael Paquier
> >> http://michael.otacoo.com
> >
> >
> >
> >
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> > ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
> 
> 
> 
> --
> StormDB - http://www.stormdb.com
> The Database Cloud
> 
> 
> 
> -- 
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
> 

Even, I am wondering if that would be better.

But, one thing that is essential is catalog updates. Are you suggesting
that the catalog updates too should be done using some SQL?

Can you please expand more on your idea, may be providing some examples,
pseudo-code etc.?

On Fri, Jul 13, 2012 at 10:36 AM, Nikhil Sontakke <ni...@st...>wrote:

> Just a thought.
>
> If we have a utility which spews out all of these statements to
> redistribute a table across node modifications, then we can just wrap
> them inside a transaction block and just run that?
>
> Wont it save all of the core changes?
>
> Regards,
> Nikhils
>
> On Fri, Jul 13, 2012 at 12:29 AM, Michael Paquier
> <mic...@gm...> wrote:
> > Hi all,
> >
> > Please find attached an updated patch adding redistribution optimizations
> > for replicated tables.
> > If the node subset of a replicated table is reduced, the necessary nodes
> are
> > simply truncated.
> > If it is increased, a COPY TO is done to fetch the data, and COPY FROM is
> > done only on the necessary nodes.
> > New regression tests have been added to test that.
> >
> > Regards,
> >
> >
> > On Thu, Jul 12, 2012 at 5:30 PM, Michael Paquier <
> mic...@gm...>
> > wrote:
> >>
> >> OK, here is the mammoth patch: 3000 lines including docs, implementation
> >> and  regressions.
> >> The code has been realigned with current master.
> >> This patch introduces the latest thing I am working on: the
> redistribution
> >> command tree planning and execution.
> >>
> >> As I explained before, a redistribution consists of a series of commands
> >> (TRUNCATE, REINDEX, DELETE, COPY FROM, COPY TO) that need to be
> determined
> >> depending on the new and old locator information of the relation. Each
> >> action can be done on a subset of nodes.
> >> This patch introduces the basic infrastructure of the command tree build
> >> and execution.
> >> For the time being, redistribution uses only what is called the default
> >> command tree consisting of:
> >> 1) COPY TO
> >> 2) TRUNCATE
> >> 3) COPY FROM
> >> 4) REINDEX
> >> But this structure can be easily completed with more complicated
> >> operations.
> >> In this patch there is still a small thing missing which is the
> >> possibility to launch a COPY FROM on a subset of nodes, particularly
> useful
> >> when redistribution consists of a replicated table whose set of nodes is
> >> increased.
> >> Compared to the last versions, the impact of redistribution in
> tablecmds.c
> >> is limited.
> >>
> >> Regards,
> >>
> >> --
> >> Michael Paquier
> >> http://michael.otacoo.com
> >
> >
> >
> >
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>
>
>
> --
> StormDB - http://www.stormdb.com
> The Database Cloud
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

On Fri, Jul 13, 2012 at 2:06 PM, Nikhil Sontakke <ni...@st...>wrote:

> Just a thought.
>
> If we have a utility which spews out all of these statements to
> redistribute a table across node modifications, then we can just wrap
> them inside a transaction block and just run that?
> Wont it save all of the core changes?
>
On the contrary it is a core feature. You need to provide users a direct
way to rebalance data with core of XC.
Btw, I made my stuff such as the core is minimally impacted and only in
gram.y and tablecmds.c.
-- 
Michael Paquier
http://michael.otacoo.com

Just a thought.

If we have a utility which spews out all of these statements to
redistribute a table across node modifications, then we can just wrap
them inside a transaction block and just run that?

Wont it save all of the core changes?

Regards,
Nikhils

On Fri, Jul 13, 2012 at 12:29 AM, Michael Paquier
<mic...@gm...> wrote:
> Hi all,
>
> Please find attached an updated patch adding redistribution optimizations
> for replicated tables.
> If the node subset of a replicated table is reduced, the necessary nodes are
> simply truncated.
> If it is increased, a COPY TO is done to fetch the data, and COPY FROM is
> done only on the necessary nodes.
> New regression tests have been added to test that.
>
> Regards,
>
>
> On Thu, Jul 12, 2012 at 5:30 PM, Michael Paquier <mic...@gm...>
> wrote:
>>
>> OK, here is the mammoth patch: 3000 lines including docs, implementation
>> and  regressions.
>> The code has been realigned with current master.
>> This patch introduces the latest thing I am working on: the redistribution
>> command tree planning and execution.
>>
>> As I explained before, a redistribution consists of a series of commands
>> (TRUNCATE, REINDEX, DELETE, COPY FROM, COPY TO) that need to be determined
>> depending on the new and old locator information of the relation. Each
>> action can be done on a subset of nodes.
>> This patch introduces the basic infrastructure of the command tree build
>> and execution.
>> For the time being, redistribution uses only what is called the default
>> command tree consisting of:
>> 1) COPY TO
>> 2) TRUNCATE
>> 3) COPY FROM
>> 4) REINDEX
>> But this structure can be easily completed with more complicated
>> operations.
>> In this patch there is still a small thing missing which is the
>> possibility to launch a COPY FROM on a subset of nodes, particularly useful
>> when redistribution consists of a replicated table whose set of nodes is
>> increased.
>> Compared to the last versions, the impact of redistribution in tablecmds.c
>> is limited.
>>
>> Regards,
>>
>> --
>> Michael Paquier
>> http://michael.otacoo.com
>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
StormDB - http://www.stormdb.com
The Database Cloud

On Fri, Jul 13, 2012 at 11:45 AM, nop <no...@in...> wrote:

> Hi,
>
> Currently HEAD won't build because of a merge conflict. The attached patch
> resolves that.
>
> Cheers,
> Andrew
>
Thanks this has been fixed:
http://github.com/postgres-xc/postgres-xc/commit/5655bc5
I didn't reused your patch, as you did not take into account the options
"-undefined suppress -flat_namespace" in LINK.shared that are used when
compiling code on MacOS.
Regards.
-- 
Michael Paquier
http://michael.otacoo.com