postgres-xc-general Mailing List for Postgres-XC

On Tue, Apr 2, 2013 at 10:27 PM, German M. Bravo <ger...@gm...>wrote:

> I have this question, as I want to do distribution by some other criteria
> other than replication, modulo/hash or round-robin. I need to distribute
> tables as to have certain rows replicated in certain nodes and other rows
> replicated in yet other nodes.
>
> My idea is that of implementing something similar to what is proposed by
> the very interesting paper "The Little Engine(s) That Could: Scaling Online
> Social Networks" (http://ccr.sigcomm.org/online/files/p375.pdf), where
> one can partition and replicate a OSN based on the relationships the users
> have among them. The proposal guarantees all data for each user, as well as
> that of all of its neighborhoods' (or direct friends') are located in the
> same node, for performance reasons and to reduce network traffic. For that
> intend, some users end up being replicated in several nodes.
>
> To make this possible in Postgres XC, I'd need to be able to tell pgxc
> where (and how) to distribute each row. My thinking was: I wish I could
> distribute by a column that is a "vector" of values (if only a value it'd
> be simple distribution, as that of modulo or hash, if multiple values, it'd
> be replication as well as distribution), where, at the application level, I
> could tell the database: this row (user X) should be located in node X *and
> also* replicated in all the nodes of all of its friends (User A - User
> J)... so the distribution vector would be (X, A, B, C, ..., J). Replication
> of row for user X, thus, would exist in all of those nodes.
>
> Any thoughts about how difficult/tricky this would be? My first guess is
> this shouldn't be too hard to implement, thinking pgxc already supports
> both replication and distribution and this would be something like
> something in-between.
>
> Michael Paquier suggested there might be other solutions to this problem,
> using what is already there in place within existent PgXC. Can anyone think
> and explain about that?
>
For the time being you should use partitions on tables. You create a parent
table, and child subtables having different distribution types, that are
partitioned a given way. more information about this idea here:
http://sourceforge.net/mailarchive/message.php?msg_id=30421838
Regards,
-- 
Michael

I have this question, as I want to do distribution by some other criteria other than replication, modulo/hash or round-robin. I need to distribute tables as to have certain rows replicated in certain nodes and other rows replicated in yet other nodes.

My idea is that of implementing something similar to what is proposed by the very interesting paper "The Little Engine(s) That Could: Scaling Online Social Networks" (http://ccr.sigcomm.org/online/files/p375.pdf), where one can partition and replicate a OSN based on the relationships the users have among them. The proposal guarantees all data for each user, as well as that of all of its neighborhoods' (or direct friends') are located in the same node, for performance reasons and to reduce network traffic. For that intend, some users end up being replicated in several nodes.

To make this possible in Postgres XC, I'd need to be able to tell pgxc where (and how) to distribute each row. My thinking was: I wish I could distribute by a column that is a "vector" of values (if only a value it'd be simple distribution, as that of modulo or hash, if multiple values, it'd be replication as well as distribution), where, at the application level, I could tell the database: this row (user X) should be located in node X *and also* replicated in all the nodes of all of its friends (User A - User J)... so the distribution vector would be (X, A, B, C, ..., J). Replication of row for user X, thus, would exist in all of those nodes.

Any thoughts about how difficult/tricky this would be? My first guess is this shouldn't be too hard to implement, thinking pgxc already supports both replication and distribution and this would be something like something in-between.
 
Michael Paquier suggested there might be other solutions to this problem, using what is already there in place within existent PgXC. Can anyone think and explain about that?