postgres-xc-general Mailing List for Postgres-XC

On Wed, Aug 1, 2012 at 10:52 PM, Benjamin Henrion <bh...@ud...> wrote:

> On Wed, Aug 1, 2012 at 3:31 PM, Michael Paquier
> <mic...@gm...> wrote:
> >
> > On 2012/08/01, at 22:28, Benjamin Henrion <bh...@ud...> wrote:
> >
> >> On Fri, Jul 27, 2012 at 6:09 PM, Joshua D. Drake <jd...@co...>
> wrote:
> >>>
> >>> Hello,
> >>>
> >>> That would be very helpful. Thank you for offering.
> >>
> >> I just installed postgres-xc debian package that's available in SID in
> >> an openvz container, now I have this stuff running:
> >>
> >> ========================================================================
> >> root@sid /var/lib/postgres-xc [13]# ps aux
> >> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> >> root         1  0.0  0.0  10588   852 ?        Ss   15:20   0:00 init
> [2]
> >> root       911  0.0  0.0  52584  1624 ?        Sl   15:20   0:00
> >> /usr/sbin/rsyslogd -c5
> >> root       922  0.0  0.0  18816   852 ?        Ss   15:20   0:00
> /usr/sbin/cron
> >> root       929  0.0  0.0  18644   624 ?        Ss   15:20   0:00 vzctl:
> pts/0
> >> root       930  0.0  0.0  17788  2008 pts/0    Ss   15:20   0:00 -bash
> >> 101       8083  0.3  0.1 101552  9140 ?        S    15:24   0:00
> >> /usr/bin/postgres -C -D /var/lib/postgres-xc/coord
> >> 101       8093  0.0  0.0 101536  1608 ?        Ss   15:24   0:00
> >> postgres: pooler process
> >> 101       8094  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
> >> postgres: writer process
> >> 101       8095  0.0  0.0 101536  1812 ?        Ss   15:24   0:00
> >> postgres: wal writer process
> >> 101       8096  0.0  0.0 102416  3232 ?        Ss   15:24   0:00
> >> postgres: autovacuum launcher process
> >> 101       8097  0.0  0.0  69496  1700 ?        Ss   15:24   0:00
> >> postgres: stats collector process
> >> 101       8127  0.3  0.1 101552  9140 ?        S    15:24   0:00
> >> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode1
> >> 101       8135  0.0  0.0 101536  1988 ?        Ss   15:24   0:00
> >> postgres: writer process
> >> 101       8136  0.0  0.0 101536  1784 ?        Ss   15:24   0:00
> >> postgres: wal writer process
> >> 101       8137  0.0  0.0 102284  2776 ?        Ss   15:24   0:00
> >> postgres: autovacuum launcher process
> >> 101       8138  0.0  0.0  69496  1636 ?        Ss   15:24   0:00
> >> postgres: stats collector process
> >> 101       8145  0.3  0.1 101552  9136 ?        S    15:24   0:00
> >> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode2
> >> 101       8153  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
> >> postgres: writer process
> >> 101       8154  0.0  0.0 101536  1780 ?        Ss   15:24   0:00
> >> postgres: wal writer process
> >> 101       8155  0.0  0.0 102284  2772 ?        Ss   15:24   0:00
> >> postgres: autovacuum launcher process
> >> 101       8156  0.0  0.0  69496  1632 ?        Ss   15:24   0:00
> >> postgres: stats collector process
> >> root      8781  0.0  0.0  17784  1984 pts/0    S    15:27   0:00 bash
> >> root      8792  0.0  0.0  15236  1136 pts/0    R+   15:29   0:00 ps aux
> >> root@sid /var/lib/postgres-xc [14]#
> >> ========================================================================
> >>
> >> If anybody is interested in a copy of the openvz container (basically
> >> a rootfs), let me know, I will push it somewhere.
> > That would be cool!
> >>
> >> From the debian package, how do I configure another box so that the
> >> two databases are in master-master mode?
> > I am sure Vladimir knows about that, I am not using the Debian packages
> at all.
>
> Basically I end up with the following setup:
>
>
> http://michael.otacoo.com/postgresql-2/start-a-postgres-xc-cluster-in-more-or-less-10-commands/
>
> Now I do not understand how master-master can work since the
> coordinator is still a SPOF.
>
OK. I have never used the debian packages so I thought that it installed a
small cluster for you automatically like what we can see for postgres on
ubuntu for example.

When you want to create a cluster with multiple Coordinators, the setting
is the same, except that when registering nodes on Coordinators you need to
do it for each Coordinator, and you need to register on each Coordinator
all the other Coordinators.
For example, for a 2Coordinator/2Datanode cluster on the same server
1) Initialize:
cd $HOME/pgsql
initgtm -Z gtm -D gtm # Initialize GTM
initdb -D datanode1 --nodename dn1 # Initialize Datanode 1
initdb -D datanode2 --nodename dn2 # Initialize Datanode 2
initdb -D coord1 --nodename co1 # Initialize Coordinator 1
initdb -D coord2 --nodename co2 # Initialize Coordinator 2
2) Change the port numbers if necessary...
3) start-up
gtm -D gtm & # Start-up GTM
postgres -X -D datanode1 -i & # Start Datanode 1
postgres -X -D datanode2 -i & # Start Datanode 2
postgres -C -D coord1 -i & # Start Coordinator 1
postgres -C -D coord2 -i & # Start Coordinator 2
4) Define all the nodes on Coordinator 1 and update pooler cache.
psql -p $CO1_PORT -c "CREATE NODE dn1 WITH (TYPE='datanode', PORT=$DN1_PORT)"
postgres #define dn1
psql -p $CO1_PORT -c "CREATE NODE dn2 WITH (TYPE='datanode', PORT=$DN2_PORT)"
postgres #define dn2
psql -p $CO1_PORT -c "CREATE NODE co2 WITH (TYPE='coordinator',
PORT=$CO2_PORT)" postgres #define co2
psql -p $CO1_PORT -c "SELECT pgxc_pool_reload()" postgres
5) Define all the nodes on Coordinator 2 and update pooler cache
psql -p $CO2_PORT -c "CREATE NODE dn1 WITH (TYPE='datanode', PORT=$DN1_PORT)"
postgres #define dn1
psql -p $CO2_PORT -c "CREATE NODE dn2 WITH (TYPE='datanode', PORT=$DN2_PORT)"
postgres #define dn2
psql -p $CO2_PORT -c "CREATE NODE co1 WITH (TYPE='coordinator',
PORT=$CO1_PORT)" postgres #define co1
psql -p $CO2_PORT -c "SELECT pgxc_pool_reload()" postgres

So simply you need to register on each Coordinator all the other nodes of
your cluster.
-- 
Michael Paquier
http://michael.otacoo.com

On Wed, Aug 1, 2012 at 3:31 PM, Michael Paquier
<mic...@gm...> wrote:
>
> On 2012/08/01, at 22:28, Benjamin Henrion <bh...@ud...> wrote:
>
>> On Fri, Jul 27, 2012 at 6:09 PM, Joshua D. Drake <jd...@co...> wrote:
>>>
>>> Hello,
>>>
>>> That would be very helpful. Thank you for offering.
>>
>> I just installed postgres-xc debian package that's available in SID in
>> an openvz container, now I have this stuff running:
>>
>> ========================================================================
>> root@sid /var/lib/postgres-xc [13]# ps aux
>> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> root         1  0.0  0.0  10588   852 ?        Ss   15:20   0:00 init [2]
>> root       911  0.0  0.0  52584  1624 ?        Sl   15:20   0:00
>> /usr/sbin/rsyslogd -c5
>> root       922  0.0  0.0  18816   852 ?        Ss   15:20   0:00 /usr/sbin/cron
>> root       929  0.0  0.0  18644   624 ?        Ss   15:20   0:00 vzctl: pts/0
>> root       930  0.0  0.0  17788  2008 pts/0    Ss   15:20   0:00 -bash
>> 101       8083  0.3  0.1 101552  9140 ?        S    15:24   0:00
>> /usr/bin/postgres -C -D /var/lib/postgres-xc/coord
>> 101       8093  0.0  0.0 101536  1608 ?        Ss   15:24   0:00
>> postgres: pooler process
>> 101       8094  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
>> postgres: writer process
>> 101       8095  0.0  0.0 101536  1812 ?        Ss   15:24   0:00
>> postgres: wal writer process
>> 101       8096  0.0  0.0 102416  3232 ?        Ss   15:24   0:00
>> postgres: autovacuum launcher process
>> 101       8097  0.0  0.0  69496  1700 ?        Ss   15:24   0:00
>> postgres: stats collector process
>> 101       8127  0.3  0.1 101552  9140 ?        S    15:24   0:00
>> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode1
>> 101       8135  0.0  0.0 101536  1988 ?        Ss   15:24   0:00
>> postgres: writer process
>> 101       8136  0.0  0.0 101536  1784 ?        Ss   15:24   0:00
>> postgres: wal writer process
>> 101       8137  0.0  0.0 102284  2776 ?        Ss   15:24   0:00
>> postgres: autovacuum launcher process
>> 101       8138  0.0  0.0  69496  1636 ?        Ss   15:24   0:00
>> postgres: stats collector process
>> 101       8145  0.3  0.1 101552  9136 ?        S    15:24   0:00
>> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode2
>> 101       8153  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
>> postgres: writer process
>> 101       8154  0.0  0.0 101536  1780 ?        Ss   15:24   0:00
>> postgres: wal writer process
>> 101       8155  0.0  0.0 102284  2772 ?        Ss   15:24   0:00
>> postgres: autovacuum launcher process
>> 101       8156  0.0  0.0  69496  1632 ?        Ss   15:24   0:00
>> postgres: stats collector process
>> root      8781  0.0  0.0  17784  1984 pts/0    S    15:27   0:00 bash
>> root      8792  0.0  0.0  15236  1136 pts/0    R+   15:29   0:00 ps aux
>> root@sid /var/lib/postgres-xc [14]#
>> ========================================================================
>>
>> If anybody is interested in a copy of the openvz container (basically
>> a rootfs), let me know, I will push it somewhere.
> That would be cool!
>>
>> From the debian package, how do I configure another box so that the
>> two databases are in master-master mode?
> I am sure Vladimir knows about that, I am not using the Debian packages at all.

Basically I end up with the following setup:

http://michael.otacoo.com/postgresql-2/start-a-postgres-xc-cluster-in-more-or-less-10-commands/

Now I do not understand how master-master can work since the
coordinator is still a SPOF.

--
Benjamin Henrion <bhenrion at ffii.org>
FFII Brussels - +32-484-566109 - +32-2-3500762
"In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators."

On 2012/08/01, at 22:28, Benjamin Henrion <bh...@ud...> wrote:

> On Fri, Jul 27, 2012 at 6:09 PM, Joshua D. Drake <jd...@co...> wrote:
>> 
>> Hello,
>> 
>> That would be very helpful. Thank you for offering.
> 
> I just installed postgres-xc debian package that's available in SID in
> an openvz container, now I have this stuff running:
> 
> ========================================================================
> root@sid /var/lib/postgres-xc [13]# ps aux
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root         1  0.0  0.0  10588   852 ?        Ss   15:20   0:00 init [2]
> root       911  0.0  0.0  52584  1624 ?        Sl   15:20   0:00
> /usr/sbin/rsyslogd -c5
> root       922  0.0  0.0  18816   852 ?        Ss   15:20   0:00 /usr/sbin/cron
> root       929  0.0  0.0  18644   624 ?        Ss   15:20   0:00 vzctl: pts/0
> root       930  0.0  0.0  17788  2008 pts/0    Ss   15:20   0:00 -bash
> 101       8083  0.3  0.1 101552  9140 ?        S    15:24   0:00
> /usr/bin/postgres -C -D /var/lib/postgres-xc/coord
> 101       8093  0.0  0.0 101536  1608 ?        Ss   15:24   0:00
> postgres: pooler process
> 101       8094  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
> postgres: writer process
> 101       8095  0.0  0.0 101536  1812 ?        Ss   15:24   0:00
> postgres: wal writer process
> 101       8096  0.0  0.0 102416  3232 ?        Ss   15:24   0:00
> postgres: autovacuum launcher process
> 101       8097  0.0  0.0  69496  1700 ?        Ss   15:24   0:00
> postgres: stats collector process
> 101       8127  0.3  0.1 101552  9140 ?        S    15:24   0:00
> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode1
> 101       8135  0.0  0.0 101536  1988 ?        Ss   15:24   0:00
> postgres: writer process
> 101       8136  0.0  0.0 101536  1784 ?        Ss   15:24   0:00
> postgres: wal writer process
> 101       8137  0.0  0.0 102284  2776 ?        Ss   15:24   0:00
> postgres: autovacuum launcher process
> 101       8138  0.0  0.0  69496  1636 ?        Ss   15:24   0:00
> postgres: stats collector process
> 101       8145  0.3  0.1 101552  9136 ?        S    15:24   0:00
> /usr/bin/postgres -X -D /var/lib/postgres-xc/datanode2
> 101       8153  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
> postgres: writer process
> 101       8154  0.0  0.0 101536  1780 ?        Ss   15:24   0:00
> postgres: wal writer process
> 101       8155  0.0  0.0 102284  2772 ?        Ss   15:24   0:00
> postgres: autovacuum launcher process
> 101       8156  0.0  0.0  69496  1632 ?        Ss   15:24   0:00
> postgres: stats collector process
> root      8781  0.0  0.0  17784  1984 pts/0    S    15:27   0:00 bash
> root      8792  0.0  0.0  15236  1136 pts/0    R+   15:29   0:00 ps aux
> root@sid /var/lib/postgres-xc [14]#
> ========================================================================
> 
> If anybody is interested in a copy of the openvz container (basically
> a rootfs), let me know, I will push it somewhere.
That would be cool!
> 
> From the debian package, how do I configure another box so that the
> two databases are in master-master mode?
I am sure Vladimir knows about that, I am not using the Debian packages at all.

Thanks, 
Michael
> 
> Best,
> 
> --
> Benjamin Henrion <bhenrion at ffii.org>
> FFII Brussels - +32-484-566109 - +32-2-3500762
> "In July 2005, after several failed attempts to legalise software
> patents in Europe, the patent establishment changed its strategy.
> Instead of explicitly seeking to sanction the patentability of
> software, they are now seeking to create a central European patent
> court, which would establish and enforce patentability rules in their
> favor, without any possibility of correction by competing courts or
> democratically elected legislators."

On 2012/08/01, at 21:09, Ashutosh Bapat <ash...@en...> wrote:

> Can Development group members twit on this channel (of course twits about XC only).
The goal of this bot is only automatic git commit information and perhaps release info, not more. You can still use your personal twitter channel for things you want to tell about xc. Don't forget the hash tag #pgxc!

> 
> On Wed, Aug 1, 2012 at 5:36 PM, Michael Paquier <mic...@gm...> wrote:
> Hi all,
> 
> I spent some time today to setting up a twitter account for Postgres-XC project.
> Here is more about it:
> - Twitter username: @PostgresXCBot
> - Twitter URL: http://twitter.com/PostgresXCBot
> 
> This twitter thread will be used to send information about Postgres-XC like releases or official information.
> Also, it acts as a Git commit bot, meaning that each time a commit is done in Github repository a twit the commit is sent automatically.
> This way you can easily follow the latest development of Postgres-XC.
> Thanks,
> -- 
> Michael Paquier
> http://michael.otacoo.com
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> 
> 
> 
> 
> -- 
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
> 

On Fri, Jul 27, 2012 at 6:09 PM, Joshua D. Drake <jd...@co...> wrote:
>
> Hello,
>
> That would be very helpful. Thank you for offering.

I just installed postgres-xc debian package that's available in SID in
an openvz container, now I have this stuff running:

========================================================================
root@sid /var/lib/postgres-xc [13]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  10588   852 ?        Ss   15:20   0:00 init [2]
root       911  0.0  0.0  52584  1624 ?        Sl   15:20   0:00
/usr/sbin/rsyslogd -c5
root       922  0.0  0.0  18816   852 ?        Ss   15:20   0:00 /usr/sbin/cron
root       929  0.0  0.0  18644   624 ?        Ss   15:20   0:00 vzctl: pts/0
root       930  0.0  0.0  17788  2008 pts/0    Ss   15:20   0:00 -bash
101       8083  0.3  0.1 101552  9140 ?        S    15:24   0:00
/usr/bin/postgres -C -D /var/lib/postgres-xc/coord
101       8093  0.0  0.0 101536  1608 ?        Ss   15:24   0:00
postgres: pooler process
101       8094  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
postgres: writer process
101       8095  0.0  0.0 101536  1812 ?        Ss   15:24   0:00
postgres: wal writer process
101       8096  0.0  0.0 102416  3232 ?        Ss   15:24   0:00
postgres: autovacuum launcher process
101       8097  0.0  0.0  69496  1700 ?        Ss   15:24   0:00
postgres: stats collector process
101       8127  0.3  0.1 101552  9140 ?        S    15:24   0:00
/usr/bin/postgres -X -D /var/lib/postgres-xc/datanode1
101       8135  0.0  0.0 101536  1988 ?        Ss   15:24   0:00
postgres: writer process
101       8136  0.0  0.0 101536  1784 ?        Ss   15:24   0:00
postgres: wal writer process
101       8137  0.0  0.0 102284  2776 ?        Ss   15:24   0:00
postgres: autovacuum launcher process
101       8138  0.0  0.0  69496  1636 ?        Ss   15:24   0:00
postgres: stats collector process
101       8145  0.3  0.1 101552  9136 ?        S    15:24   0:00
/usr/bin/postgres -X -D /var/lib/postgres-xc/datanode2
101       8153  0.0  0.0 101536  1984 ?        Ss   15:24   0:00
postgres: writer process
101       8154  0.0  0.0 101536  1780 ?        Ss   15:24   0:00
postgres: wal writer process
101       8155  0.0  0.0 102284  2772 ?        Ss   15:24   0:00
postgres: autovacuum launcher process
101       8156  0.0  0.0  69496  1632 ?        Ss   15:24   0:00
postgres: stats collector process
root      8781  0.0  0.0  17784  1984 pts/0    S    15:27   0:00 bash
root      8792  0.0  0.0  15236  1136 pts/0    R+   15:29   0:00 ps aux
root@sid /var/lib/postgres-xc [14]#
========================================================================

If anybody is interested in a copy of the openvz container (basically
a rootfs), let me know, I will push it somewhere.

>From the debian package, how do I configure another box so that the
two databases are in master-master mode?

Best,

--
Benjamin Henrion <bhenrion at ffii.org>
FFII Brussels - +32-484-566109 - +32-2-3500762
"In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators."

Can Development group members twit on this channel (of course twits about
XC only).

On Wed, Aug 1, 2012 at 5:36 PM, Michael Paquier
<mic...@gm...>wrote:

> Hi all,
>
> I spent some time today to setting up a twitter account for Postgres-XC
> project.
> Here is more about it:
> - Twitter username: @PostgresXCBot
> - Twitter URL: http://twitter.com/PostgresXCBot
>
> This twitter thread will be used to send information about Postgres-XC
> like releases or official information.
> Also, it acts as a Git commit bot, meaning that each time a commit is done
> in Github repository a twit the commit is sent automatically.
> This way you can easily follow the latest development of Postgres-XC.
> Thanks,
> --
> Michael Paquier
> http://michael.otacoo.com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi all,

I spent some time today to setting up a twitter account for Postgres-XC
project.
Here is more about it:
- Twitter username: @PostgresXCBot
- Twitter URL: http://twitter.com/PostgresXCBot

This twitter thread will be used to send information about Postgres-XC like
releases or official information.
Also, it acts as a Git commit bot, meaning that each time a commit is done
in Github repository a twit the commit is sent automatically.
This way you can easily follow the latest development of Postgres-XC.
Thanks,
-- 
Michael Paquier
http://michael.otacoo.com

On Wed, Aug 1, 2012 at 6:21 PM, Vladimir Stavrinov <vst...@gm...>wrote:

> On Tue, Jul 31, 2012 at 8:02 PM, Mason Sharp <ma...@st...> wrote:
>
> > I think you misunderstood. Tables can be either distributed or
> > replicated across the database segments. Each segment in turn can be
> > have multiple synchronous replicas, similar to PostgreSQL's
> > synchronous replication.
>
> Thank You very much for clarification! It is the same as written on XC
> home page. If I don't understand that, I couldn't write all above in
> this thread as well I couldn't provide overall tests of all of those
> feature before write here. Do You read this thread completely?
>
> > multiple nodes, gaining write scalability.  The overhead and added
> > latency for having replicas of each database segment is relatively
> > small, so you need not think of that as preventing "write balance", as
> > you say.
>
> Write scalability ( I prefer term, which You are using here - "write
> balance" because scalability means changing of data nodes number)
> means that You can write to all N nodes faster then to single one.
> This is possible only for distributed data. If You write all 100% data
> to every node it is not possible. If You don't want consider standby
> server as node - it is wrong, because for load balancing every
> hardware node is meaningful.
>
> Meanwhile, I don't like idea of using standby at all, because it
> should be consider as external solution. When I wrote above about
> "asynchronous replication", I imply improving existing XC replication
> technology, but on node level instead of table.
>
> > know about database segment replicas. Up until now the project has
> > focused on the challenges of the core database and not so much dealing
> > with stuff on the outside of it, like HA.
>
> I thought HA & LB is main feature of any cluster.
>
Transparency and scalability are even more important.
-- 
Michael Paquier
http://michael.otacoo.com

On Tue, Jul 31, 2012 at 8:02 PM, Mason Sharp <ma...@st...> wrote:

> I think you misunderstood. Tables can be either distributed or
> replicated across the database segments. Each segment in turn can be
> have multiple synchronous replicas, similar to PostgreSQL's
> synchronous replication.

Thank You very much for clarification! It is the same as written on XC
home page. If I don't understand that, I couldn't write all above in
this thread as well I couldn't provide overall tests of all of those
feature before write here. Do You read this thread completely?

> multiple nodes, gaining write scalability.  The overhead and added
> latency for having replicas of each database segment is relatively
> small, so you need not think of that as preventing "write balance", as
> you say.

Write scalability ( I prefer term, which You are using here - "write
balance" because scalability means changing of data nodes number)
means that You can write to all N nodes faster then to single one.
This is possible only for distributed data. If You write all 100% data
to every node it is not possible. If You don't want consider standby
server as node - it is wrong, because for load balancing every
hardware node is meaningful.

Meanwhile, I don't like idea of using standby at all, because it
should be consider as external solution. When I wrote above about
"asynchronous replication", I imply improving existing XC replication
technology, but on node level instead of table.

> know about database segment replicas. Up until now the project has
> focused on the challenges of the core database and not so much dealing
> with stuff on the outside of it, like HA.

I thought HA & LB is main feature of any cluster.

Hi all,

Well, I am sure you have noticed that I committed the 9.2 merge code :)
The commit message was really huge, so it has been bypassed from the commit
ML.
I have been able to drastically reduce the number of failures and bugs that
have been introduced by the 9.2 merge.
And the number of regression failures I am seeing is down to 6 for 146
tests. The code is now fairly stable even if regressions need to be
polished a bit.
I have not been able to make long test runs yet but stabilization can be
done a bit later.
Features like streaming replication work correctly. I haven't done tests
with cascading yet. However I expect it to work also smoothly.
Some regression tests might need to be reinforced with additional ORDER BY.
But this tunning can come later.

An important thing => Now option -C is used by the binary postgres,
conflicting with out -C to startup coordinator.
In consequence, I changed -C to --coordinator and -X to --datanode. All the
docs are updated. There are no consequences in pg_ctl and related.
A query has been disabled in join.sql (execute foo(true)) because it was
taking too long. Ashutosh is working on improving the join there.

Then, I have spotted 3 remaining issues, 2 minor and 1 major.
So here is the list of all the remaining issues I found and needing a
little bit of effort to be fixed, at least for the 2 first ones:
1) The aggregate test is using some new bytea tests with functions like
decode, that may result in such diffs.
  select string_agg(v, '') from
bytea_test_table;


string_agg


------------

!
\xaaff

  (1 row)
====
  select string_agg(v, '') from
bytea_test_table;


string_agg


------------

!
\xffaa

  (1 row)
It might be possible to order the output but I am still not sure how.
This has been registered as bug 3553035.
2) Failure of test select_views, problem with leaky scenarios
In some tests of select_views permissions to the relation are not set
correctly.
SELECT * FROM my_property_normal WHERE
f_leak(passwd);

ERROR:  permission denied for relation customer
This may be related to some security issues...
This has been registered as bug 3553036
3) Problems with parameters in plpgsql, plancache and rangefuncts
There are multiple errors related to parameters. PREPARE/EXECUTE is working
correctly so this looks to be related to the past parameter problems we saw
before the merge.
- Here is an error in plancache.
  select cache_test(2);
ERROR:  cache lookup failed for type 0
CONTEXT:  SQL statement "insert into t1 values($1)"
PL/pgSQL function cache_test(integer) line 5 at SQL statement
- One in plpgsql:
  select * from PField_v1 where pfname = 'PF0_1' order by slotname;
ERROR:  cache lookup failed for type 0
CONTEXT:  SQL statement "select          * from PSlot where slotname = $1"
PL/pgSQL function pslot_backlink_view(character) line 8 at SQL statement
- One in rangefuncs:
  SELECT * FROM getfoo(1) AS t1;
ERROR:  cache lookup failed for type 0
CONTEXT:  SQL statement "SELECT fooid             FROM foo WHERE fooid =
$1"
PL/pgSQL function getfoo(integer) line 1 at SQL statement
This is registered as issue

By Friday, I am pretty sure that I will be able to fix issue 1. And I will
have a look at issue 2.
I'll also continue polishing the regression tests depending on the diffs I
find in my environments.
The 3rd issue is a little bit more complicated, but as far as i understood
Ashutosh might have a solution for it.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Yes, reading from datanode slaves will enhance read scalability.

In terms of reading from datanode slave, I think we still need a
couple of improvements:

1. If you want to connect directly to a slave, current datanode
expects all the connections are from coordinators and they supply GXID
and snapshot from GTM, which psql or current libpq don't do.   If
datanode is in recovery mode and standby_mode is on, then it should
use XID and snapshot from WAL, which is now overridden by GXID and
snapshot from coordinators/GTM.

2. If you want to connect via coordinator, which is not supported yet
and we need coordinator extension.

3. If you want to visit multiple datanodes, you may get different
visibility from datanode to datanode, because synchronous replication
implies time lag from "receiving" WAL records to "replaying" them.
The time lag  may be different from datanode to datanode and the query
result could be incorrect.  I guess "BARRIER" may work to synchronize
the visibility among the datanode but we may need another visibility
control infrastructure for hot standby.

Any more inputs are welcome.

Regards;
----------
Koichi Suzuki


2012/7/31 Mason Sharp <ma...@st...>:
> On Tue, Jul 31, 2012 at 5:19 AM, Vladimir Stavrinov
> <vst...@gm...> wrote:
>> On Tue, Jul 31, 2012 at 05:35:45PM +0900, Michael Paquier wrote:
>>
>>>    The main problem that I see here is that replicating data
>>>    asynchronously breaks MVCC.
>>
>> May I cite myself?
>>
>>         When read request come in, it should go to replicated node if and only if
>>         requested data exists there, otherwise such request should go to distributed
>>         node where those data in question exists in any case.
>>
>>>    So you are never sure that the data will be here or not on your
>>>    background nodes.
>>
>> If we control where the data stored in distributed nodes, why not to control
>> the state of replicated nodes? In both cases we should know what data is where.
>>
>
> If a data node has one or more sync rep standbys, it should be
> theoretically possible to read balance those if that intelligence is
> added to the coordinator. It would not matter if that data is in a
> "distributed" or "replicated" table.
>
> If it were asynchronous, there would be more tracking that would have
> to be done to know if it is safe to load balance.
>
> In some tests we have done at StormDB, the extra overhead for sync rep
> is small, so you might as well use sync rep. With multiple replicas,
> in sync rep mode it will continue working even there is a failure with
> one of the replicas.
>
>
>>>    Ideas are of course always welcome, but if you want to add some new
>>>    features you will need to be more specific.
>>
>> I don't think what we discussing here is simply feature that may be added
>> with an patch. The idea to move storage control on cluster level touches
>> the basics and concept of XC.
>
> Yeah, that is sounding different than what is done here. I am not sure
> I understand what your requirements are and what exactly it is you
> need.  If it is about HA, there are a lot of basics in place to build
> out HA from that will probably meet your needs.
>
>>
>> --
>>
>> ***************************
>> ##  Vladimir Stavrinov
>> ##  vst...@gm...
>> ***************************
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
> --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general