postgres-xc-general Mailing List for Postgres-XC

XC needs some more infrastructure to be such kind of HA, besides automatic monitoring and failover RA.

1. Maintain replica of each piece of distributed tables,
2. Allow operation to run with some of the nodes missing.

I don’t think its high technical challenge but will need some development resources.

Regards;
---
Koichi Suzuki

2014/08/22 5:59、Mason Sharp <ms...@tr...<mailto:ms...@tr...>> のメール：



On Thu, Aug 21, 2014 at 4:32 PM, Joseph Mays <ma...@wi...<mailto:ma...@wi...>> wrote:
What they are looking for in the end is high-availability and load balancing. They want either system to be able to receive read and write requests, and for either one to be able to take over all the work if the other is down. My understanding is that postgres-xc is capable of this, it’s just a matter of getting there from where they are. All of this would be easier if their current database server weren’t running on one of the machines that needs to be in the two-machine cluster when we’re done.

And yes, minimizing downtime is a priority, too.


How heavy are the writes? If it is read-mainly, you could be ok. With what you describe, all tables would have to be designated as replicated, adding latency. Or, you could define two datanodes, and have a warm standby on each of the other's portion.

In any event, Postgres-XC does not have automated high availability. You would have to have something external monitor and decide when to failover. I have used Corosync/Pacemaker for example.


--
Mason Sharp

TransLattice - http://www.translattice.com<http://www.translattice.com/>
Distributed and Clustered Database Solutions


------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

On Thu, Aug 21, 2014 at 4:32 PM, Joseph Mays <ma...@wi...> wrote:

>   What they are looking for in the end is high-availability and load
> balancing. They want either system to be able to receive read and write
> requests, and for either one to be able to take over all the work if the
> other is down. My understanding is that postgres-xc is capable of this,
> it’s just a matter of getting there from where they are. All of this would
> be easier if their current database server weren’t running on one of the
> machines that needs to be in the two-machine cluster when we’re done.
>
> And yes, minimizing downtime is a priority, too.
>
>

How heavy are the writes? If it is read-mainly, you could be ok. With what
you describe, all tables would have to be designated as replicated, adding
latency. Or, you could define two datanodes, and have a warm standby on
each of the other's portion.

In any event, Postgres-XC does not have automated high availability. You
would have to have something external monitor and decide when to failover.
I have used Corosync/Pacemaker for example.


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

What they are looking for in the end is high-availability and load balancing. They want either system to be able to receive read and write requests, and for either one to be able to take over all the work if the other is down. My understanding is that postgres-xc is capable of this, it’s just a matter of getting there from where they are. All of this would be easier if their current database server weren’t running on one of the machines that needs to be in the two-machine cluster when we’re done.

And yes, minimizing downtime is a priority, too.

From: Mason Sharp 
Sent: Thursday, August 21, 2014 4:02 PM
To: Joseph Mays 
Cc: mailto:pos...@li... 
Subject: Re: [Postgres-xc-general] Move from a single-cluster machine to a multi cluster machine.





On Thu, Aug 21, 2014 at 3:04 PM, Joseph Mays <ma...@wi...> wrote:

  Okay, I have a postgres-xc “cluster” operating with only one machine in the cluster. What I want to do is load the database from the standard postgres cluster into it, then switch over to make it operational as the postgres server. Then I want to rebuild the original standard postgres machine to be another postgres-xc cluster machine and add it as a second machine to the cluster.
  There are still several things about this I have to figure out, but one question I don’t know the answer to is how I bring the two machines into “sync” at the database. Is there some way to add the second machine into the cluster and have it synchronize its database to the first machine in the cluster, or do I have to dump the database, get them running in a two machine cluster with a blank database, and the restore the dump into the two machine cluster?
  If the latter, if one machine goes down or something, so that the other is carrying all the load for a bit, how I do bring the two machines back into sync when the failed machine comes back up?

It sounds like you are trying to minimize downtime while you switchover.

If you were running on PostgreSQL 9.4, this may have been easier with the help of logical streaming replication. In the meantime, you may want to use a trigger-based scheme or Slony, where you enable triggers to track the changes, dump the data, load into XC and then play back those changes that were captured.

Regarding adding the other server back, if the load can handle it, you could already define it as a datanode and have data be loaded there. If you have a spare system and recent version of PostgreSQL, you could use streaming replication to it, promote it to being the master, then configure the other two systems.

Regards,


-- 

Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

On Thu, Aug 21, 2014 at 3:04 PM, Joseph Mays <ma...@wi...> wrote:

>   Okay, I have a postgres-xc “cluster” operating with only one machine in
> the cluster. What I want to do is load the database from the standard
> postgres cluster into it, then switch over to make it operational as the
> postgres server. Then I want to rebuild the original standard postgres
> machine to be another postgres-xc cluster machine and add it as a second
> machine to the cluster.
>
>  There are still several things about this I have to figure out, but one
> question I don’t know the answer to is how I bring the two machines into
> “sync” at the database. Is there some way to add the second machine into
> the cluster and have it synchronize its database to the first machine in
> the cluster, or do I have to dump the database, get them running in a two
> machine cluster with a blank database, and the restore the dump into the
> two machine cluster?
>
>  If the latter, if one machine goes down or something, so that the other
> is carrying all the load for a bit, how I do bring the two machines back
> into sync when the failed machine comes back up?
>
>

It sounds like you are trying to minimize downtime while you switchover.

If you were running on PostgreSQL 9.4, this may have been easier with the
help of logical streaming replication. In the meantime, you may want to use
a trigger-based scheme or Slony, where you enable triggers to track the
changes, dump the data, load into XC and then play back those changes that
were captured.

Regarding adding the other server back, if the load can handle it, you
could already define it as a datanode and have data be loaded there. If you
have a spare system and recent version of PostgreSQL, you could use
streaming replication to it, promote it to being the master, then configure
the other two systems.

Regards,


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Okay, I have a postgres-xc “cluster” operating with only one machine in the cluster. What I want to do is load the database from the standard postgres cluster into it, then switch over to make it operational as the postgres server. Then I want to rebuild the original standard postgres machine to be another postgres-xc cluster machine and add it as a second machine to the cluster.
There are still several things about this I have to figure out, but one question I don’t know the answer to is how I bring the two machines into “sync” at the database. Is there some way to add the second machine into the cluster and have it synchronize its database to the first machine in the cluster, or do I have to dump the database, get them running in a two machine cluster with a blank database, and the restore the dump into the two machine cluster?
If the latter, if one machine goes down or something, so that the other is carrying all the load for a bit, how I do bring the two machines back into sync when the failed machine comes back up?
Thank you for your patience and help.
Joe Mays

Thanks for your help! I did a dumpall out of the existing server and loaded it into the new “cluster”. It seemed to work fine, but I did not get this message until after, so I did not add any “Distribute By” statements to the CREATE TABLE commands. I will have to do the import again before turning up the server for real. I will try to do this on the next important.

My last two questions are about adding a server to the cluster, and startup commands.

First off, after I get the first server running as a clustered machine, I will be transferring all our sql client software over to it. Then I will be taking what was the original, reconfiguring it to be another machine in the cluster, and adding it to the cluster for a two-machine cluster.

Unless I am looking in the wrong place I have found very little info on how to add a machine to the cluster. Everything seems to assume that all the machines in the cluster are being created together. Does anyone know where I can find info on adding a machine, including configuring the second machine, adding it so it talks to the other, bring the databases into sync with each other, etc.?

Lastly, I am running under redhat, and I haven’t found anything about how to set up the software to start under chkconfig. It’s tricky, especially since you have 3 daemons to start (gtm, postgres datanode, and postgres coord) and I believe it matters what order they are started in.

From: Mason Sharp 
Sent: Monday, August 11, 2014 4:45 PM
To: Joseph Mays 
Subject: Re: [Postgres-xc-general] Install postgres-xc on redhat

Hi Joseph,




On Wed, Aug 6, 2014 at 4:47 PM, Joseph Mays <ma...@wi...> wrote:

  I did several searches of the entire file system for it, to no avail. I finally downloaded the source to 1.0.4 and built the binaries from the source code and installed, and initgtm as well as the other pieces are there now.

  What I am trying to do is turned a what was a slaved streaming backup of another pgsql server into a server in a postgres-xc cluster. When I have it operational, and have the database running on it, I’ll point things at it instead of the primary, then make what used to be the primary into a second server in this cluster.

  So, I a couple of questions. Is there any problem with using databases created under a standard postgres server on a postgres-xc cluster?

You would have to export it and then import it into Postgres-XC (or Postgres-XL).  You should spend some time on your CREATE TABLE statements to add the appropriate DISTRIBUTE BY clause; it may make a big difference in performance.


  Also it says gtm should not be run on the same server with the other components. Is this correct? 

You should be fine running it together if the number of servers is small and it meets your performance requirements.

  I have only two servers here, and I assumed I was going to be running gtm, as well as the other software, on both, so either could function as the primary or secondary at will. Is this a problem?

You can do that-- have datanode masters on server 1 and server 2, have server 1's datanode slave(s) on server 2, and vice versa.

Good luck!

-- 

Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

1.1 has better performance and bug fixes compared to the previous release.


On Thu, Aug 7, 2014 at 5:13 PM, Joseph Mays <ma...@wi...> wrote:

>   Okay, I eventually got a working 1-server “cluster” running using
> postgres-xc 1.04. I think probably could 1.1 working now, but all the
> documentation, everything that exists, seems to be aimed at 1.04. The
> existence of 1.1 is barely acknowledge. Is there any reason to choose 1.1
> over 1.0.4? If so, should switch to that?
>
>

Regards,

-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Okay, I eventually got a working 1-server “cluster” running using postgres-xc 1.04. I think probably could 1.1 working now, but all the documentation, everything that exists, seems to be aimed at 1.04. The existence of 1.1 is barely acknowledge. Is there any reason to choose 1.1 over 1.0.4? If so, should switch to that?

On Wed, Aug 6, 2014 at 1:51 PM, Joseph Mays <ma...@wi...> wrote:

> I was able to download the binaries and install them; the install seemed to
> run fine, all libraries seemed to run fine, then I went to take the first
> step in the docs, running initgtm, only to find it's nowhere in the system.
>
> Here is the list of packages I installed. I don't think I missed anything
> --
> I installed every postgres-xc 1.1 package that existed, but can someone
> tell
> me if there is something that is not here?
>
> postgres-xc-1.1-10.1.x86_64.rpm
> postgres-xc-contrib-1.1-10.1.x86_64.rpm
> postgres-xc-debuginfo-1.1-10.1.x86_64.rpm
> postgres-xc-devel-1.1-10.1.x86_64.rpm
> postgres-xc-docs-1.1-10.1.x86_64.rpm
> postgres-xc-libs-1.1-10.1.x86_64.rpm
> postgres-xc-plperl-1.1-10.1.x86_64.rpm
> postgres-xc-plpython-1.1-10.1.x86_64.rpm
> postgres-xc-pltcl-1.1-10.1.x86_64.rpm
> postgres-xc-server-1.1-10.1.x86_64.rpm
>

My guess it is probably there but not in your $PATH. IIRC, it is somewhere
under /usr.

-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

I was able to download the binaries and install them; the install seemed to 
run fine, all libraries seemed to run fine, then I went to take the first 
step in the docs, running initgtm, only to find it's nowhere in the system.

Here is the list of packages I installed. I don't think I missed anything --  
I installed every postgres-xc 1.1 package that existed, but can someone tell 
me if there is something that is not here?

postgres-xc-1.1-10.1.x86_64.rpm
postgres-xc-contrib-1.1-10.1.x86_64.rpm
postgres-xc-debuginfo-1.1-10.1.x86_64.rpm
postgres-xc-devel-1.1-10.1.x86_64.rpm
postgres-xc-docs-1.1-10.1.x86_64.rpm
postgres-xc-libs-1.1-10.1.x86_64.rpm
postgres-xc-plperl-1.1-10.1.x86_64.rpm
postgres-xc-plpython-1.1-10.1.x86_64.rpm
postgres-xc-pltcl-1.1-10.1.x86_64.rpm
postgres-xc-server-1.1-10.1.x86_64.rpm

-----Original Message----- 
From: Koichi Suzuki
Sent: Thursday, July 17, 2014 9:07 PM
To: Joseph Mays
Cc: Postgres-XC mailing list
Subject: Re: [Postgres-xc-general] Install postgres-xc on redhat

There's a page you can download XC rpm.   They provide only 1.0.x rpm 
though.

Try http://www.stormdb.com/content/postgres-xc-rpms-available

For later versions, you need to build binaries manually.

Regards;
---
Koichi Suzuki


2014-07-18 3:28 GMT+09:00 Joseph Mays <ma...@wi...>:
> Hi! Looking to install a postgres-xc cluster on two RHEL servers. Is there 
> a
> repo I can use to install the binaries from yum?
>
>
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>