postgres-xc-general Mailing List for Postgres-XC (Page 2)

On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote:
> Thanks for the question.
>
> With XC-1.0.x, we cannot add a node at runtime.   This feature is
> included in the current development and will be available with the
> next major release.

I may have missed emails discussing the design.  Can you please
summarize the design?

We could have GTM manage node membership. For example, a new candidate
data node registers itself as available to GTM (we may want some kind
of GTM authentication instead of merely connecting to the ip:port).
ADD NODE is executed on one single coordinator, which validates
against GTM, updates the status, adds it to the cluster and notifies
the other Coordinators. There would no longer be any need for the
manual DDL that is done now.

This could be expanded in the future to help support multiple
read-only replicas for read intensive workloads, where they are
identified by distribution segment and registered in a similar
fashion. (Instead of just node id, we use segment id, where a node can
be associated with a segment, either as the primary or a read-only
slave.)

>
> The safest way is to backup the whole database with pg_dump,
> reinitialize all the nodes including the additional ones, configure
> the cluster (CREATE NODE and ALTER NODE) and then restore the data
> with pg_restore.   You may need to create each table in advance so
> that it can be distributed/replicated among all the nodes.
>
> New feature of node addition will include:
> 1) Create new coordinators and datanodes on new servers at runtime,
> which copies all the necessary catalog such as role, database, etc.,
> from existing components,
> 2) Add them to the cluster,
> 3) Reballance tables with new components.
>
> It's very helpful if you submit your requirement for this feature.
>
> Regards;
> ----------
> Koichi Suzuki
>
>
> 2012/9/6 "Marco Güttler" <mar...@gm...>:
>> Hey all,
>>
>> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition?
>> If not so, what is an alternative way to add a coordinator and datanode?
>>
>> Thanks and regards
>> Marco
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general



-- 
Mason Sharp

StormDB - http://www.stormdb.com
The Database Cloud

Well, next April to May is my wish.
----------
Koichi Suzuki


2012/9/6 Mephysto <mep...@gm...>:
> Hi Koichi,
> this feature could be very helpful for me too. Is there a forecast for
> release date of next major release of postgres XC?
>
> Thanks in advance.
>
> Mephysto
>
> On 6 September 2012 12:00, Koichi Suzuki <koi...@gm...> wrote:
>>
>> Thanks for the question.
>>
>> With XC-1.0.x, we cannot add a node at runtime.   This feature is
>> included in the current development and will be available with the
>> next major release.
>>
>> The safest way is to backup the whole database with pg_dump,
>> reinitialize all the nodes including the additional ones, configure
>> the cluster (CREATE NODE and ALTER NODE) and then restore the data
>> with pg_restore.   You may need to create each table in advance so
>> that it can be distributed/replicated among all the nodes.
>>
>> New feature of node addition will include:
>> 1) Create new coordinators and datanodes on new servers at runtime,
>> which copies all the necessary catalog such as role, database, etc.,
>> from existing components,
>> 2) Add them to the cluster,
>> 3) Reballance tables with new components.
>>
>> It's very helpful if you submit your requirement for this feature.
>>
>> Regards;
>> ----------
>> Koichi Suzuki
>>
>>
>> 2012/9/6 "Marco Güttler" <mar...@gm...>:
>> > Hey all,
>> >
>> > I have a question concerning node addition (coordinator and datanode) at
>> > runtime. Is it possible to add a node at runtime? If so, how to do node
>> > addition?
>> > If not so, what is an alternative way to add a coordinator and datanode?
>> >
>> > Thanks and regards
>> > Marco
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Live Security Virtual Conference
>> > Exclusive live event will cover all the ways today's security and
>> > threat landscape has changed and how IT managers can respond.
>> > Discussions
>> > will include endpoint security, mobile security and the latest in
>> > malware
>> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

Hi Koichi,
this feature could be very helpful for me too. Is there a forecast for
release date of next major release of postgres XC?

Thanks in advance.

Mephysto

On 6 September 2012 12:00, Koichi Suzuki <koi...@gm...> wrote:

> Thanks for the question.
>
> With XC-1.0.x, we cannot add a node at runtime.   This feature is
> included in the current development and will be available with the
> next major release.
>
> The safest way is to backup the whole database with pg_dump,
> reinitialize all the nodes including the additional ones, configure
> the cluster (CREATE NODE and ALTER NODE) and then restore the data
> with pg_restore.   You may need to create each table in advance so
> that it can be distributed/replicated among all the nodes.
>
> New feature of node addition will include:
> 1) Create new coordinators and datanodes on new servers at runtime,
> which copies all the necessary catalog such as role, database, etc.,
> from existing components,
> 2) Add them to the cluster,
> 3) Reballance tables with new components.
>
> It's very helpful if you submit your requirement for this feature.
>
> Regards;
> ----------
> Koichi Suzuki
>
>
> 2012/9/6 "Marco Güttler" <mar...@gm...>:
> > Hey all,
> >
> > I have a question concerning node addition (coordinator and datanode) at
> runtime. Is it possible to add a node at runtime? If so, how to do node
> addition?
> > If not so, what is an alternative way to add a coordinator and datanode?
> >
> > Thanks and regards
> > Marco
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

Thanks for the question.

With XC-1.0.x, we cannot add a node at runtime.   This feature is
included in the current development and will be available with the
next major release.

The safest way is to backup the whole database with pg_dump,
reinitialize all the nodes including the additional ones, configure
the cluster (CREATE NODE and ALTER NODE) and then restore the data
with pg_restore.   You may need to create each table in advance so
that it can be distributed/replicated among all the nodes.

New feature of node addition will include:
1) Create new coordinators and datanodes on new servers at runtime,
which copies all the necessary catalog such as role, database, etc.,
from existing components,
2) Add them to the cluster,
3) Reballance tables with new components.

It's very helpful if you submit your requirement for this feature.

Regards;
----------
Koichi Suzuki


2012/9/6 "Marco Güttler" <mar...@gm...>:
> Hey all,
>
> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition?
> If not so, what is an alternative way to add a coordinator and datanode?
>
> Thanks and regards
> Marco
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

Hey all,

I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition?
If not so, what is an alternative way to add a coordinator and datanode?

Thanks and regards
Marco

Hi all,

Postgres-XC, read/write-scalable multi-master symmetric cluster based on
PostgreSQL, version 1.0.1 has been released today.

This minor release is based on the latest PostgreSQL 9.1.5+alpha, meaning
that all the patches in PostgreSQL 9.1 stable branch have been merged up to
commit d10ddf4 (3rd of September 2012).

You can download the source tarball directly from here:
http://sourceforge.net/projects/postgres-xc/files/latest/download<https://sourceforge.net/projects/postgres-xc/files/latest/download>
Like PostgreSQL, this tarball contains all the HTML and man documentation.

The documentation of 1.0, including release notes, is available here:
http://postgres-xc.sourceforge.net/docs/1_0/.

Around 20 bugs have been fixed since 1.0.0, with in particular those fixes:
- Applications like pgadmin had problems to connect to Postgres-XC server.
- Drop of sequence was not managed correctly when its database was dropped.
- etc.
You can find all the details in the release notes here:
http://postgres-xc.sourceforge.net/docs/1_0/release-xc-1-0-1.html

The project can be followed on Source Forge: http://postgres-xc
.sourceforge.net/.
And a couple of GIT repositories are used for development:
- SourceForge: http://postgres-xc.git.sourceforge.net/git/gitweb.cgi?p=
postgres-xc/postgres-xc;a=summary
- Github: https://github.com/postgres-xc/postgres-xc
- Twitter: @PostgresXCBot, bot giving tweets about the commits in
Postgres-XC GIT repository

The project members are currently working hard on the next version of
Postgres-XC that will include those features:
- triggers (being implemented)
- Merge with PostgreSQL 9.2 code (already committed)
- RETURNING, WHERE CURRENT OF (being implemented)
- Insure consistency of utilities that cannot run inside transaction block
(ex: CREATE DATABASE safely insured in multiple nodes, being implemented)
- Change table distribution type with ALTER TABLE (already committed)
- Support for cursors (already committed)
- Stuff related to node addition and deletion
- and other things...

The project is under the same license as PostgreSQL, and is managed under a
single entity called "Postgres-XC Development Group".
In order to keep in touch with the project, whose development follows the
same model as PostgreSQL, you can register to the following mailing lists:
- pos...@li..., for general questions.
Registration can be done here: https://lists.sourceforge.net/lists/listinfo/
postgres-xc-general
- pos...@li.... hachers mailing list.
Registration can be done here: https://lists.sourceforge.net/lists/listinfo/
postgres-xc-developers

See you soon on the project mailing list, and have fun with this stable
release.
-- 
Michael Paquier
http://michael.otacoo.com