postgres-xc-general Mailing List for Postgres-XC

Hi all,

Among the features described in: 
https://github.com/koichi-szk/PGXC-Tools/blob/master/pgxc_ctl/manual.txt 
is deleting the databases (Dropdb) and users (Dropuser) and when I try 
make use of these commands pgxc_ctl answers: command not found

PGXC Createdb testdb
Selected coord2.
PGXC Dropdb testdb
sh: Dropdb: command not found
PGXC Createuser usertest1
Selected coord1.
PGXC Dropuser usertest1
sh: Dropuser: command not found
PGXC

Carefully review the source code and found that in the folder: 
postgres-xc/contrib/pgxc_ctl , there is a file (do_command.c) in which 
reference is made and performed the execution of Createdb (line 2339) 
and Createuser (line 2369).

In this file there is no reference whatsoever to Dropdb or Dropuser .

There is another file (in the same directory) called: pgxc_ctl.bash, in 
which reference is made and run the corresponding command to Createdb, 
Dropdb, Createuser and Dropuser.

Do not remember reading during pgxc compliacion and deployment (or 
pgxc_ctl in the area of contributions ) anything regarding how to handle 
this situation.

How to resolve this issue?

The pgxc_ctl in its binary version lacks Dropdb and Dropuser commands?
I must choose between the binary version and the version bash? What 
would be the impact of this change ?

Can anyone guide me please

Thanks in advance,

Hector M. Jacas

>> My goal - I have an application that needs SQL DB and must always be 
>> up (I have a backup machine for this purpose).
>Have you thought about PostgreSQL itself for your solution. Is there any reason you'd need XC? Do you have an amount of data that >forces you to use multi-master architecture or perhaps PG itself could handle it?

I need multi-master capability, as clients might connect to both machines at the same time; Yes - my tables will be replicated.

>Yep, this is doable. If all your data is replicated you would be able to do that. However you need to keep in mind that you will not be able to write new data to node B if node A is not accessible. If you data is replicated and you need to update a table, both nodes need to work.

This is a surprise for me, this wasn't clear in the documentation I read nor at some PG-XC presentations I looked at in the internet.
Isn't this point one of the conditions for High-Availability of DB - allowing work to continue even if one of the machines failed?

>Or if you want B to be still writable, you could update the node information inside it, make it workable alone, and when server A is up again recreate a new XC node from scratch and add it again to the cluster.

What is the correct procedure for doing that? Is there a pgxc_ctl commands for doing that?

>> My questions:
>>
>> 1.       In your docs, you always put the GTM in dedicated machine.
>> a.       Is this a requirement, just an easy to understand topology or best
>> practice?
>GTM consumes a certain amount of CPU and does not need much RAM, while for your nodes you might prioritize the opposite.
>> b.      In case of best practice, what is the expected penalty in case the
>> GTM is deployed on the same machine with coordinator and datanode?
>CPU resource consumption and reduction of performance if your queries need some CPU with for example internal sort operations among other things.
O.K  got it; For now I'm trying to make it work, afterwards I'll take care for make it work faster.

>> 2.       What should I do after Machine A is back to life if I want:
>> a.       Make it act as a new slave?
>> b.      Make it become the master again?
>There is no principle of master/slave in XC like in Postgres (well you could create a slave node for an individual Coordinator/Datanode). >But basically in your configuration machine A and B have the same state.
>Only GTM is a slave.

Sorry, I meant in the context of GTM - how should I make MachineA a new GTM-slave or make it a GTM-master again?

The changes have to be in pgxc_make_modifytable() to construct a shippable
DML out of the scan plan and the modification plan. Changes will be needed
in ExecUpdate/Delete/Insert for execution of this shippable DML. Also,
there will be changes needed in pgxc_shippability_walker() to improve FQS
technique for the same.


On Mon, Oct 7, 2013 at 9:10 PM, Sandeep Gupta <gup...@gm...>wrote:

> Hi Ashutosh,
>
>  Thanks for the note.  I cannot commit right away. However, whenever you
> have some time can you mention relevant portions in the codebase where
> changes have to be made. I have a general understanding of the execution
> engine. I will take a look see if it feasible for me.
>
> Thanks.
> Sandeep
>
>
>
> On Sun, Oct 6, 2013 at 11:53 PM, Ashutosh Bapat <
> ash...@en...> wrote:
>
>>
>>
>>
>> On Sat, Oct 5, 2013 at 7:44 PM, Sandeep Gupta <gup...@gm...>wrote:
>>
>>> Thanks Michael. I understand. The only issue is that we have an update
>>> query as
>>>
>>> update T set T.a = -1 from A where A.x = T.x
>>>
>>>
>>> Both A and T and distributed by x column. The problem is that
>>> coordinator first does the join and then
>>> calls update several times at each datanode. This is turning out to be
>>> too slow. Would have
>>> been better if the entire query was shipped to the datanodes.
>>>
>>>
>> Right now there is no way to ship a DML with more than one relation
>> involved there. But that's something, I have been thinking about. If you
>> have developer resources and can produce a patch. I can help.
>>
>>
>>> Thanks.
>>> Sandeep
>>>
>>>
>>>
>>> On Sat, Oct 5, 2013 at 6:27 AM, Michael Paquier <
>>> mic...@gm...> wrote:
>>>
>>>> On Sat, Oct 5, 2013 at 2:58 AM, Sandeep Gupta <gup...@gm...>
>>>> wrote:
>>>> >  I understand that the datanodes are read only and that
>>>> updates/insert can
>>>> > happen at coordinator.
>>>> You got it.
>>>>
>>>> > Also, it does not allow modification of column over which the records
>>>> are distributed.
>>>> Hum no, 1.1 allows ALTER TABLE that you can use to change the
>>>> distribution type of a table.
>>>>
>>>> > However, in case I know what I am doing, it there anyway possible to
>>>> modify
>>>> > the values directly at datanodes.
>>>> > The modifications are not over column over which distribution happens.
>>>> If you mean by connecting directly to the Datanodes, no. You would
>>>> break data consistency if table is replicated by the way by doing
>>>> that. Let the Coordinator planner do the job and choose the remote
>>>> nodes for you.
>>>>
>>>> There have been discussion to merge Coordinators and Datanodes
>>>> together though. This would allow what you say, with a simpler cluster
>>>> design.
>>>> --
>>>> Michael
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>>> from
>>> the latest Intel processors and coprocessors. See abstracts and register
>>> >
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EnterpriseDB Corporation
>> The Postgres Database Company
>>
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On 2013/10/07, at 23:27, Julian <jul...@gm...<mailto:jul...@gm...>> wrote:


I can not just issue "add coordinator master",it alway asked me for more
here is my step:

# Deploy binaries
PGXC$ depoly all
#initialize everyting: one GTM Master, three GTM_Proxy, three coordinator, three datanode
PGXC$ init all

These commands are for the initial ones.     I think you've already done that.

.....
PGXC$ Createdb pgxc
.....
PGXC$ Psql
pgxc=# select * from pgxc_node; node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred |   node_id
.....
pgxc=# create table user_info_hash(id int primary key,firstname text,lastname text,info text) distribute by hash(id);
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "user_info_hash_pkey" for table "user_info_hash"
CREATE TABLE
....
PGXC$ deploy node4
PGXC$ add datanode master datanode4
ERROR: please specify the host for the datanode masetr
PGXC$ add datanode master datanode4 node4 20008 /opt/pgxc/nodes/dn_master
ERROR: sorry found some inconflicts in datanode master configuration.

add command syntax will be found at http://postgres-xc.sourceforge.net/docs/1_1/pgxc-ctl.html.   Please take a look at the section F.32.12.

--------------------------------------------------------------


2.
and i found when the dump file contain table with

CREATE TABLE user_info_hash (
    id integer NOT NULL,
    firstname text,
    lastname text,
    info text
)
DISTRIBUTE BY HASH (id)
TO NODE (datanode1,datanode2,datanode3);

is alway failed to add new coordinator
but with this

CREATE TABLE user_id (
    id integer NOT NULL
)
DISTRIBUTE BY HASH (id)
TO NODE (dn2,dn1,dn3);

it's sucessed to add the new coordinator to the cluster

No.   To add a coordinator, you should have correct resource in "add" command.   The error message above indicates there are some conflict in the resource you specified with another node.   You must specify unique port number and work directory within the server.

To add a coordinator, you should issue the following command:

PGXC$ add coordinator master name host port pooler dir

When you issue this command and it is successful, your pgxc_ctl.conf will have additional line with this additional node.   Please take a look at it.  If it fails and you'd like to  restore to the previous configuration, you can edit pgxc_ctl.conf by yourself.   In this point, pgxc_ctl is still "primitive".

Regards;
---
Koichi Suzuki


於 7/10/2013 18:18, Koichi Suzuki 提到:
I found your configuration, in terms of owner and user, is the same as my demonstration scenario.   Then what you should do is:

1. Log in to your operating system as pgxcUser,
2. Initialize Postgres-XC cluster with "init all" command of pgxc_ctl,

Then the database user $pgxcOwner should have been created and you don't have to worry about it.   When you add a coordinator, simply issue "add coordinator master" command.   It should work.  If you have any other issue, please let me know.

My pgxc_ctl demonstration scenario will be found at https://sourceforge.net/apps/mediawiki/postgres-xc/index.php?title=PGOpen2013_Postgres_Open_2013

The configuration is with full slaves, which you can disable.   The demo adds a datanode, not a coordinator.   I believe there's no significant differences.

Regards;

---
Koichi Suzuki


2013/10/7 Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>>
I understood the situation.   Could you let me know if pgxc is not the operating system user name you are using?  If so, I will run the test with this situation and see what is going to happen.   Maybe a bug (not Postgres-XC core, but pgxc_ctl).   If pgxc is the operating system user you are using, then there could be another cause.  Also, please let me know your setting of pgxcUser in pgxc_ctl.conf?

Best;

---
Koichi Suzuki


2013/10/7 Julian <jul...@gm...<mailto:jul...@gm...>>
Sorry, i found i have already config pgxcOwner=pgxc in pxc_ctl.conf.

And i forgot to mention there's something strange about to add new coordinator,
i can’t add coord4 by "add coordinator master coord4 node4 20004 20010 /opt/pgxc/nodes/coord”

it will show me
ERROR: sorry found some inconflicts in coordinator master configuration.PGXC$

unless i add coord4 to pgxc_ctl.conf and remove it at PGXC shell
PGXC$ remove coordinator master coord4
ERROR:  PGXC Node coord4: object not defined
ERROR:  PGXC Node coord4: object not defined
ERROR:  PGXC Node coord4: object not defined

Is it normal ?

attach file is my pgxc config

Julian
使用 Sparrow<http://www.sparrowmailapp.com/?sig> 發信


On 2013年10月7日Monday at 下午5:31, Koichi Suzuki wrote:

Okay.   I'm wondering why internal pg_restore complains pgxc already exists..    If you do not specify pgxc in pgxc_ctl and created it manually, newly-craeted node should not contain the role pgxc, though we have different massage.

Could you let me know if you have any idea?

Best;

---
Koichi Suzuki


2013/10/7 Julian <jul...@gm...<mailto:jul...@gm...>>
I did not specify pgxc as a database owner, but i create user pgxc to do these job.

--
Julian
使用 Sparrow<http://www.sparrowmailapp.com/?sig> 發信


On 2013年10月7日Monday at 下午4:44, Koichi Suzuki wrote:

Did you specify pgxc as a database owner






------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general