postgres-xc-general Mailing List for Postgres-XC

Ouch!   This bash (not easy to debug) will be replaced with C version in 1.1 or later.

Anyway, I will commit this change to github.

Thanks.
---
Koichi Suzuki



On 2013/04/15, at 22:48, Theodotos Andreou <th...@ub...> wrote:

> I found another one on line 4566:
> 
> if [ "$1" == "gtm" ] || [ "$1" == "$gtmName"]; then
> 
> t needs a space before ']'
> 
> 	if [ "$1" =="gtm" ] || ["$1" =="$gtmName" ]; then
> 
> 
> 
> On 04/15/2013 04:09 AM, 鈴木 幸市 wrote:
>> Oh, it's a bug.    log_eco should be log_echo, which is pgxc_ctl internal function.
>> 
>> Verry sorry for the mistake.  I committed the fix to git repository.
>> 
>> Please checkin the latest one.
>> 
>> Thank you very much;
>> ---
>> Koichi Suzuki
>> 
>> 
>> On 2013/04/13, at 8:06, Theodotos Andreou <th...@ub...> wrote:
>> 
>>> Hello to all,
>>> 
>>> When I run the following command I get
>>> $ pgxc_ctl start all
>>> /var/lib/postgres-xc/bin/pgxc_ctl: line 4003: log_eco: command not found
>>> 
>>> Is this an actual command or a function inside pgxc_ctl?
>>> 
>>> ------------------------------------------------------------------------------
>>> Precog is a next-generation analytics platform capable of advanced
>>> analytics on semi-structured data. The platform includes APIs for building
>>> apps and a phenomenal toolset for data science. Developers can use
>>> our toolset for easy data analysis & visualization. Get a free account!
>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>> 
> 
> 

If you don't have a PRIMARY NODE, be careful about updating replicated tables.   You have to control the order of each update to each node exactly the same to make things consistent.

Best;
---
Koichi Suzuki



On 2013/04/16, at 0:57, Paul Jones <pb...@cm...> wrote:

> I will send this when we are able to reconfigure our cluster.  RIght now, we don't have PRIMARY enabled.
> 
> PJ
> 
> 
> 
>> ________________________________
>> From: Koichi Suzuki <ko...@in...>
>> To: Paul Jones <pb...@cm...> 
>> Cc: "pos...@li..." <pos...@li...> 
>> Sent: Thursday, April 11, 2013 10:31 PM
>> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>> 
>> 
>> I did the same thing with 4CO and 4DN, one of them as primary node, and found all the datanode contains identical set or rows.
>> 
>> Could I have pgxc_node in each coordinator?
>> 
>> Regards;
>> ---
>> Koichi Suzuki
>> 
>> On Mon, 8 Apr 2013 12:58:52 -0700 (PDT)
>> Paul Jones <pb...@cm...> wrote:
>> 
>>> Thanks for everyone's explanation of PRIMARY.  It is much clearer now.
>>> 
>>> I believe, then, that we may have uncovered a bug in PRIMARY.
>>> 
>>> We created a new cluster (8 nodes, 3 coordinators), but with only one PRIMARY datanode.  The PRIMARY was
>>> declared the same on all 3 coordinators.
>>> 
>>> A table that was declared DISTRIBUTE BY REPLICATION and loaded by \copy  did not have any rows
>>> present on the PRIMARY!  Further, other tables with FK's referring to this table had RI failures when they were loaded,
>>> even though there were complete copies of the table in all the other non-PRIMARY datanodes.
>>> 
>>> When we remade the cluster without any PRIMARY, this table loaded into all datanodes and there were no RI failures.
>>> 
>>> Is this a bug?   Unfortunately I won't be able to experiment with this until we finish executing our test plan, perhaps
>>> a few days.
>>> 
>>> PJ
>>> 
>>> 
>>> 
>>> 
>>>> ________________________________
>>>> From: Koichi Suzuki <koi...@gm...>
>>>> To: Andrei Martsinchyk <and...@gm...> 
>>>> Cc: "pos...@li..." <pos...@li...> 
>>>> Sent: Sunday, April 7, 2013 9:23 AM
>>>> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>>>> 
>>>> 
>>>> Primary node is useful to maintain replicated table in a consistent status in all the datanode.  All the writes to a replicated table goes first to the primary node so all the conflicts are resolved here and prevents conflict writes in other datanodes.   In this sense, this may prvent some deadlocks but it does not remove the chance of deadlocks in general sense.
>>>> 
>>>> On the othe hand, preferred node (datanode again) saves inter-server communication to read replciated table.   It does not work to maintain replicated table consistensy but helps to gain some performance.
>>>> 
>>>> Regards;
>>>> ---
>>>> Koichi Suzuki
>>>> 
>>>> 
>>>> 
>>>> ----------
>>>> Koichi Suzuki
>>>> 
>>>> 
>>>> 2013/4/7 Andrei Martsinchyk <and...@gm...>
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 2013/4/7 Jov <am...@am...>
>>>>> 
>>>>> datanode use primary node to solve replication table write,it is good,but how coordinator solve the dead lock problem? the coordinator nodes replication all globle catalog tables across coords,they are some kind replication table. 
>>>>>> 
>>>>>> 
>>>>>> eg.
>>>>>> 
>>>>>> client 1 run alter table tb on coord node A,it will lock local catalog data on A,and wait other coord node B.
>>>>>> client 2 run alter table tb on coord node B,it will lock local catalog data on B,and wait other coord node A.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> so how XC handle this dead lock?
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> XC does  not handle this, it will be deadlocked.
>>>>> Fortunately, chance of concurrent DDL much less then chance of concurrent replicated update.
>>>>> 
>>>>> 
>>>>>  
>>>>> 
>>>>>> 
>>>>>> 2013/4/6 Andrei Martsinchyk <and...@gm...>
>>>>>> 
>>>>>> PRIMARY was introduced to avoid distributed deadlocks when updating replicated tables.
>>>>>>> To better understand the problem, imagine two transactions A and B are updating the same tuple in replicated concurrently.
>>>>>>> Normally coordinator sends the same commands to all datanodes at the same time, and if on some node A updates the tuple first, B will be waiting for the end of transaction A. If on other node B wins the race, both transactions will be waiting for each other. It is hard to detect such deadlock, the information about locks is not sent across network.
>>>>>>> But it is possible to avoid. The idea is to set one datanode as a primary, and execute distributed update on primary node first, and go on with other nodes only if operation succeeds on primary.
>>>>>>> With this approach row lock on primary would stop concurrent transactions from taking row locks on other nodes that could prevent command completion.
>>>>>>> So, to have this stuff working properly you should 
>>>>>>> 1) set only one datanode as a primary;
>>>>>>> 2) if you have multiple coordinators, the same datanode should be set as a primary on each of them.
>>>>>>> Obvious drawback of the approach is double execution time of replicated updates.
>>>>>>> Note: "update" means any write access.  
>>>>>>> Hope this answers 1)-3)
>>>>>>> Regarding 4), the query
>>>>>>> 
>>>>>>> 
>>>>>>> select nodeoids from pg_class, pgxc_class where pg_class.oid = pcrelid and relname = '<your table name>';
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> returns the list of nodes, where the specified table is distributed on. I guess there are 7 of them.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 2013/4/5 Paul Jones <pb...@cm...>
>>>>>>> 
>>>>>>> 
>>>>>>>> We are experimenting with an 8-datanode, 3-coordinator cluster and we
>>>>>>>> have some questions about the use of PRIMARY and a problem. 
>>>>>>>> 
>>>>>>>> The manual explains what PRIMARY means but does not provide much detail
>>>>>>>> about when you would use it or not use it.
>>>>>>>> 
>>>>>>>> 1) Can PRIMARY apply to coordinators and if so, when would you
>>>>>>>>    want it or not?
>>>>>>>> 
>>>>>>>> 2) When would you use PRIMARY for datanodes or not, and would you
>>>>>>>>    ever want more than one datanode to be a primary?
>>>>>>>> 
>>>>>>>> 3) Does a pgxc_node datanode entry on its own server have to be
>>>>>>>>    the FQDN server name or can it be 'localhost'?
>>>>>>>>   
>>>>>>>> 4) We have a table that is defined as DISTRIBUTE BY REPLICATION.
>>>>>>>>    It only loads on the first 7 nodes.  It will just not load on
>>>>>>>>    node 8.  There are a lot of FK references from other tables to it,
>>>>>>>>    but it itself only has a simple CHAR(11) PK, one constraint,
>>>>>>>>    and 3 indices.
>>>>>>>>   
>>>>>>>>    Has anyone seen anything like this before?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Paul Jones
>>>>>>>> 
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Minimize network downtime and maximize team effectiveness.
>>>>>>>> Reduce network management and security costs.Learn how to hire
>>>>>>>> the most talented Cisco Certified professionals. Visit the
>>>>>>>> Employer Resources Portal
>>>>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>>>>> _______________________________________________
>>>>>>>> Postgres-xc-general mailing list
>>>>>>>> Pos...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> -- 
>>>>>>> Andrei Martsinchyk
>>>>>>> 
>>>>>>> StormDB - http://www.stormdb.com
>>>>>>> The Database Cloud
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Minimize network downtime and maximize team effectiveness.
>>>>>>> Reduce network management and security costs.Learn how to hire
>>>>>>> the most talented Cisco Certified professionals. Visit the
>>>>>>> Employer Resources Portal
>>>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>>>> _______________________________________________
>>>>>>> Postgres-xc-general mailing list
>>>>>>> Pos...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> Jov
>>>>>> 
>>>>>> blog: http:amutu.com/blog
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> Andrei Martsinchyk
>>>>> 
>>>>> StormDB - http://www.stormdb.com
>>>>> The Database Cloud
>>>>> 
>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> Minimize network downtime and maximize team effectiveness.
>>>>> Reduce network management and security costs.Learn how to hire
>>>>> the most talented Cisco Certified professionals. Visit the
>>>>> Employer Resources Portal
>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>> _______________________________________________
>>>>> Postgres-xc-general mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>> 
>>>>> 
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Minimize network downtime and maximize team effectiveness.
>>>> Reduce network management and security costs.Learn how to hire 
>>>> the most talented Cisco Certified professionals. Visit the 
>>>> Employer Resources Portal
>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>> _______________________________________________
>>>> Postgres-xc-general mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>> 
>>>> 
>>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Minimize network downtime and maximize team effectiveness.
>>> Reduce network management and security costs.Learn how to hire 
>>> the most talented Cisco Certified professionals. Visit the 
>>> Employer Resources Portal
>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>> 
>> 
>> 
>> 
>