Menu
▾
▴
postgres-xc-general — General info and messages
You can subscribe to this list here.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
(6) |
Sep
|
Oct
(19) |
Nov
(1) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(12) |
Feb
(1) |
Mar
(4) |
Apr
(4) |
May
(32) |
Jun
(12) |
Jul
(11) |
Aug
(1) |
Sep
(6) |
Oct
(3) |
Nov
|
Dec
(10) |
| 2012 |
Jan
(11) |
Feb
(1) |
Mar
(3) |
Apr
(25) |
May
(53) |
Jun
(38) |
Jul
(103) |
Aug
(54) |
Sep
(31) |
Oct
(66) |
Nov
(77) |
Dec
(20) |
| 2013 |
Jan
(91) |
Feb
(86) |
Mar
(103) |
Apr
(107) |
May
(25) |
Jun
(37) |
Jul
(17) |
Aug
(59) |
Sep
(38) |
Oct
(78) |
Nov
(29) |
Dec
(15) |
| 2014 |
Jan
(23) |
Feb
(82) |
Mar
(118) |
Apr
(101) |
May
(103) |
Jun
(45) |
Jul
(6) |
Aug
(10) |
Sep
|
Oct
(32) |
Nov
|
Dec
(9) |
| 2015 |
Jan
(3) |
Feb
(5) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
(9) |
Aug
(4) |
Sep
(3) |
Oct
|
Nov
|
Dec
|
| 2016 |
Jan
(3) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2018 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(4) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: 鈴木 幸市 <ko...@in...> - 2013-05-13 07:03:47
|
Katsumata-san; Unfortunately it was not included in 1.0.2 to save development resources. I'd like to know how serious it is for 1.0, not for 1.1. Depending upon severity, I may be able to re-prioritize my resource. Regards; --- Koichi Suzuki On 2013/05/13, at 14:39, Tomonari Katsumata <kat...@po...> wrote: > Hi, > > I'm thinking about how to recover the system after GTM crash. > > After GTM stopped, I just start it again. > It works fine with Postgres-XC 1.0.2, and I could do "CREATE TABLE", > "INSERT" some data and "SELECT" them. > > But with Postgres-XC 1.0.1, It doesn't work same. > I have to make GTM_proxy reconnect to GTM manually(gtm_ctl reconnect). > In some cases, I have to restart GTM_proxy. > > I want to know 2 things about this. > 1. I think this new feature(*) is included in Postgres-XC 1.0.2, right? > (*) reconnect doesn't be required with restarting GTM. > 2. May I think this feature will never change for future releases? > (I've confirmed the feature is alive in Postgres-XC 1.1dev !) > > ============= > Environment > ============= > 2 servers are there. > components are destributed like bellow. > > server1 - GTM, GTM_proxy1, Coordinator1, Datanode1 > server2 - GTM_proxy2, Coordinator2, Datanode2 > > ========== > reproduce > ========== > 1. stop GTM manually (kill -9 <PID of GTM>) > 2. start GTM again manually. > > > regards, > ----------------- > NTT Software Corporation > Tomonari Katsumata > > > > ------------------------------------------------------------------------------ > Learn Graph Databases - Download FREE O'Reilly Book > "Graph Databases" is the definitive new guide to graph databases and > their applications. This 200-page book is written by three acclaimed > leaders in the field. The early access version is available now. > Download your free book today! http://p.sf.net/sfu/neotech_d2d_may > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > |
|
From: Tomonari K. <kat...@po...> - 2013-05-13 05:43:22
|
Hi, I'm thinking about how to recover the system after GTM crash. After GTM stopped, I just start it again. It works fine with Postgres-XC 1.0.2, and I could do "CREATE TABLE", "INSERT" some data and "SELECT" them. But with Postgres-XC 1.0.1, It doesn't work same. I have to make GTM_proxy reconnect to GTM manually(gtm_ctl reconnect). In some cases, I have to restart GTM_proxy. I want to know 2 things about this. 1. I think this new feature(*) is included in Postgres-XC 1.0.2, right? (*) reconnect doesn't be required with restarting GTM. 2. May I think this feature will never change for future releases? (I've confirmed the feature is alive in Postgres-XC 1.1dev !) ============= Environment ============= 2 servers are there. components are destributed like bellow. server1 - GTM, GTM_proxy1, Coordinator1, Datanode1 server2 - GTM_proxy2, Coordinator2, Datanode2 ========== reproduce ========== 1. stop GTM manually (kill -9 <PID of GTM>) 2. start GTM again manually. regards, ----------------- NTT Software Corporation Tomonari Katsumata |
|
From: Adam D. <ada...@sm...> - 2013-05-12 16:12:20
|
# ---------------------- # GTM configuration file # ---------------------- # # This file must be placed on gtm working directory # specified by -D command line option of gtm or gtm_ctl. The # configuration file name must be "gtm.conf" # # # This file consists of lines of the form # # name = value # # (The "=" is optional.) Whitespace may be used. Comments are # introduced with "#" anywhere on a line. The complete list of # parameter names and allowed values can be found in the # Postgres-XC documentation. # # The commented-out settings shown in this file represent the default # values. # # Re-commenting a setting is NOT sufficient to revert it to the default # value. # # You need to restart the server. #------------------------------------------------------------------------------ # GENERAL PARAMETERS #------------------------------------------------------------------------------ nodename = 'GTM1' # Specifies the node name. # (changes requires restart) listen_addresses = '*' # Listen addresses of this GTM. # (changes requires restart) port = 20001 # Port number of this GTM. # (changes requires restart) #startup = ACT # Start mode. ACT/STANDBY. #------------------------------------------------------------------------------ # GTM STANDBY PARAMETERS #------------------------------------------------------------------------------ #Those parameters are effective when GTM is activated as a standby server #active_host = '' # Listen address of active GTM. # (changes requires restart) #active_port = # Port number of active GTM. # (changes requires restart) #--------------------------------------- # OTHER OPTIONS #--------------------------------------- #keepalives_idle = 0 # Keepalives_idle parameter. #keepalives_interval = 0 # Keepalives_interval parameter. #keepalives_count = 0 # Keepalives_count internal parameter. #log_file = 'gtm.log' # Log file name #log_min_messages = DEBUG5 # log_min_messages. Default WARNING. # Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3, # DEBUG2, DEBUG1, INFO, NOTICE, WARNING, # ERROR, LOG, FATAL, PANIC #synchronous_backup = off # If backup to standby is synchronous |
|
From: 鈴木 幸市 <ko...@in...> - 2013-05-09 05:53:02
|
Thanx. --- Koichi Suzuki On 2013/05/08, at 14:46, "Masaki HISADA" <his...@la...> wrote: > Hi, Koichi-san, > >> Could you open bug tickets in sourceforge and assign them to me? > > ID: 3612883 > > Rgds, > > Hisada > >> -----Original Message----- >> From: 鈴木 幸市 [mailto:ko...@in...] >> Sent: Wednesday, May 01, 2013 9:51 AM >> To: Masaki HISADA >> Cc: pos...@li... >> Subject: Re: [Postgres-xc-general] GTM standby behaivor when cannot > connect >> with GTM active >> >> I have also observed connection error message in gtm slave although > there're >> no network failure and gtm master is running. We should look into it. >> >> Also failure to stop gtm standby could be a bug. >> >> Could you open bug tickets in sourceforge and assign them to me? >> --- >> Koichi Suzuki >> >> >> >> On 2013/04/30, at 16:04, Masaki HISADA <his...@la...> > wrote: >> >>> Hi Kohichi-san, >>> >>> >>>> At present, gtm_standby does not use tcp_keepalives. Gtm_standby > just >>> wait >>>> for GTM to send backups and does not detect network fault until the >>>> kernel timeout (usually 60min or so). >>> >>> From gtm.log, it seems that gtm_standby detecting connection failure >>> with GTM about 1 minute after starting up gtm_standby. >>> >>>>> $ less gtm/data/gtm.log >>>>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the >>>>> GTM active on 192.168.2.1:19000... >>>>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 >>>>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to >>>>> establish a connection to active-GTM. >>>>> LOCATION: main, main.c:553 >>> >>>> Because it continues to wait for incoming backup commands, status >>>> will >>> return >>>> that it's running. If Gtm slave starts during the network fault, it >>> cannot >>>> get initial response from master and it will fail. >>> >>> I have tried stop immediate in this situation but could not stop >>> gtm_standby but cannot stop gtm_standby .... >>> Although gtm_standby is waiting for GTM backup, I guess gtm_standby >>> needs to be shutting down when immediate shutdown. >>> >>>> (4) Stop gtm_stadby >>> >>> $ gtm_ctl -D gtm/data/ -Z gtm start >>> server starting >>> $ gtm_ctl -D gtm/data/ -Z gtm stop -m immediate waiting for server to >>> shut down................ >>> ............................................... failed >>> gtm_ctl: server does not shut down >>> >>> Regards, >>> >>> Mark >>> >>> >>>> Regards; >>>> --- >>>> Koichi Suzuki >>>> >>>> >>>> >>>> On 2013/04/26, at 16:00, Masaki HISADA <his...@la...> >>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have been testing pgxc behavior when network between gtm and >>>>> gtm_standy has down. >>>>> >>>>> I have started up gtm_standby and linked down bw gtm and gtm_standby >>>>> by using iptables. I expected that gtm_standby would fail with error >>>>> because gtm_standby cannot establish synchronization with gtm active. >>>>> >>>>> Problem : >>>>> 1. gtm_standby started without error response although gtm_standby >>>>> has failed to establish connection with gtm active from gtm.log. Is >>>>> gtm alive or not? >>>>> 2. Assuming gtm alive I have tried immediate stop to gtm_standby but >>> failed. >>>>> 3. Checking at gtm status, it is stopped. >>>>> >>>>> Question >>>>> (1) Am I testing right? >>>>> (2) Is this expected behavior for gtm_standby when it cannot >>>>> establish connection with gtm? >>>>> >>>>> Test procedure as follows; >>>>> >>>>> (1) Start up GTM active at Server A >>>>> $ gtm_ctl -D gtm/data/ -Z gtm status >>>>> >>>>> (2) Network down (simulated with setting iptables) $ iptables -I >>>>> INPUT -i bond1 -j DROP $ iptables -I OUTPUT -o bond1 -j DROP >>>>> >>>>> (3) Start up GTM standby at Server B $ gtm_ctl -D gtm/data/ -Z gtm >>>>> start server starting >>>>> >>>>> (4) Stop gtm_stadby >>>>> waiting for server to shut down................ >>>>> ............................................... failed >>>>> gtm_ctl: server does not shut down >>>>> >>>>> (5) Check status >>>>> $ gtm_ctl -D gtm/data/ -Z gtm status >>>>> gtm_ctl: no server running >>>>> $ ps aux | grep gtm | grep -v grep >>>>> # No output >>>>> $ ll gtm/data/ >>>>> 合計 12 >>>>> -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf >>>>> -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log >>>>> -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid >>>>> >>>>> $ less gtm/data/gtm.log >>>>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the >>>>> GTM active on 192.168.2.1:19000... >>>>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 >>>>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to >>>>> establish a connection to active-GTM. >>>>> LOCATION: main, main.c:553 >>>>> >>>>> pos...@li... >>>>> >>>>> Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) >>>>> >>>>> Rgds, >>>>> >>>>> Mark >>>>> >>>>> >>>>> -------------------------------------------------------------------- >>>>> -- >>>>> -------- Try New Relic Now & We'll Send You this Cool Shirt New >>>>> Relic is the only SaaS-based application performance monitoring >>>>> service that delivers powerful full stack analytics. Optimize and >>>>> monitor your browser, app, & servers with just a few lines of code. >>>>> Try New Relic and get this awesome Nerd Life shirt! >>>>> http://p.sf.net/sfu/newrelic_d2d_apr >>>>> _______________________________________________ >>>>> Postgres-xc-general mailing list >>>>> Pos...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> ---- >>>> ----- >>>> Introducing AppDynamics Lite, a free troubleshooting tool for >>>> Java/.NET >>> Get >>>> 100% visibility into your production application - at no cost. >>>> Code-level diagnostics for performance bottlenecks with <2% overhead >>> Download >>>> for free and get started troubleshooting in minutes. >>>> http://p.sf.net/sfu/appdyn_d2d_ap1 >>>> _______________________________________________ >>>> Postgres-xc-general mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >>> >> >> >> ------------------------------------------------------------------------- >> ----- >> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET > Get >> 100% visibility into your production application - at no cost. >> Code-level diagnostics for performance bottlenecks with <2% overhead > Download >> for free and get started troubleshooting in minutes. >> http://p.sf.net/sfu/appdyn_d2d_ap1 >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > |
|
From: Masaki H. <his...@la...> - 2013-05-08 05:45:59
|
Hi, Koichi-san, > Could you open bug tickets in sourceforge and assign them to me? ID: 3612883 Rgds, Hisada > -----Original Message----- > From: 鈴木 幸市 [mailto:ko...@in...] > Sent: Wednesday, May 01, 2013 9:51 AM > To: Masaki HISADA > Cc: pos...@li... > Subject: Re: [Postgres-xc-general] GTM standby behaivor when cannot connect > with GTM active > > I have also observed connection error message in gtm slave although there're > no network failure and gtm master is running. We should look into it. > > Also failure to stop gtm standby could be a bug. > > Could you open bug tickets in sourceforge and assign them to me? > --- > Koichi Suzuki > > > > On 2013/04/30, at 16:04, Masaki HISADA <his...@la...> wrote: > > > Hi Kohichi-san, > > > > > >> At present, gtm_standby does not use tcp_keepalives. Gtm_standby just > > wait > >> for GTM to send backups and does not detect network fault until the > >> kernel timeout (usually 60min or so). > > > > From gtm.log, it seems that gtm_standby detecting connection failure > > with GTM about 1 minute after starting up gtm_standby. > > > >>> $ less gtm/data/gtm.log > >>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the > >>> GTM active on 192.168.2.1:19000... > >>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 > >>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to > >>> establish a connection to active-GTM. > >>> LOCATION: main, main.c:553 > > > >> Because it continues to wait for incoming backup commands, status > >> will > > return > >> that it's running. If Gtm slave starts during the network fault, it > > cannot > >> get initial response from master and it will fail. > > > > I have tried stop immediate in this situation but could not stop > > gtm_standby but cannot stop gtm_standby .... > > Although gtm_standby is waiting for GTM backup, I guess gtm_standby > > needs to be shutting down when immediate shutdown. > > > >> (4) Stop gtm_stadby > > > > $ gtm_ctl -D gtm/data/ -Z gtm start > > server starting > > $ gtm_ctl -D gtm/data/ -Z gtm stop -m immediate waiting for server to > > shut down................ > > ............................................... failed > > gtm_ctl: server does not shut down > > > > Regards, > > > > Mark > > > > > >> Regards; > >> --- > >> Koichi Suzuki > >> > >> > >> > >> On 2013/04/26, at 16:00, Masaki HISADA <his...@la...> > > wrote: > >> > >>> Hi, > >>> > >>> I have been testing pgxc behavior when network between gtm and > >>> gtm_standy has down. > >>> > >>> I have started up gtm_standby and linked down bw gtm and gtm_standby > >>> by using iptables. I expected that gtm_standby would fail with error > >>> because gtm_standby cannot establish synchronization with gtm active. > >>> > >>> Problem : > >>> 1. gtm_standby started without error response although gtm_standby > >>> has failed to establish connection with gtm active from gtm.log. Is > >>> gtm alive or not? > >>> 2. Assuming gtm alive I have tried immediate stop to gtm_standby but > > failed. > >>> 3. Checking at gtm status, it is stopped. > >>> > >>> Question > >>> (1) Am I testing right? > >>> (2) Is this expected behavior for gtm_standby when it cannot > >>> establish connection with gtm? > >>> > >>> Test procedure as follows; > >>> > >>> (1) Start up GTM active at Server A > >>> $ gtm_ctl -D gtm/data/ -Z gtm status > >>> > >>> (2) Network down (simulated with setting iptables) $ iptables -I > >>> INPUT -i bond1 -j DROP $ iptables -I OUTPUT -o bond1 -j DROP > >>> > >>> (3) Start up GTM standby at Server B $ gtm_ctl -D gtm/data/ -Z gtm > >>> start server starting > >>> > >>> (4) Stop gtm_stadby > >>> waiting for server to shut down................ > >>> ............................................... failed > >>> gtm_ctl: server does not shut down > >>> > >>> (5) Check status > >>> $ gtm_ctl -D gtm/data/ -Z gtm status > >>> gtm_ctl: no server running > >>> $ ps aux | grep gtm | grep -v grep > >>> # No output > >>> $ ll gtm/data/ > >>> 合計 12 > >>> -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf > >>> -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log > >>> -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid > >>> > >>> $ less gtm/data/gtm.log > >>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the > >>> GTM active on 192.168.2.1:19000... > >>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 > >>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to > >>> establish a connection to active-GTM. > >>> LOCATION: main, main.c:553 > >>> > >>> pos...@li... > >>> > >>> Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) > >>> > >>> Rgds, > >>> > >>> Mark > >>> > >>> > >>> -------------------------------------------------------------------- > >>> -- > >>> -------- Try New Relic Now & We'll Send You this Cool Shirt New > >>> Relic is the only SaaS-based application performance monitoring > >>> service that delivers powerful full stack analytics. Optimize and > >>> monitor your browser, app, & servers with just a few lines of code. > >>> Try New Relic and get this awesome Nerd Life shirt! > >>> http://p.sf.net/sfu/newrelic_d2d_apr > >>> _______________________________________________ > >>> Postgres-xc-general mailing list > >>> Pos...@li... > >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > >>> > >> > >> > >> --------------------------------------------------------------------- > >> ---- > >> ----- > >> Introducing AppDynamics Lite, a free troubleshooting tool for > >> Java/.NET > > Get > >> 100% visibility into your production application - at no cost. > >> Code-level diagnostics for performance bottlenecks with <2% overhead > > Download > >> for free and get started troubleshooting in minutes. > >> http://p.sf.net/sfu/appdyn_d2d_ap1 > >> _______________________________________________ > >> Postgres-xc-general mailing list > >> Pos...@li... > >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > > > > > ------------------------------------------------------------------------- > ----- > Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET Get > 100% visibility into your production application - at no cost. > Code-level diagnostics for performance bottlenecks with <2% overhead Download > for free and get started troubleshooting in minutes. > http://p.sf.net/sfu/appdyn_d2d_ap1 > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general |
|
From: 鈴木 幸市 <ko...@in...> - 2013-05-02 05:28:59
|
Thanks Tim for the info. I visited the site and took a look at the patch they provide for postgres-xc core. I'm afraid the patch may not for general use because they ignore some kind of errors. Original code is to abort the current transaction but modified patch requires to ignore it. It's helpful if StormDB people or other people review and test to show this patch works generally and doesn't bring any issues. Maybe PostGis installation issues Postgres-XC SQL statements and they complain that there're no datanode. The patch looks like a workaround for them. The installation and configuration procedure assumes that all the components run on the same server, using identical binary. If you have more than one server, installation may need something more. If it is agreed that the patch is for general use, it can be included in XC core. Regards; --- Koichi Suzuki On 2013/05/01, at 12:17, Tim Keitt <tk...@ut...> wrote: > Here's the link to postgis-postgres-xc integration: > > http://goo.gl/JHHjH > > THK > > -- > http://www.keittlab.org/ > ------------------------------------------------------------------------------ > Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET > Get 100% visibility into your production application - at no cost. > Code-level diagnostics for performance bottlenecks with <2% overhead > Download for free and get started troubleshooting in minutes. > http://p.sf.net/sfu/appdyn_d2d_ap1_______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general |
|
From: Tim K. <tk...@ut...> - 2013-05-01 03:17:57
|
Here's the link to postgis-postgres-xc integration: http://goo.gl/JHHjH THK -- http://www.keittlab.org/ |
|
From: 鈴木 幸市 <ko...@in...> - 2013-05-01 00:51:36
|
I have also observed connection error message in gtm slave although there're no network failure and gtm master is running. We should look into it. Also failure to stop gtm standby could be a bug. Could you open bug tickets in sourceforge and assign them to me? --- Koichi Suzuki On 2013/04/30, at 16:04, Masaki HISADA <his...@la...> wrote: > Hi Kohichi-san, > > >> At present, gtm_standby does not use tcp_keepalives. Gtm_standby just > wait >> for GTM to send backups and does not detect network fault until the kernel >> timeout (usually 60min or so). > > From gtm.log, it seems that gtm_standby detecting connection failure with > GTM about 1 minute after starting up gtm_standby. > >>> $ less gtm/data/gtm.log >>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the >>> GTM active on 192.168.2.1:19000... >>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 >>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to >>> establish a connection to active-GTM. >>> LOCATION: main, main.c:553 > >> Because it continues to wait for incoming backup commands, status will > return >> that it's running. If Gtm slave starts during the network fault, it > cannot >> get initial response from master and it will fail. > > I have tried stop immediate in this situation but could not stop gtm_standby > but cannot stop gtm_standby .... > Although gtm_standby is waiting for GTM backup, I guess gtm_standby needs to > be shutting down when immediate shutdown. > >> (4) Stop gtm_stadby > > $ gtm_ctl -D gtm/data/ -Z gtm start > server starting > $ gtm_ctl -D gtm/data/ -Z gtm stop -m immediate > waiting for server to shut down................ > ............................................... failed > gtm_ctl: server does not shut down > > Regards, > > Mark > > >> Regards; >> --- >> Koichi Suzuki >> >> >> >> On 2013/04/26, at 16:00, Masaki HISADA <his...@la...> > wrote: >> >>> Hi, >>> >>> I have been testing pgxc behavior when network between gtm and >>> gtm_standy has down. >>> >>> I have started up gtm_standby and linked down bw gtm and gtm_standby >>> by using iptables. I expected that gtm_standby would fail with error >>> because gtm_standby cannot establish synchronization with gtm active. >>> >>> Problem : >>> 1. gtm_standby started without error response although gtm_standby has >>> failed to establish connection with gtm active from gtm.log. Is gtm >>> alive or not? >>> 2. Assuming gtm alive I have tried immediate stop to gtm_standby but > failed. >>> 3. Checking at gtm status, it is stopped. >>> >>> Question >>> (1) Am I testing right? >>> (2) Is this expected behavior for gtm_standby when it cannot establish >>> connection with gtm? >>> >>> Test procedure as follows; >>> >>> (1) Start up GTM active at Server A >>> $ gtm_ctl -D gtm/data/ -Z gtm status >>> >>> (2) Network down (simulated with setting iptables) $ iptables -I INPUT >>> -i bond1 -j DROP $ iptables -I OUTPUT -o bond1 -j DROP >>> >>> (3) Start up GTM standby at Server B >>> $ gtm_ctl -D gtm/data/ -Z gtm start >>> server starting >>> >>> (4) Stop gtm_stadby >>> waiting for server to shut down................ >>> ............................................... failed >>> gtm_ctl: server does not shut down >>> >>> (5) Check status >>> $ gtm_ctl -D gtm/data/ -Z gtm status >>> gtm_ctl: no server running >>> $ ps aux | grep gtm | grep -v grep >>> # No output >>> $ ll gtm/data/ >>> 合計 12 >>> -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf >>> -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log >>> -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid >>> >>> $ less gtm/data/gtm.log >>> 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the >>> GTM active on 192.168.2.1:19000... >>> LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 >>> 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to >>> establish a connection to active-GTM. >>> LOCATION: main, main.c:553 >>> >>> pos...@li... >>> >>> Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) >>> >>> Rgds, >>> >>> Mark >>> >>> >>> ---------------------------------------------------------------------- >>> -------- Try New Relic Now & We'll Send You this Cool Shirt New Relic >>> is the only SaaS-based application performance monitoring service that >>> delivers powerful full stack analytics. Optimize and monitor your >>> browser, app, & servers with just a few lines of code. Try New Relic >>> and get this awesome Nerd Life shirt! >>> http://p.sf.net/sfu/newrelic_d2d_apr >>> _______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >> >> >> ------------------------------------------------------------------------- >> ----- >> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET > Get >> 100% visibility into your production application - at no cost. >> Code-level diagnostics for performance bottlenecks with <2% overhead > Download >> for free and get started troubleshooting in minutes. >> http://p.sf.net/sfu/appdyn_d2d_ap1 >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > |
|
From: Masaki H. <his...@la...> - 2013-04-30 07:03:29
|
Hi Kohichi-san, > At present, gtm_standby does not use tcp_keepalives. Gtm_standby just wait > for GTM to send backups and does not detect network fault until the kernel > timeout (usually 60min or so). >From gtm.log, it seems that gtm_standby detecting connection failure with GTM about 1 minute after starting up gtm_standby. > > $ less gtm/data/gtm.log > > 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the > > GTM active on 192.168.2.1:19000... > > LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 > > 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to > > establish a connection to active-GTM. > > LOCATION: main, main.c:553 > Because it continues to wait for incoming backup commands, status will return > that it's running. If Gtm slave starts during the network fault, it cannot > get initial response from master and it will fail. I have tried stop immediate in this situation but could not stop gtm_standby but cannot stop gtm_standby .... Although gtm_standby is waiting for GTM backup, I guess gtm_standby needs to be shutting down when immediate shutdown. > (4) Stop gtm_stadby $ gtm_ctl -D gtm/data/ -Z gtm start server starting $ gtm_ctl -D gtm/data/ -Z gtm stop -m immediate waiting for server to shut down................ ............................................... failed gtm_ctl: server does not shut down Regards, Mark > Regards; > --- > Koichi Suzuki > > > > On 2013/04/26, at 16:00, Masaki HISADA <his...@la...> wrote: > > > Hi, > > > > I have been testing pgxc behavior when network between gtm and > > gtm_standy has down. > > > > I have started up gtm_standby and linked down bw gtm and gtm_standby > > by using iptables. I expected that gtm_standby would fail with error > > because gtm_standby cannot establish synchronization with gtm active. > > > > Problem : > > 1. gtm_standby started without error response although gtm_standby has > > failed to establish connection with gtm active from gtm.log. Is gtm > > alive or not? > > 2. Assuming gtm alive I have tried immediate stop to gtm_standby but failed. > > 3. Checking at gtm status, it is stopped. > > > > Question > > (1) Am I testing right? > > (2) Is this expected behavior for gtm_standby when it cannot establish > > connection with gtm? > > > > Test procedure as follows; > > > > (1) Start up GTM active at Server A > > $ gtm_ctl -D gtm/data/ -Z gtm status > > > > (2) Network down (simulated with setting iptables) $ iptables -I INPUT > > -i bond1 -j DROP $ iptables -I OUTPUT -o bond1 -j DROP > > > > (3) Start up GTM standby at Server B > > $ gtm_ctl -D gtm/data/ -Z gtm start > > server starting > > > > (4) Stop gtm_stadby > > waiting for server to shut down................ > > ............................................... failed > > gtm_ctl: server does not shut down > > > > (5) Check status > > $ gtm_ctl -D gtm/data/ -Z gtm status > > gtm_ctl: no server running > > $ ps aux | grep gtm | grep -v grep > > # No output > > $ ll gtm/data/ > > 合計 12 > > -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf > > -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log > > -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid > > > > $ less gtm/data/gtm.log > > 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the > > GTM active on 192.168.2.1:19000... > > LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 > > 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to > > establish a connection to active-GTM. > > LOCATION: main, main.c:553 > > > > pos...@li... > > > > Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) > > > > Rgds, > > > > Mark > > > > > > ---------------------------------------------------------------------- > > -------- Try New Relic Now & We'll Send You this Cool Shirt New Relic > > is the only SaaS-based application performance monitoring service that > > delivers powerful full stack analytics. Optimize and monitor your > > browser, app, & servers with just a few lines of code. Try New Relic > > and get this awesome Nerd Life shirt! > > http://p.sf.net/sfu/newrelic_d2d_apr > > _______________________________________________ > > Postgres-xc-general mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > > > ------------------------------------------------------------------------- > ----- > Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET Get > 100% visibility into your production application - at no cost. > Code-level diagnostics for performance bottlenecks with <2% overhead Download > for free and get started troubleshooting in minutes. > http://p.sf.net/sfu/appdyn_d2d_ap1 > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general |
|
From: Ashutosh B. <ash...@en...> - 2013-04-30 04:19:52
|
AFAIK, postgis is an extension on PostgreSQL. Postgres-XC supports extensions. There are other users who had asked similar question about postgis; you may want to search the archives. On Tue, Apr 30, 2013 at 12:25 AM, Tim Keitt <tk...@ut...> wrote: > We are heavy postgis users and often throughput bound with large datasets. > I had begun to despair of moving to a new platform when I noticed > postgres-xc. I have a couple of questions: > > 1) I might be in the position to acquire funds to build a postgres-xc > cluster. Any recommendations on ideal specs for a smallish postgres-xc > cluster of say 2-8 nodes? > > 2) Is default postgis integration planned? > > 3) Are there plans for an ubuntu repo? > > This looks like a great project. I especially like that it does not impose > a lot of new learning on the user (albeit perhaps for the admin). Despite > 18 years postgresql experience including back-end development and writing R > packages for querying postgresql, we really don't like to have to learn a > lot of new and specific syntax as our primary goal is our research. > > THK > > -- > http://www.keittlab.org/ > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Postgres Database Company |
|
From: 鈴木 幸市 <ko...@in...> - 2013-04-30 01:36:24
|
At present, gtm_standby does not use tcp_keepalives. Gtm_standby just wait for GTM to send backups and does not detect network fault until the kernel timeout (usually 60min or so). Because it continues to wait for incoming backup commands, status will return that it's running. If Gtm slave starts during the network fault, it cannot get initial response from master and it will fail. Regards; --- Koichi Suzuki On 2013/04/26, at 16:00, Masaki HISADA <his...@la...> wrote: > Hi, > > I have been testing pgxc behavior when network between gtm and gtm_standy > has down. > > I have started up gtm_standby and linked down bw gtm and gtm_standby by > using iptables. I expected that gtm_standby would fail with error because > gtm_standby cannot establish synchronization with gtm active. > > Problem : > 1. gtm_standby started without error response although gtm_standby has > failed to establish connection with gtm active from gtm.log. Is gtm alive or > not? > 2. Assuming gtm alive I have tried immediate stop to gtm_standby but failed. > 3. Checking at gtm status, it is stopped. > > Question > (1) Am I testing right? > (2) Is this expected behavior for gtm_standby when it cannot establish > connection with gtm? > > Test procedure as follows; > > (1) Start up GTM active at Server A > $ gtm_ctl -D gtm/data/ -Z gtm status > > (2) Network down (simulated with setting iptables) > $ iptables -I INPUT -i bond1 -j DROP > $ iptables -I OUTPUT -o bond1 -j DROP > > (3) Start up GTM standby at Server B > $ gtm_ctl -D gtm/data/ -Z gtm start > server starting > > (4) Stop gtm_stadby > waiting for server to shut down................ > ............................................... failed > gtm_ctl: server does not shut down > > (5) Check status > $ gtm_ctl -D gtm/data/ -Z gtm status > gtm_ctl: no server running > $ ps aux | grep gtm | grep -v grep > # No output > $ ll gtm/data/ > 合計 12 > -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf > -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log > -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid > > $ less gtm/data/gtm.log > 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the GTM > active on 192.168.2.1:19000... > LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 > 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to establish a > connection to active-GTM. > LOCATION: main, main.c:553 > > pos...@li... > > Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) > > Rgds, > > Mark > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > |
|
From: Tim K. <tk...@ut...> - 2013-04-29 18:55:50
|
We are heavy postgis users and often throughput bound with large datasets. I had begun to despair of moving to a new platform when I noticed postgres-xc. I have a couple of questions: 1) I might be in the position to acquire funds to build a postgres-xc cluster. Any recommendations on ideal specs for a smallish postgres-xc cluster of say 2-8 nodes? 2) Is default postgis integration planned? 3) Are there plans for an ubuntu repo? This looks like a great project. I especially like that it does not impose a lot of new learning on the user (albeit perhaps for the admin). Despite 18 years postgresql experience including back-end development and writing R packages for querying postgresql, we really don't like to have to learn a lot of new and specific syntax as our primary goal is our research. THK -- http://www.keittlab.org/ |
|
From: Abbas B. <abb...@en...> - 2013-04-27 01:02:45
|
Thanks for detailed bug report. SourceForge bug ID is (Artifact 3611989<https://sourceforge.net/tracker/?func=detail&aid=3611989&group_id=311227&atid=1310232> ) On Sat, Apr 27, 2013 at 5:03 AM, Michael Paquier <mic...@gm...>wrote: > > > > On Sat, Apr 27, 2013 at 12:44 AM, Paul Jones <pb...@cm...> wrote: > >> EXECUTE DIRECT ON data_east03 'SELECT count(1) FROM foo'; >> >> count >> ---------- >> 1 >> (1 row) >> >> >> EXECUTE DIRECT ON data_east02 'SELECT count(1) FROM foo'; >> >> count >> ------------ >> 100000 >> (1 row) >> >> I get the same result even if I properly shutdown and restart the cluster. >> > There is obviously a bug with COPY FROM and replicated tables, rows are > not fed to all the necessary nodes. > > -- > Michael > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- -- *Abbas* Architect Ph: 92.334.5100153 Skype ID: gabbasb www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/> * Follow us on Twitter* @EnterpriseDB Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community> |
|
From: Michael P. <mic...@gm...> - 2013-04-27 00:03:26
|
On Sat, Apr 27, 2013 at 12:44 AM, Paul Jones <pb...@cm...> wrote: > EXECUTE DIRECT ON data_east03 'SELECT count(1) FROM foo'; > > count > ---------- > 1 > (1 row) > > > EXECUTE DIRECT ON data_east02 'SELECT count(1) FROM foo'; > > count > ------------ > 100000 > (1 row) > > I get the same result even if I properly shutdown and restart the cluster. > There is obviously a bug with COPY FROM and replicated tables, rows are not fed to all the necessary nodes. -- Michael |
|
From: Paul J. <pb...@cm...> - 2013-04-26 15:44:21
|
EXECUTE DIRECT ON data_east03 'SELECT count(1) FROM foo';
count
----------
1
(1 row)
EXECUTE DIRECT ON data_east02 'SELECT count(1) FROM foo';
count
------------
100000
(1 row)
I get the same result even if I properly shutdown and restart the cluster.
PJ
>________________________________
> From: Nikhil Sontakke <ni...@st...>
>To: Paul Jones <pb...@cm...>
>Cc: "pos...@li..." <pos...@li...>
>Sent: Thursday, April 25, 2013 10:47 PM
>Subject: Re: [Postgres-xc-general] BUG in \copy ...was Re: Questions about PRIMARY and a problem
>
>
>
>Hi Paul,
>
>It's not recommended to log in directly into a datanode to run queries. All queries ought to be directed via coordinators to use sane XIDs.
>
>What does the following query return when run from the coordinator?
>
>
>EXECUTE DIRECT ON (data_east03) 'SELECT count(1) FROM foo';
>
>
>Regards,
>Nikhils
>
>
>
>On Fri, Apr 26, 2013 at 3:16 AM, Paul Jones <pb...@cm...> wrote:
>
>I think I have discovered the problem with PRIMARY and our loads. The PRIMARY data node does not load
>>data for tables that are DISTRIBUTE BY REPLICATION. We are on PGXC 1.0.2. Here is a way to
>>demonstrate the problem:
>>
>># Demonstrate PRIMARY node \copy bug.
>>
>># 3 Coordinators
>># 8 Datanodes
>>
>># Prepare some data:
>>$ awk 'BEGIN{for(i=1;i<=100000;i++) print i; }' > DATA
>>
>>$ psql -h HOST242 -U postgres
>>postgres=# select * from pgxc_node;
>> node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id
>>----------------+-----------+-----------+---------------------+----------------+------------------+-------------
>>coordw | C | 5432 | localhost | f | f | 670793242
>>coorde | C | 5432 | HOST225.example.com | f | f | 329164574
>>coordc | C | 5432 | HOST232.example.com | f | f | -1588937622
>>data_east01 | D | 25432 | HOST225.example.com | f | f | -2053435448
>>data_east02 | D | 25432 | HOST226.example.com | f | f | 94547764
>>data_east03 | D | 25432 | HOST230.example.com | t | f | 197970754
>>data_central01 | D | 25432 | HOST231.example.com | f | f | 124274836
>>data_central02 | D | 25432 | HOST232.example.com | f | f | 1002175669
>>data_central03 | D | 25432 | HOST238.example.com | f | f | -1150964881
>>data_west01 | D | 25432 | HOST239.example.com | f | f | 2129529735
>>data_west02 | D | 25432 | HOST242.example.com | f | f | -717656524
>>(11 rows)
>>
>>#################################################
>># If we create a table distributed by replication
>># and fill it with an insert, we see the same
>># row counts from a coordinator and the PRIMARY
>>#################################################
>>
>>psql -h HOST242 -U user
>>user=> create table foo (aaa int) distribute by replication;
>>CREATE TABLE
>>user=> insert into foo select generate_series(1,100000);
>>INSERT 0 100000
>>user=> select count(1) from foo;
>>count
>>--------
>>100000
>>(1 row)
>>user=> \q
>>
>>$ psql -p 25432 -h HOST230 -U user
>>psql (9.2.4, server 9.1.7)
>>
>>user=> select count(1) from foo;
>>count
>>--------
>>100000
>>(1 row)
>>
>>user=> \q
>>
>>###############################################
>># Now truncate the table and load it with \copy
>>###############################################
>>
>>$ psql -h HOST242 -U user
>>user=> truncate table foo;
>>TRUNCATE TABLE
>>user=>
>>user=> \copy foo from DATA
>>user=> select count(1) from foo;
>>count
>>--------
>>100000
>>(1 row)
>>
>>user=> \q
>>
>>#############################################
>># The count was correct from the coordinator,
>># but not in the PRIMARY node.
>>#############################################
>>
>>$ psql -p 25432 -h HOST230 -U user
>>psql (9.2.4, server 9.1.7)
>>
>>user=> select count(1) from foo;
>>count
>>-------
>> 1
>>(1 row)
>>
>>user=> \q
>>
>>#######################################
>># The count is correct in a non-PRIMARY
>>#######################################
>>
>>$ psql -p 25432 -h HOST231 -U user
>>psql (9.2.4, server 9.1.7)
>>
>>user=> select count(1) from foo;
>>count
>>--------
>>100000
>>(1 row)
>>
>>user=> \q
>>
>>
>>
>>----- Original Message -----
>>> From: Paul Jones <pb...@cm...>
>>> To: "pos...@li..." <pos...@li...>
>>> Cc:
>>> Sent: Monday, April 8, 2013 2:58 PM
>>> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>>>
>>>T hanks for everyone's explanation of PRIMARY. It is much clearer now.
>>>
>>> I believe, then, that we may have uncovered a bug in PRIMARY.
>>>
>>> We created a new cluster (8 nodes, 3 coordinators), but with only one PRIMARY
>>> datanode. The PRIMARY was
>>> declared the same on all 3 coordinators.
>>>
>>> A table that was declared DISTRIBUTE BY REPLICATION and loaded by \copy did
>>> not have any rows
>>> present on the PRIMARY! Further, other tables with FK's referring to this
>>> table had RI failures when they were loaded,
>>> even though there were complete copies of the table in all the other non-PRIMARY
>>> datanodes.
>>>
>>> When we remade the cluster without any PRIMARY, this table loaded into all
>>> datanodes and there were no RI failures.
>>>
>>> Is this a bug? Unfortunately I won't be able to experiment with this until
>>> we finish executing our test plan, perhaps
>>> a few days.
>>>
>>> PJ
>>>
>>>
>>>
>>>
>>>> ________________________________
>>>> From: Koichi Suzuki <koi...@gm...>
>>>> To: Andrei Martsinchyk <and...@gm...>
>>>> Cc: "pos...@li..."
>>> <pos...@li...>
>>>> Sent: Sunday, April 7, 2013 9:23 AM
>>>> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>>>>
>>>>
>>>> Primary node is useful to maintain replicated table in a consistent status
>>> in all the datanode. All the writes to a replicated table goes first to the
>>> primary node so all the conflicts are resolved here and prevents conflict writes
>>> in other datanodes. In this sense, this may prvent some deadlocks but it does
>>> not remove the chance of deadlocks in general sense.
>>>>
>>>> On the othe hand, preferred node (datanode again) saves inter-server
>>> communication to read replciated table. It does not work to maintain
>>> replicated table consistensy but helps to gain some performance.
>>>>
>>>> Regards;
>>>> ---
>>>> Koichi Suzuki
>>>>
>>>>
>>>>
>>>> ----------
>>>> Koichi Suzuki
>>>>
>>>>
>>>> 2013/4/7 Andrei Martsinchyk <and...@gm...>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2013/4/7 Jov <am...@am...>
>>>>>
>>>>> datanode use primary node to solve replication table write,it is
>>> good,but how coordinator solve the dead lock problem? the coordinator nodes
>>> replication all globle catalog tables across coords,they are some kind
>>> replication table.
>>>>>>
>>>>>>
>>>>>> eg.
>>>>>>
>>>>>> client 1 run alter table tb on coord node A,it will lock local
>>> catalog data on A,and wait other coord node B.
>>>>>> client 2 run alter table tb on coord node B,it will lock local
>>> catalog data on B,and wait other coord node A.
>>>>>>
>>>>>>
>>>>>>
>>>>>> so how XC handle this dead lock?
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> XC does not handle this, it will be deadlocked.
>>>>> Fortunately, chance of concurrent DDL much less then chance of
>>> concurrent replicated update.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> 2013/4/6 Andrei Martsinchyk <and...@gm...>
>>>>>>
>>>>>> PRIMARY was introduced to avoid distributed deadlocks when updating
>>> replicated tables.
>>>>>>> To better understand the problem, imagine two transactions A and
>>> B are updating the same tuple in replicated concurrently.
>>>>>>> Normally coordinator sends the same commands to all datanodes at
>>> the same time, and if on some node A updates the tuple first, B will be waiting
>>> for the end of transaction A. If on other node B wins the race, both
>>> transactions will be waiting for each other. It is hard to detect such deadlock,
>>> the information about locks is not sent across network.
>>>>>>> But it is possible to avoid. The idea is to set one datanode as
>>> a primary, and execute distributed update on primary node first, and go on with
>>> other nodes only if operation succeeds on primary.
>>>>>>> With this approach row lock on primary would stop concurrent
>>> transactions from taking row locks on other nodes that could prevent command
>>> completion.
>>>>>>> So, to have this stuff working properly you should
>>>>>>> 1) set only one datanode as a primary;
>>>>>>> 2) if you have multiple coordinators, the same datanode should
>>> be set as a primary on each of them.
>>>>>>> Obvious drawback of the approach is double execution time of
>>> replicated updates.
>>>>>>> Note: "update" means any write access.
>>>>>>> Hope this answers 1)-3)
>>>>>>> Regarding 4), the query
>>>>>>>
>>>>>>>
>>>>>>> select nodeoids from pg_class, pgxc_class where pg_class.oid =
>>> pcrelid and relname = '<your table name>';
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> returns the list of nodes, where the specified table is
>>> distributed on. I guess there are 7 of them.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/4/5 Paul Jones <pb...@cm...>
>>>>>>>
>>>>>>>
>>>>>>>> We are experimenting with an 8-datanode, 3-coordinator
>>> cluster and we
>>>>>>>> have some questions about the use of PRIMARY and a problem.
>>>>>>>>
>>>>>>>> The manual explains what PRIMARY means but does not provide
>>> much detail
>>>>>>>> about when you would use it or not use it.
>>>>>>>>
>>>>>>>> 1) Can PRIMARY apply to coordinators and if so, when would
>>> you
>>>>>>>> want it or not?
>>>>>>>>
>>>>>>>> 2) When would you use PRIMARY for datanodes or not, and
>>> would you
>>>>>>>> ever want more than one datanode to be a primary?
>>>>>>>>
>>>>>>>> 3) Does a pgxc_node datanode entry on its own server have to
>>> be
>>>>>>>> the FQDN server name or can it be 'localhost'?
>>>>>>>>
>>>>>>>> 4) We have a table that is defined as DISTRIBUTE BY
>>> REPLICATION.
>>>>>>>> It only loads on the first 7 nodes. It will just not
>>> load on
>>>>>>>> node 8. There are a lot of FK references from other
>>> tables to it,
>>>>>>>> but it itself only has a simple CHAR(11) PK, one
>>> constraint,
>>>>>>>> and 3 indices.
>>>>>>>>
>>>>>>>> Has anyone seen anything like this before?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Paul Jones
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Minimize network downtime and maximize team effectiveness.
>>>>>>>> Reduce network management and security costs.Learn how to
>>> hire
>>>>>>>> the most talented Cisco Certified professionals. Visit the
>>>>>>>> Employer Resources Portal
>>>>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>>>>> _______________________________________________
>>>>>>>> Postgres-xc-general mailing list
>>>>>>>> Pos...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Andrei Martsinchyk
>>>>>>>
>>>>>>> StormDB - http://www.stormdb.com
>>>>>>> The Database Cloud
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Minimize network downtime and maximize team effectiveness.
>>>>>>> Reduce network management and security costs.Learn how to hire
>>>>>>> the most talented Cisco Certified professionals. Visit the
>>>>>>> Employer Resources Portal
>>>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>>>> _______________________________________________
>>>>>>> Postgres-xc-general mailing list
>>>>>>> Pos...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jov
>>>>>>
>>>>>> blog: http:amutu.com/blog
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andrei Martsinchyk
>>>>>
>>>>> StormDB - http://www.stormdb.com
>>>>> The Database Cloud
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Minimize network downtime and maximize team effectiveness.
>>>>> Reduce network management and security costs.Learn how to hire
>>>>> the most talented Cisco Certified professionals. Visit the
>>>>> Employer Resources Portal
>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>> _______________________________________________
>>>>> Postgres-xc-general mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>
>>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Minimize network downtime and maximize team effectiveness.
>>>> Reduce network management and security costs.Learn how to hire
>>>> the most talented Cisco Certified professionals. Visit the
>>>> Employer Resources Portal
>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>> _______________________________________________
>>>> Postgres-xc-general mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>
>>>>
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Minimize network downtime and maximize team effectiveness.
>>> Reduce network management and security costs.Learn how to hire
>>> the most talented Cisco Certified professionals. Visit the
>>> Employer Resources Portal
>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>
>>------------------------------------------------------------------------------
>>Try New Relic Now & We'll Send You this Cool Shirt
>>New Relic is the only SaaS-based application performance monitoring service
>>that delivers powerful full stack analytics. Optimize and monitor your
>>browser, app, & servers with just a few lines of code. Try New Relic
>>and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
>>_______________________________________________
>>Postgres-xc-general mailing list
>>Pos...@li...
>>https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>
>--
>StormDB - http://www.stormdb.com
>The Database Cloud
>
>
|
|
From: Masaki H. <his...@la...> - 2013-04-26 07:00:23
|
Hi, I have been testing pgxc behavior when network between gtm and gtm_standy has down. I have started up gtm_standby and linked down bw gtm and gtm_standby by using iptables. I expected that gtm_standby would fail with error because gtm_standby cannot establish synchronization with gtm active. Problem : 1. gtm_standby started without error response although gtm_standby has failed to establish connection with gtm active from gtm.log. Is gtm alive or not? 2. Assuming gtm alive I have tried immediate stop to gtm_standby but failed. 3. Checking at gtm status, it is stopped. Question (1) Am I testing right? (2) Is this expected behavior for gtm_standby when it cannot establish connection with gtm? Test procedure as follows; (1) Start up GTM active at Server A $ gtm_ctl -D gtm/data/ -Z gtm status (2) Network down (simulated with setting iptables) $ iptables -I INPUT -i bond1 -j DROP $ iptables -I OUTPUT -o bond1 -j DROP (3) Start up GTM standby at Server B $ gtm_ctl -D gtm/data/ -Z gtm start server starting (4) Stop gtm_stadby waiting for server to shut down................ ............................................... failed gtm_ctl: server does not shut down (5) Check status $ gtm_ctl -D gtm/data/ -Z gtm status gtm_ctl: no server running $ ps aux | grep gtm | grep -v grep # No output $ ll gtm/data/ 合計 12 -rw------- 1 pgxc pgxc 2217 2月 15 06:15 2013 gtm.conf -rw------- 1 pgxc pgxc 295 2月 25 04:39 2013 gtm.log -rw------- 1 pgxc pgxc 45 2月 25 04:38 2013 gtm.pid $ less gtm/data/gtm.log 1:140357722076928:2013-02-25 04:38:23.850 JST -LOG: Connecting the GTM active on 192.168.2.1:19000... LOCATION: gtm_standby_connectToActiveGTM, gtm_standby.c:511 1:140357722076928:2013-02-25 04:39:26.856 JST -FATAL: Failed to establish a connection to active-GTM. LOCATION: main, main.c:553 pos...@li... Tested Version : commitID(3f15aef4ed3f8b2014d42bce3ed75650c4db907d) Rgds, Mark |
|
From: Nikhil S. <ni...@st...> - 2013-04-26 03:47:55
|
Hi Paul,
It's not recommended to log in directly into a datanode to run queries. All
queries ought to be directed via coordinators to use sane XIDs.
What does the following query return when run from the coordinator?
EXECUTE DIRECT ON (data_east03) 'SELECT count(1) FROM foo';
Regards,
Nikhils
On Fri, Apr 26, 2013 at 3:16 AM, Paul Jones <pb...@cm...> wrote:
> I think I have discovered the problem with PRIMARY and our loads. The
> PRIMARY data node does not load
> data for tables that are DISTRIBUTE BY REPLICATION. We are on PGXC
> 1.0.2. Here is a way to
> demonstrate the problem:
>
> # Demonstrate PRIMARY node \copy bug.
>
> # 3 Coordinators
> # 8 Datanodes
>
> # Prepare some data:
> $ awk 'BEGIN{for(i=1;i<=100000;i++) print i; }' > DATA
>
> $ psql -h HOST242 -U postgres
> postgres=# select * from pgxc_node;
> node_name | node_type | node_port | node_host |
> nodeis_primary | nodeis_preferred | node_id
>
> ----------------+-----------+-----------+---------------------+----------------+------------------+-------------
> coordw | C | 5432 | localhost | f
> | f | 670793242
> coorde | C | 5432 | HOST225.example.com | f
> | f | 329164574
> coordc | C | 5432 | HOST232.example.com | f
> | f | -1588937622
> data_east01 | D | 25432 | HOST225.example.com | f
> | f | -2053435448
> data_east02 | D | 25432 | HOST226.example.com | f
> | f | 94547764
> data_east03 | D | 25432 | HOST230.example.com | t
> | f | 197970754
> data_central01 | D | 25432 | HOST231.example.com | f
> | f | 124274836
> data_central02 | D | 25432 | HOST232.example.com | f
> | f | 1002175669
> data_central03 | D | 25432 | HOST238.example.com | f
> | f | -1150964881
> data_west01 | D | 25432 | HOST239.example.com | f
> | f | 2129529735
> data_west02 | D | 25432 | HOST242.example.com | f
> | f | -717656524
> (11 rows)
>
> #################################################
> # If we create a table distributed by replication
> # and fill it with an insert, we see the same
> # row counts from a coordinator and the PRIMARY
> #################################################
>
> psql -h HOST242 -U user
> user=> create table foo (aaa int) distribute by replication;
> CREATE TABLE
> user=> insert into foo select generate_series(1,100000);
> INSERT 0 100000
> user=> select count(1) from foo;
> count
> --------
> 100000
> (1 row)
> user=> \q
>
> $ psql -p 25432 -h HOST230 -U user
> psql (9.2.4, server 9.1.7)
>
> user=> select count(1) from foo;
> count
> --------
> 100000
> (1 row)
>
> user=> \q
>
> ###############################################
> # Now truncate the table and load it with \copy
> ###############################################
>
> $ psql -h HOST242 -U user
> user=> truncate table foo;
> TRUNCATE TABLE
> user=>
> user=> \copy foo from DATA
> user=> select count(1) from foo;
> count
> --------
> 100000
> (1 row)
>
> user=> \q
>
> #############################################
> # The count was correct from the coordinator,
> # but not in the PRIMARY node.
> #############################################
>
> $ psql -p 25432 -h HOST230 -U user
> psql (9.2.4, server 9.1.7)
>
> user=> select count(1) from foo;
> count
> -------
> 1
> (1 row)
>
> user=> \q
>
> #######################################
> # The count is correct in a non-PRIMARY
> #######################################
>
> $ psql -p 25432 -h HOST231 -U user
> psql (9.2.4, server 9.1.7)
>
> user=> select count(1) from foo;
> count
> --------
> 100000
> (1 row)
>
> user=> \q
>
>
>
> ----- Original Message -----
> > From: Paul Jones <pb...@cm...>
> > To: "pos...@li..." <
> pos...@li...>
> > Cc:
> > Sent: Monday, April 8, 2013 2:58 PM
> > Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
> >
> >T hanks for everyone's explanation of PRIMARY. It is much clearer now.
> >
> > I believe, then, that we may have uncovered a bug in PRIMARY.
> >
> > We created a new cluster (8 nodes, 3 coordinators), but with only one
> PRIMARY
> > datanode. The PRIMARY was
> > declared the same on all 3 coordinators.
> >
> > A table that was declared DISTRIBUTE BY REPLICATION and loaded by \copy
> did
> > not have any rows
> > present on the PRIMARY! Further, other tables with FK's referring to
> this
> > table had RI failures when they were loaded,
> > even though there were complete copies of the table in all the other
> non-PRIMARY
> > datanodes.
> >
> > When we remade the cluster without any PRIMARY, this table loaded into
> all
> > datanodes and there were no RI failures.
> >
> > Is this a bug? Unfortunately I won't be able to experiment with this
> until
> > we finish executing our test plan, perhaps
> > a few days.
> >
> > PJ
> >
> >
> >
> >
> >> ________________________________
> >> From: Koichi Suzuki <koi...@gm...>
> >> To: Andrei Martsinchyk <and...@gm...>
> >> Cc: "pos...@li..."
> > <pos...@li...>
> >> Sent: Sunday, April 7, 2013 9:23 AM
> >> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
> >>
> >>
> >> Primary node is useful to maintain replicated table in a consistent
> status
> > in all the datanode. All the writes to a replicated table goes first to
> the
> > primary node so all the conflicts are resolved here and prevents
> conflict writes
> > in other datanodes. In this sense, this may prvent some deadlocks but
> it does
> > not remove the chance of deadlocks in general sense.
> >>
> >> On the othe hand, preferred node (datanode again) saves inter-server
> > communication to read replciated table. It does not work to maintain
> > replicated table consistensy but helps to gain some performance.
> >>
> >> Regards;
> >> ---
> >> Koichi Suzuki
> >>
> >>
> >>
> >> ----------
> >> Koichi Suzuki
> >>
> >>
> >> 2013/4/7 Andrei Martsinchyk <and...@gm...>
> >>
> >>
> >>>
> >>>
> >>>
> >>>
> >>> 2013/4/7 Jov <am...@am...>
> >>>
> >>> datanode use primary node to solve replication table write,it is
> > good,but how coordinator solve the dead lock problem? the coordinator
> nodes
> > replication all globle catalog tables across coords,they are some kind
> > replication table.
> >>>>
> >>>>
> >>>> eg.
> >>>>
> >>>> client 1 run alter table tb on coord node A,it will lock local
> > catalog data on A,and wait other coord node B.
> >>>> client 2 run alter table tb on coord node B,it will lock local
> > catalog data on B,and wait other coord node A.
> >>>>
> >>>>
> >>>>
> >>>> so how XC handle this dead lock?
> >>>>
> >>>>
> >>>
> >>>
> >>> XC does not handle this, it will be deadlocked.
> >>> Fortunately, chance of concurrent DDL much less then chance of
> > concurrent replicated update.
> >>>
> >>>
> >>>
> >>>
> >>>>
> >>>> 2013/4/6 Andrei Martsinchyk <and...@gm...>
> >>>>
> >>>> PRIMARY was introduced to avoid distributed deadlocks when updating
> > replicated tables.
> >>>>> To better understand the problem, imagine two transactions A and
> > B are updating the same tuple in replicated concurrently.
> >>>>> Normally coordinator sends the same commands to all datanodes at
> > the same time, and if on some node A updates the tuple first, B will be
> waiting
> > for the end of transaction A. If on other node B wins the race, both
> > transactions will be waiting for each other. It is hard to detect such
> deadlock,
> > the information about locks is not sent across network.
> >>>>> But it is possible to avoid. The idea is to set one datanode as
> > a primary, and execute distributed update on primary node first, and go
> on with
> > other nodes only if operation succeeds on primary.
> >>>>> With this approach row lock on primary would stop concurrent
> > transactions from taking row locks on other nodes that could prevent
> command
> > completion.
> >>>>> So, to have this stuff working properly you should
> >>>>> 1) set only one datanode as a primary;
> >>>>> 2) if you have multiple coordinators, the same datanode should
> > be set as a primary on each of them.
> >>>>> Obvious drawback of the approach is double execution time of
> > replicated updates.
> >>>>> Note: "update" means any write access.
> >>>>> Hope this answers 1)-3)
> >>>>> Regarding 4), the query
> >>>>>
> >>>>>
> >>>>> select nodeoids from pg_class, pgxc_class where pg_class.oid =
> > pcrelid and relname = '<your table name>';
> >>>>>
> >>>>>
> >>>>>
> >>>>> returns the list of nodes, where the specified table is
> > distributed on. I guess there are 7 of them.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> 2013/4/5 Paul Jones <pb...@cm...>
> >>>>>
> >>>>>
> >>>>>> We are experimenting with an 8-datanode, 3-coordinator
> > cluster and we
> >>>>>> have some questions about the use of PRIMARY and a problem.
> >>>>>>
> >>>>>> The manual explains what PRIMARY means but does not provide
> > much detail
> >>>>>> about when you would use it or not use it.
> >>>>>>
> >>>>>> 1) Can PRIMARY apply to coordinators and if so, when would
> > you
> >>>>>> want it or not?
> >>>>>>
> >>>>>> 2) When would you use PRIMARY for datanodes or not, and
> > would you
> >>>>>> ever want more than one datanode to be a primary?
> >>>>>>
> >>>>>> 3) Does a pgxc_node datanode entry on its own server have to
> > be
> >>>>>> the FQDN server name or can it be 'localhost'?
> >>>>>>
> >>>>>> 4) We have a table that is defined as DISTRIBUTE BY
> > REPLICATION.
> >>>>>> It only loads on the first 7 nodes. It will just not
> > load on
> >>>>>> node 8. There are a lot of FK references from other
> > tables to it,
> >>>>>> but it itself only has a simple CHAR(11) PK, one
> > constraint,
> >>>>>> and 3 indices.
> >>>>>>
> >>>>>> Has anyone seen anything like this before?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Paul Jones
> >>>>>>
> >>>>>>
> ------------------------------------------------------------------------------
> >>>>>> Minimize network downtime and maximize team effectiveness.
> >>>>>> Reduce network management and security costs.Learn how to
> > hire
> >>>>>> the most talented Cisco Certified professionals. Visit the
> >>>>>> Employer Resources Portal
> >>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
> >>>>>> _______________________________________________
> >>>>>> Postgres-xc-general mailing list
> >>>>>> Pos...@li...
> >>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Andrei Martsinchyk
> >>>>>
> >>>>> StormDB - http://www.stormdb.com
> >>>>> The Database Cloud
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> ------------------------------------------------------------------------------
> >>>>> Minimize network downtime and maximize team effectiveness.
> >>>>> Reduce network management and security costs.Learn how to hire
> >>>>> the most talented Cisco Certified professionals. Visit the
> >>>>> Employer Resources Portal
> >>>>> http://www.cisco.com/web/learning/employer_resources/index.html
> >>>>> _______________________________________________
> >>>>> Postgres-xc-general mailing list
> >>>>> Pos...@li...
> >>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Jov
> >>>>
> >>>> blog: http:amutu.com/blog
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Andrei Martsinchyk
> >>>
> >>> StormDB - http://www.stormdb.com
> >>> The Database Cloud
> >>>
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> Minimize network downtime and maximize team effectiveness.
> >>> Reduce network management and security costs.Learn how to hire
> >>> the most talented Cisco Certified professionals. Visit the
> >>> Employer Resources Portal
> >>> http://www.cisco.com/web/learning/employer_resources/index.html
> >>> _______________________________________________
> >>> Postgres-xc-general mailing list
> >>> Pos...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>>
> >>>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Minimize network downtime and maximize team effectiveness.
> >> Reduce network management and security costs.Learn how to hire
> >> the most talented Cisco Certified professionals. Visit the
> >> Employer Resources Portal
> >> http://www.cisco.com/web/learning/employer_resources/index.html
> >> _______________________________________________
> >> Postgres-xc-general mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>
> >>
> >>
> >
> >
> ------------------------------------------------------------------------------
> > Minimize network downtime and maximize team effectiveness.
> > Reduce network management and security costs.Learn how to hire
> > the most talented Cisco Certified professionals. Visit the
> > Employer Resources Portal
> > http://www.cisco.com/web/learning/employer_resources/index.html
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
--
StormDB - http://www.stormdb.com
The Database Cloud
|
|
From: Paul J. <pb...@cm...> - 2013-04-25 21:46:44
|
I think I have discovered the problem with PRIMARY and our loads. The PRIMARY data node does not load
data for tables that are DISTRIBUTE BY REPLICATION. We are on PGXC 1.0.2. Here is a way to
demonstrate the problem:
# Demonstrate PRIMARY node \copy bug.
# 3 Coordinators
# 8 Datanodes
# Prepare some data:
$ awk 'BEGIN{for(i=1;i<=100000;i++) print i; }' > DATA
$ psql -h HOST242 -U postgres
postgres=# select * from pgxc_node;
node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id
----------------+-----------+-----------+---------------------+----------------+------------------+-------------
coordw | C | 5432 | localhost | f | f | 670793242
coorde | C | 5432 | HOST225.example.com | f | f | 329164574
coordc | C | 5432 | HOST232.example.com | f | f | -1588937622
data_east01 | D | 25432 | HOST225.example.com | f | f | -2053435448
data_east02 | D | 25432 | HOST226.example.com | f | f | 94547764
data_east03 | D | 25432 | HOST230.example.com | t | f | 197970754
data_central01 | D | 25432 | HOST231.example.com | f | f | 124274836
data_central02 | D | 25432 | HOST232.example.com | f | f | 1002175669
data_central03 | D | 25432 | HOST238.example.com | f | f | -1150964881
data_west01 | D | 25432 | HOST239.example.com | f | f | 2129529735
data_west02 | D | 25432 | HOST242.example.com | f | f | -717656524
(11 rows)
#################################################
# If we create a table distributed by replication
# and fill it with an insert, we see the same
# row counts from a coordinator and the PRIMARY
#################################################
psql -h HOST242 -U user
user=> create table foo (aaa int) distribute by replication;
CREATE TABLE
user=> insert into foo select generate_series(1,100000);
INSERT 0 100000
user=> select count(1) from foo;
count
--------
100000
(1 row)
user=> \q
$ psql -p 25432 -h HOST230 -U user
psql (9.2.4, server 9.1.7)
user=> select count(1) from foo;
count
--------
100000
(1 row)
user=> \q
###############################################
# Now truncate the table and load it with \copy
###############################################
$ psql -h HOST242 -U user
user=> truncate table foo;
TRUNCATE TABLE
user=>
user=> \copy foo from DATA
user=> select count(1) from foo;
count
--------
100000
(1 row)
user=> \q
#############################################
# The count was correct from the coordinator,
# but not in the PRIMARY node.
#############################################
$ psql -p 25432 -h HOST230 -U user
psql (9.2.4, server 9.1.7)
user=> select count(1) from foo;
count
-------
1
(1 row)
user=> \q
#######################################
# The count is correct in a non-PRIMARY
#######################################
$ psql -p 25432 -h HOST231 -U user
psql (9.2.4, server 9.1.7)
user=> select count(1) from foo;
count
--------
100000
(1 row)
user=> \q
----- Original Message -----
> From: Paul Jones <pb...@cm...>
> To: "pos...@li..." <pos...@li...>
> Cc:
> Sent: Monday, April 8, 2013 2:58 PM
> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>
>T hanks for everyone's explanation of PRIMARY. It is much clearer now.
>
> I believe, then, that we may have uncovered a bug in PRIMARY.
>
> We created a new cluster (8 nodes, 3 coordinators), but with only one PRIMARY
> datanode. The PRIMARY was
> declared the same on all 3 coordinators.
>
> A table that was declared DISTRIBUTE BY REPLICATION and loaded by \copy did
> not have any rows
> present on the PRIMARY! Further, other tables with FK's referring to this
> table had RI failures when they were loaded,
> even though there were complete copies of the table in all the other non-PRIMARY
> datanodes.
>
> When we remade the cluster without any PRIMARY, this table loaded into all
> datanodes and there were no RI failures.
>
> Is this a bug? Unfortunately I won't be able to experiment with this until
> we finish executing our test plan, perhaps
> a few days.
>
> PJ
>
>
>
>
>> ________________________________
>> From: Koichi Suzuki <koi...@gm...>
>> To: Andrei Martsinchyk <and...@gm...>
>> Cc: "pos...@li..."
> <pos...@li...>
>> Sent: Sunday, April 7, 2013 9:23 AM
>> Subject: Re: [Postgres-xc-general] Questions about PRIMARY and a problem
>>
>>
>> Primary node is useful to maintain replicated table in a consistent status
> in all the datanode. All the writes to a replicated table goes first to the
> primary node so all the conflicts are resolved here and prevents conflict writes
> in other datanodes. In this sense, this may prvent some deadlocks but it does
> not remove the chance of deadlocks in general sense.
>>
>> On the othe hand, preferred node (datanode again) saves inter-server
> communication to read replciated table. It does not work to maintain
> replicated table consistensy but helps to gain some performance.
>>
>> Regards;
>> ---
>> Koichi Suzuki
>>
>>
>>
>> ----------
>> Koichi Suzuki
>>
>>
>> 2013/4/7 Andrei Martsinchyk <and...@gm...>
>>
>>
>>>
>>>
>>>
>>>
>>> 2013/4/7 Jov <am...@am...>
>>>
>>> datanode use primary node to solve replication table write,it is
> good,but how coordinator solve the dead lock problem? the coordinator nodes
> replication all globle catalog tables across coords,they are some kind
> replication table.
>>>>
>>>>
>>>> eg.
>>>>
>>>> client 1 run alter table tb on coord node A,it will lock local
> catalog data on A,and wait other coord node B.
>>>> client 2 run alter table tb on coord node B,it will lock local
> catalog data on B,and wait other coord node A.
>>>>
>>>>
>>>>
>>>> so how XC handle this dead lock?
>>>>
>>>>
>>>
>>>
>>> XC does not handle this, it will be deadlocked.
>>> Fortunately, chance of concurrent DDL much less then chance of
> concurrent replicated update.
>>>
>>>
>>>
>>>
>>>>
>>>> 2013/4/6 Andrei Martsinchyk <and...@gm...>
>>>>
>>>> PRIMARY was introduced to avoid distributed deadlocks when updating
> replicated tables.
>>>>> To better understand the problem, imagine two transactions A and
> B are updating the same tuple in replicated concurrently.
>>>>> Normally coordinator sends the same commands to all datanodes at
> the same time, and if on some node A updates the tuple first, B will be waiting
> for the end of transaction A. If on other node B wins the race, both
> transactions will be waiting for each other. It is hard to detect such deadlock,
> the information about locks is not sent across network.
>>>>> But it is possible to avoid. The idea is to set one datanode as
> a primary, and execute distributed update on primary node first, and go on with
> other nodes only if operation succeeds on primary.
>>>>> With this approach row lock on primary would stop concurrent
> transactions from taking row locks on other nodes that could prevent command
> completion.
>>>>> So, to have this stuff working properly you should
>>>>> 1) set only one datanode as a primary;
>>>>> 2) if you have multiple coordinators, the same datanode should
> be set as a primary on each of them.
>>>>> Obvious drawback of the approach is double execution time of
> replicated updates.
>>>>> Note: "update" means any write access.
>>>>> Hope this answers 1)-3)
>>>>> Regarding 4), the query
>>>>>
>>>>>
>>>>> select nodeoids from pg_class, pgxc_class where pg_class.oid =
> pcrelid and relname = '<your table name>';
>>>>>
>>>>>
>>>>>
>>>>> returns the list of nodes, where the specified table is
> distributed on. I guess there are 7 of them.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2013/4/5 Paul Jones <pb...@cm...>
>>>>>
>>>>>
>>>>>> We are experimenting with an 8-datanode, 3-coordinator
> cluster and we
>>>>>> have some questions about the use of PRIMARY and a problem.
>>>>>>
>>>>>> The manual explains what PRIMARY means but does not provide
> much detail
>>>>>> about when you would use it or not use it.
>>>>>>
>>>>>> 1) Can PRIMARY apply to coordinators and if so, when would
> you
>>>>>> want it or not?
>>>>>>
>>>>>> 2) When would you use PRIMARY for datanodes or not, and
> would you
>>>>>> ever want more than one datanode to be a primary?
>>>>>>
>>>>>> 3) Does a pgxc_node datanode entry on its own server have to
> be
>>>>>> the FQDN server name or can it be 'localhost'?
>>>>>>
>>>>>> 4) We have a table that is defined as DISTRIBUTE BY
> REPLICATION.
>>>>>> It only loads on the first 7 nodes. It will just not
> load on
>>>>>> node 8. There are a lot of FK references from other
> tables to it,
>>>>>> but it itself only has a simple CHAR(11) PK, one
> constraint,
>>>>>> and 3 indices.
>>>>>>
>>>>>> Has anyone seen anything like this before?
>>>>>>
>>>>>> Thanks,
>>>>>> Paul Jones
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Minimize network downtime and maximize team effectiveness.
>>>>>> Reduce network management and security costs.Learn how to
> hire
>>>>>> the most talented Cisco Certified professionals. Visit the
>>>>>> Employer Resources Portal
>>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>>> _______________________________________________
>>>>>> Postgres-xc-general mailing list
>>>>>> Pos...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andrei Martsinchyk
>>>>>
>>>>> StormDB - http://www.stormdb.com
>>>>> The Database Cloud
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Minimize network downtime and maximize team effectiveness.
>>>>> Reduce network management and security costs.Learn how to hire
>>>>> the most talented Cisco Certified professionals. Visit the
>>>>> Employer Resources Portal
>>>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>>>> _______________________________________________
>>>>> Postgres-xc-general mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Jov
>>>>
>>>> blog: http:amutu.com/blog
>>>
>>>
>>>
>>>
>>> --
>>> Andrei Martsinchyk
>>>
>>> StormDB - http://www.stormdb.com
>>> The Database Cloud
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Minimize network downtime and maximize team effectiveness.
>>> Reduce network management and security costs.Learn how to hire
>>> the most talented Cisco Certified professionals. Visit the
>>> Employer Resources Portal
>>> http://www.cisco.com/web/learning/employer_resources/index.html
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>> ------------------------------------------------------------------------------
>> Minimize network downtime and maximize team effectiveness.
>> Reduce network management and security costs.Learn how to hire
>> the most talented Cisco Certified professionals. Visit the
>> Employer Resources Portal
>> http://www.cisco.com/web/learning/employer_resources/index.html
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>>
>
> ------------------------------------------------------------------------------
> Minimize network downtime and maximize team effectiveness.
> Reduce network management and security costs.Learn how to hire
> the most talented Cisco Certified professionals. Visit the
> Employer Resources Portal
> http://www.cisco.com/web/learning/employer_resources/index.html
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
|
|
From: Koichi S. <koi...@gm...> - 2013-04-23 14:46:58
|
Postgres-XC development group release postgres-xc 1.0.3 with some fixes/improvement as well as upgrade the base code to PostgreSQL 9.1.9. You will find resources at: http://postgres-xc.sourceforge.net/ http://postgres-xc.sourceforge.net/docs/ https://sourceforge.net/projects/postgres-xc/files/Version_1.0/ Enjoy. Upgrading base PostgreSQL version to 9.2 will be included in the next major release. ---- Postgres-XC is a symmetric PostgreSQL cluster which provides both read and write scalability using mixture of table sharding and replication. Each Postgres-XC cluster node provides single database view, where application can connect to any cluster node and run any transactions. Result of transactions are visible from all the cluster nodes without delay. ---------- Koichi Suzuki |
|
From: Anson A. <ans...@gm...> - 2013-04-23 14:08:51
|
Sorry forgot about GTM proxy config. As for Coordinator, realized that I
had configured wrong gtm_host @ the time. So coordinator is working.
ANyway, here's the GTM_proxy config:
#-----------------------------
# GTM Proxy configuration file
#-----------------------------
#
# This file must be placed on gtm working directory
# specified by -D command line option of gtm_proxy or gtm_ctl.
# The configuration file name must be "gtm_proxy.conf"
#
#
# This file consists of lines of the form
#
# name = value
#
# (The "=" is optional.) Whitespace may be used. Comments are
# introduced with "#" anywhere on a line. The complete list of
# parameter names and allowed values can be found in the
# Postgres-XC documentation.
#
# The commented-out settings shown in this file represent the default
# values.
#
# Re-commenting a setting is NOT sufficient to revert it to the default
# value.
#
# You need to restart the server.
#------------------------------------------------------------------------------
# GENERAL PARAMETERS
#------------------------------------------------------------------------------
nodename = 'on # Specifies the node name.
# (changes requires restart)
listen_addresses = '*' # Listen addresses of this GTM.
# (changes requires restart)
port = 6666 # Port number of this GTM.
# (changes requires restart)
#------------------------------------------------------------------------------
# GTM PROXY PARAMETERS
#------------------------------------------------------------------------------
#worker_threads = 1 # Number of the worker thread of this
# GTM proxy
# (changes requires restart)
#------------------------------------------------------------------------------
# GTM CONNECTION PARAMETERS
#------------------------------------------------------------------------------
# Those parameters are used to connect to a GTM server
gtm_host = '<ip address to gtm_host>' # Listen address of the active GTM.
# (changes requires restart)
gtm_port = 6668 # Port number of the active GTM.
# (changes requires restart)
#------------------------------------------------------------------------------
# Behavior at GTM communication error
#------------------------------------------------------------------------------
#gtm_connect_retry_idle = 0 # How long (in secs) to wait before GTM proxy
# retries to connect to GTM.
#gtm_connect_retry_count = 0 # How many times GTM proxy retries to connect
# to GTM.
#gtm_connect_retry_interval = 0 # How long (in secs) to wait until the next
# retry to connect to GTM.
#err_wait_idle = 0 # How long (in secs) GTM proxy should wait
# before it detects reconnect from gtm_ctl.
#err_wait_count = 0 # How many times GTM should detect reconnect
# from gtm_ctl.
#err_wait_interval = 0 # How long (in secs) GTM proxy should wait
# until next detection of reconnect from
# gtm_ctl.
#
#
#------------------------------------------------------------------------------
# Other options
#------------------------------------------------------------------------------
#keepalives_idle = 0 # Keepalives_idle parameter.
#keepalives_interval = 0 # Keepalives_interval parameter.
#keepalives_count = 0 # Keepalives_count internal parameter.
log_file = 'gtm_proxy.log' # Log file name
#log_min_messages = WARNING # log_min_messages. Default WARNING.
# Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3,
# DEBUG2, DEBUG1, INFO, NOTICE, WARNING,
# ERROR, LOG, FATAL, PANIC.
================================
The gtm and gtm_proxy are on same server, and their is no firewall blocking
these ports.
On Tue, Apr 23, 2013 at 12:41 AM, Nikhil Sontakke <ni...@st...>wrote:
> You did not share the contents of gtm_proxy.conf. Also double-check the
> values of gtm_host and gtm_port parameters in the coordinator's
> postgresql.conf.
>
> If this is on a Linux platform, check if iptables/firewall is blocking
> connection to port 6666 or something as well.
>
> HTH,
> Nikhils
>
>
> On Tue, Apr 23, 2013 at 1:24 AM, Anson Abraham <ans...@gm...>wrote:
>
>> Also when I started up coodinator I get error also:
>>
>> LOG: database system was interrupted; last known up at 2013-04-22
>> 15:14:08 EDT
>> LOG: database system was not properly shut down; automatic recovery in
>> progress
>> LOG: record with zero length at 0/16E4EA0
>> LOG: redo is not required
>> LOG: database system is ready to accept connections
>> LOG: autovacuum launcher started
>> WARNING: can not connect to GTM: Success
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: Xid is invalid.
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: Xid is invalid.
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: can not connect to GTM: Bad file descriptor
>> ERROR: GTM error, could not obtain snapshot
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: Xid is invalid.
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: Xid is invalid.
>> WARNING: can not connect to GTM: Bad file descriptor
>> WARNING: can not connect to GTM: Bad file descriptor
>> ERROR: GTM error, could not obtain snapshot
>>
>>
>> I started the GTM initially as such:
>> /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm
>> -l /var/log/postgresql/gtm.log start
>>
>> But then I opted for:
>> /usr/local/pgsql/bin/gtm -D /var/lib/postgresql/pgxc/data_gtm >
>> /var/log/postgresql/gtm.log 2>&1 &
>>
>> And still no errors in the log.
>>
>>
>> On Mon, Apr 22, 2013 at 1:57 PM, Anson Abraham <ans...@gm...>wrote:
>>
>>> # specified by -D command line option of gtm or gtm_ctl. The
>>> # configuration file name must be "gtm.conf"
>>> #
>>> #
>>> # This file consists of lines of the form
>>> #
>>> # name = value
>>> #
>>> # (The "=" is optional.) Whitespace may be used. Comments are
>>> # introduced with "#" anywhere on a line. The complete list of
>>> # parameter names and allowed values can be found in the
>>> # Postgres-XC documentation.
>>> #
>>> # The commented-out settings shown in this file represent the default
>>> # values.
>>> #
>>> # Re-commenting a setting is NOT sufficient to revert it to the default
>>> # value.
>>> #
>>> # You need to restart the server.
>>>
>>>
>>> #------------------------------------------------------------------------------
>>> # GENERAL PARAMETERS
>>>
>>> #------------------------------------------------------------------------------
>>> nodename = 'one' # Specifies the node name.
>>> # (changes requires restart)
>>> #listen_addresses = '*' # Listen addresses of this GTM.
>>> # (changes requires restart)
>>> port = 6666 # Port number of this GTM.
>>> # (changes requires restart)
>>>
>>> #startup = ACT # Start mode. ACT/STANDBY.
>>>
>>>
>>> #------------------------------------------------------------------------------
>>> # GTM STANDBY PARAMETERS
>>>
>>> #------------------------------------------------------------------------------
>>> #Those parameters are effective when GTM is activated as a standby server
>>> #active_host = '' # Listen address of active GTM.
>>> # (changes requires restart)
>>> #active_port = # Port number of active GTM.
>>> # (changes requires restart)
>>>
>>> #---------------------------------------
>>> # OTHER OPTIONS
>>> #---------------------------------------
>>> #keepalives_idle = 0 # Keepalives_idle parameter.
>>> #keepalives_interval = 0 # Keepalives_interval parameter.
>>> #keepalives_count = 0 # Keepalives_count internal parameter.
>>> #log_file = 'gtm.log' # Log file name
>>> #log_min_messages = WARNING # log_min_messages. Default WARNING.
>>> # Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3,
>>> # DEBUG2, DEBUG1, INFO, NOTICE, WARNING,
>>> # ERROR, LOG, FATAL, PANIC
>>> #synchronous_backup = off # If backup to standby is synchronous
>>>
>>>
>>> On Mon, Apr 22, 2013 at 12:42 PM, Nikhil Sontakke <ni...@st...>wrote:
>>>
>>>> Can you please share the contents of the gtm's gtm.conf
>>>>
>>>> And also the gtm proxy's gtm_proxy.conf?
>>>>
>>>> Regards,
>>>> Nikhils
>>>>
>>>>
>>>> On Mon, Apr 22, 2013 at 9:34 PM, Anson Abraham <ans...@gm...
>>>> > wrote:
>>>>
>>>>> I was doing a test install of PGXC version 1.0.2.
>>>>>
>>>>> I basically did as such for GTM and GTM_Proxy on same server:
>>>>>
>>>>> /usr/local/pgsql/bin/initgtm -Z gtm -D
>>>>> /var/lib/postgresql/pgxc/data_gtm
>>>>>
>>>>> /usr/local/pgsql/bin/gtm_ctl -Z gtm -D
>>>>> /var/lib/postgresql/pgxc/data_gtm -l /var/log/postgresql/pgxc/gtm.log start
>>>>>
>>>>> which starts. I left the config file as is for now.
>>>>>
>>>>> Then
>>>>>
>>>>> /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D
>>>>> /var/lib/postgresql/pgxc/data_gtm_proxy
>>>>>
>>>>> and i left the config file alone as well and ran:
>>>>>
>>>>> /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D
>>>>> /var/lib/postgresql/pgxc/data_gtm_proxy -l
>>>>> /var/log/postgresql/pgxc/gtm_proxy.log start
>>>>>
>>>>> And I get an error of
>>>>>
>>>>> server starting
>>>>> postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$
>>>>> 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to
>>>>> GTM
>>>>> LOCATION: ConnectGTM, proxy_main.c:3356
>>>>>
>>>>> and it's stuck. Doing a ps -ef i see the gtm running but not the
>>>>> proxy. Am i missing something?
>>>>>
>>>>> I followed the instructions from the wiki and still nothing.
>>>>> Any help here would be really appreciated.
>>>>> -Anson
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Precog is a next-generation analytics platform capable of advanced
>>>>> analytics on semi-structured data. The platform includes APIs for
>>>>> building
>>>>> apps and a phenomenal toolset for data science. Developers can use
>>>>> our toolset for easy data analysis & visualization. Get a free account!
>>>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>>>>> _______________________________________________
>>>>> Postgres-xc-general mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> StormDB - http://www.stormdb.com
>>>> The Database Cloud
>>>>
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
> --
> StormDB - http://www.stormdb.com
> The Database Cloud
>
|
|
From: Nikhil S. <ni...@st...> - 2013-04-23 04:41:57
|
You did not share the contents of gtm_proxy.conf. Also double-check the values of gtm_host and gtm_port parameters in the coordinator's postgresql.conf. If this is on a Linux platform, check if iptables/firewall is blocking connection to port 6666 or something as well. HTH, Nikhils On Tue, Apr 23, 2013 at 1:24 AM, Anson Abraham <ans...@gm...>wrote: > Also when I started up coodinator I get error also: > > LOG: database system was interrupted; last known up at 2013-04-22 > 15:14:08 EDT > LOG: database system was not properly shut down; automatic recovery in > progress > LOG: record with zero length at 0/16E4EA0 > LOG: redo is not required > LOG: database system is ready to accept connections > LOG: autovacuum launcher started > WARNING: can not connect to GTM: Success > WARNING: can not connect to GTM: Bad file descriptor > WARNING: Xid is invalid. > WARNING: can not connect to GTM: Bad file descriptor > WARNING: can not connect to GTM: Bad file descriptor > WARNING: Xid is invalid. > WARNING: can not connect to GTM: Bad file descriptor > WARNING: can not connect to GTM: Bad file descriptor > ERROR: GTM error, could not obtain snapshot > WARNING: can not connect to GTM: Bad file descriptor > WARNING: can not connect to GTM: Bad file descriptor > WARNING: Xid is invalid. > WARNING: can not connect to GTM: Bad file descriptor > WARNING: can not connect to GTM: Bad file descriptor > WARNING: Xid is invalid. > WARNING: can not connect to GTM: Bad file descriptor > WARNING: can not connect to GTM: Bad file descriptor > ERROR: GTM error, could not obtain snapshot > > > I started the GTM initially as such: > /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm > -l /var/log/postgresql/gtm.log start > > But then I opted for: > /usr/local/pgsql/bin/gtm -D /var/lib/postgresql/pgxc/data_gtm > > /var/log/postgresql/gtm.log 2>&1 & > > And still no errors in the log. > > > On Mon, Apr 22, 2013 at 1:57 PM, Anson Abraham <ans...@gm...>wrote: > >> # specified by -D command line option of gtm or gtm_ctl. The >> # configuration file name must be "gtm.conf" >> # >> # >> # This file consists of lines of the form >> # >> # name = value >> # >> # (The "=" is optional.) Whitespace may be used. Comments are >> # introduced with "#" anywhere on a line. The complete list of >> # parameter names and allowed values can be found in the >> # Postgres-XC documentation. >> # >> # The commented-out settings shown in this file represent the default >> # values. >> # >> # Re-commenting a setting is NOT sufficient to revert it to the default >> # value. >> # >> # You need to restart the server. >> >> >> #------------------------------------------------------------------------------ >> # GENERAL PARAMETERS >> >> #------------------------------------------------------------------------------ >> nodename = 'one' # Specifies the node name. >> # (changes requires restart) >> #listen_addresses = '*' # Listen addresses of this GTM. >> # (changes requires restart) >> port = 6666 # Port number of this GTM. >> # (changes requires restart) >> >> #startup = ACT # Start mode. ACT/STANDBY. >> >> >> #------------------------------------------------------------------------------ >> # GTM STANDBY PARAMETERS >> >> #------------------------------------------------------------------------------ >> #Those parameters are effective when GTM is activated as a standby server >> #active_host = '' # Listen address of active GTM. >> # (changes requires restart) >> #active_port = # Port number of active GTM. >> # (changes requires restart) >> >> #--------------------------------------- >> # OTHER OPTIONS >> #--------------------------------------- >> #keepalives_idle = 0 # Keepalives_idle parameter. >> #keepalives_interval = 0 # Keepalives_interval parameter. >> #keepalives_count = 0 # Keepalives_count internal parameter. >> #log_file = 'gtm.log' # Log file name >> #log_min_messages = WARNING # log_min_messages. Default WARNING. >> # Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3, >> # DEBUG2, DEBUG1, INFO, NOTICE, WARNING, >> # ERROR, LOG, FATAL, PANIC >> #synchronous_backup = off # If backup to standby is synchronous >> >> >> On Mon, Apr 22, 2013 at 12:42 PM, Nikhil Sontakke <ni...@st...>wrote: >> >>> Can you please share the contents of the gtm's gtm.conf >>> >>> And also the gtm proxy's gtm_proxy.conf? >>> >>> Regards, >>> Nikhils >>> >>> >>> On Mon, Apr 22, 2013 at 9:34 PM, Anson Abraham <ans...@gm...>wrote: >>> >>>> I was doing a test install of PGXC version 1.0.2. >>>> >>>> I basically did as such for GTM and GTM_Proxy on same server: >>>> >>>> /usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgresql/pgxc/data_gtm >>>> >>>> /usr/local/pgsql/bin/gtm_ctl -Z gtm -D >>>> /var/lib/postgresql/pgxc/data_gtm -l /var/log/postgresql/pgxc/gtm.log start >>>> >>>> which starts. I left the config file as is for now. >>>> >>>> Then >>>> >>>> /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D >>>> /var/lib/postgresql/pgxc/data_gtm_proxy >>>> >>>> and i left the config file alone as well and ran: >>>> >>>> /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D >>>> /var/lib/postgresql/pgxc/data_gtm_proxy -l >>>> /var/log/postgresql/pgxc/gtm_proxy.log start >>>> >>>> And I get an error of >>>> >>>> server starting >>>> postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$ >>>> 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to >>>> GTM >>>> LOCATION: ConnectGTM, proxy_main.c:3356 >>>> >>>> and it's stuck. Doing a ps -ef i see the gtm running but not the proxy. >>>> Am i missing something? >>>> >>>> I followed the instructions from the wiki and still nothing. >>>> Any help here would be really appreciated. >>>> -Anson >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Precog is a next-generation analytics platform capable of advanced >>>> analytics on semi-structured data. The platform includes APIs for >>>> building >>>> apps and a phenomenal toolset for data science. Developers can use >>>> our toolset for easy data analysis & visualization. Get a free account! >>>> http://www2.precog.com/precogplatform/slashdotnewsletter >>>> _______________________________________________ >>>> Postgres-xc-general mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>>> >>>> >>> >>> >>> -- >>> StormDB - http://www.stormdb.com >>> The Database Cloud >>> >> >> > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Anson A. <ans...@gm...> - 2013-04-22 19:55:14
|
Also when I started up coodinator I get error also: LOG: database system was interrupted; last known up at 2013-04-22 15:14:08 EDT LOG: database system was not properly shut down; automatic recovery in progress LOG: record with zero length at 0/16E4EA0 LOG: redo is not required LOG: database system is ready to accept connections LOG: autovacuum launcher started WARNING: can not connect to GTM: Success WARNING: can not connect to GTM: Bad file descriptor WARNING: Xid is invalid. WARNING: can not connect to GTM: Bad file descriptor WARNING: can not connect to GTM: Bad file descriptor WARNING: Xid is invalid. WARNING: can not connect to GTM: Bad file descriptor WARNING: can not connect to GTM: Bad file descriptor ERROR: GTM error, could not obtain snapshot WARNING: can not connect to GTM: Bad file descriptor WARNING: can not connect to GTM: Bad file descriptor WARNING: Xid is invalid. WARNING: can not connect to GTM: Bad file descriptor WARNING: can not connect to GTM: Bad file descriptor WARNING: Xid is invalid. WARNING: can not connect to GTM: Bad file descriptor WARNING: can not connect to GTM: Bad file descriptor ERROR: GTM error, could not obtain snapshot I started the GTM initially as such: /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm -l /var/log/postgresql/gtm.log start But then I opted for: /usr/local/pgsql/bin/gtm -D /var/lib/postgresql/pgxc/data_gtm > /var/log/postgresql/gtm.log 2>&1 & And still no errors in the log. On Mon, Apr 22, 2013 at 1:57 PM, Anson Abraham <ans...@gm...>wrote: > # specified by -D command line option of gtm or gtm_ctl. The > # configuration file name must be "gtm.conf" > # > # > # This file consists of lines of the form > # > # name = value > # > # (The "=" is optional.) Whitespace may be used. Comments are > # introduced with "#" anywhere on a line. The complete list of > # parameter names and allowed values can be found in the > # Postgres-XC documentation. > # > # The commented-out settings shown in this file represent the default > # values. > # > # Re-commenting a setting is NOT sufficient to revert it to the default > # value. > # > # You need to restart the server. > > > #------------------------------------------------------------------------------ > # GENERAL PARAMETERS > > #------------------------------------------------------------------------------ > nodename = 'one' # Specifies the node name. > # (changes requires restart) > #listen_addresses = '*' # Listen addresses of this GTM. > # (changes requires restart) > port = 6666 # Port number of this GTM. > # (changes requires restart) > > #startup = ACT # Start mode. ACT/STANDBY. > > > #------------------------------------------------------------------------------ > # GTM STANDBY PARAMETERS > > #------------------------------------------------------------------------------ > #Those parameters are effective when GTM is activated as a standby server > #active_host = '' # Listen address of active GTM. > # (changes requires restart) > #active_port = # Port number of active GTM. > # (changes requires restart) > > #--------------------------------------- > # OTHER OPTIONS > #--------------------------------------- > #keepalives_idle = 0 # Keepalives_idle parameter. > #keepalives_interval = 0 # Keepalives_interval parameter. > #keepalives_count = 0 # Keepalives_count internal parameter. > #log_file = 'gtm.log' # Log file name > #log_min_messages = WARNING # log_min_messages. Default WARNING. > # Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3, > # DEBUG2, DEBUG1, INFO, NOTICE, WARNING, > # ERROR, LOG, FATAL, PANIC > #synchronous_backup = off # If backup to standby is synchronous > > > On Mon, Apr 22, 2013 at 12:42 PM, Nikhil Sontakke <ni...@st...>wrote: > >> Can you please share the contents of the gtm's gtm.conf >> >> And also the gtm proxy's gtm_proxy.conf? >> >> Regards, >> Nikhils >> >> >> On Mon, Apr 22, 2013 at 9:34 PM, Anson Abraham <ans...@gm...>wrote: >> >>> I was doing a test install of PGXC version 1.0.2. >>> >>> I basically did as such for GTM and GTM_Proxy on same server: >>> >>> /usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgresql/pgxc/data_gtm >>> >>> /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm >>> -l /var/log/postgresql/pgxc/gtm.log start >>> >>> which starts. I left the config file as is for now. >>> >>> Then >>> >>> /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D >>> /var/lib/postgresql/pgxc/data_gtm_proxy >>> >>> and i left the config file alone as well and ran: >>> >>> /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D >>> /var/lib/postgresql/pgxc/data_gtm_proxy -l >>> /var/log/postgresql/pgxc/gtm_proxy.log start >>> >>> And I get an error of >>> >>> server starting >>> postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$ >>> 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to >>> GTM >>> LOCATION: ConnectGTM, proxy_main.c:3356 >>> >>> and it's stuck. Doing a ps -ef i see the gtm running but not the proxy. >>> Am i missing something? >>> >>> I followed the instructions from the wiki and still nothing. >>> Any help here would be really appreciated. >>> -Anson >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Precog is a next-generation analytics platform capable of advanced >>> analytics on semi-structured data. The platform includes APIs for >>> building >>> apps and a phenomenal toolset for data science. Developers can use >>> our toolset for easy data analysis & visualization. Get a free account! >>> http://www2.precog.com/precogplatform/slashdotnewsletter >>> _______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >>> >> >> >> -- >> StormDB - http://www.stormdb.com >> The Database Cloud >> > > |
|
From: Anson A. <ans...@gm...> - 2013-04-22 17:57:28
|
# specified by -D command line option of gtm or gtm_ctl. The # configuration file name must be "gtm.conf" # # # This file consists of lines of the form # # name = value # # (The "=" is optional.) Whitespace may be used. Comments are # introduced with "#" anywhere on a line. The complete list of # parameter names and allowed values can be found in the # Postgres-XC documentation. # # The commented-out settings shown in this file represent the default # values. # # Re-commenting a setting is NOT sufficient to revert it to the default # value. # # You need to restart the server. #------------------------------------------------------------------------------ # GENERAL PARAMETERS #------------------------------------------------------------------------------ nodename = 'one' # Specifies the node name. # (changes requires restart) #listen_addresses = '*' # Listen addresses of this GTM. # (changes requires restart) port = 6666 # Port number of this GTM. # (changes requires restart) #startup = ACT # Start mode. ACT/STANDBY. #------------------------------------------------------------------------------ # GTM STANDBY PARAMETERS #------------------------------------------------------------------------------ #Those parameters are effective when GTM is activated as a standby server #active_host = '' # Listen address of active GTM. # (changes requires restart) #active_port = # Port number of active GTM. # (changes requires restart) #--------------------------------------- # OTHER OPTIONS #--------------------------------------- #keepalives_idle = 0 # Keepalives_idle parameter. #keepalives_interval = 0 # Keepalives_interval parameter. #keepalives_count = 0 # Keepalives_count internal parameter. #log_file = 'gtm.log' # Log file name #log_min_messages = WARNING # log_min_messages. Default WARNING. # Valid value: DEBUG, DEBUG5, DEBUG4, DEBUG3, # DEBUG2, DEBUG1, INFO, NOTICE, WARNING, # ERROR, LOG, FATAL, PANIC #synchronous_backup = off # If backup to standby is synchronous On Mon, Apr 22, 2013 at 12:42 PM, Nikhil Sontakke <ni...@st...>wrote: > Can you please share the contents of the gtm's gtm.conf > > And also the gtm proxy's gtm_proxy.conf? > > Regards, > Nikhils > > > On Mon, Apr 22, 2013 at 9:34 PM, Anson Abraham <ans...@gm...>wrote: > >> I was doing a test install of PGXC version 1.0.2. >> >> I basically did as such for GTM and GTM_Proxy on same server: >> >> /usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgresql/pgxc/data_gtm >> >> /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm >> -l /var/log/postgresql/pgxc/gtm.log start >> >> which starts. I left the config file as is for now. >> >> Then >> >> /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D >> /var/lib/postgresql/pgxc/data_gtm_proxy >> >> and i left the config file alone as well and ran: >> >> /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D >> /var/lib/postgresql/pgxc/data_gtm_proxy -l >> /var/log/postgresql/pgxc/gtm_proxy.log start >> >> And I get an error of >> >> server starting >> postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$ >> 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to >> GTM >> LOCATION: ConnectGTM, proxy_main.c:3356 >> >> and it's stuck. Doing a ps -ef i see the gtm running but not the proxy. >> Am i missing something? >> >> I followed the instructions from the wiki and still nothing. >> Any help here would be really appreciated. >> -Anson >> >> >> >> >> ------------------------------------------------------------------------------ >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> http://www2.precog.com/precogplatform/slashdotnewsletter >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > |
|
From: Nikhil S. <ni...@st...> - 2013-04-22 16:48:52
|
Can you please share the contents of the gtm's gtm.conf And also the gtm proxy's gtm_proxy.conf? Regards, Nikhils On Mon, Apr 22, 2013 at 9:34 PM, Anson Abraham <ans...@gm...>wrote: > I was doing a test install of PGXC version 1.0.2. > > I basically did as such for GTM and GTM_Proxy on same server: > > /usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgresql/pgxc/data_gtm > > /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm > -l /var/log/postgresql/pgxc/gtm.log start > > which starts. I left the config file as is for now. > > Then > > /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D > /var/lib/postgresql/pgxc/data_gtm_proxy > > and i left the config file alone as well and ran: > > /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D > /var/lib/postgresql/pgxc/data_gtm_proxy -l > /var/log/postgresql/pgxc/gtm_proxy.log start > > And I get an error of > > server starting > postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$ > 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to > GTM > LOCATION: ConnectGTM, proxy_main.c:3356 > > and it's stuck. Doing a ps -ef i see the gtm running but not the proxy. > Am i missing something? > > I followed the instructions from the wiki and still nothing. > Any help here would be really appreciated. > -Anson > > > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Anson A. <ans...@gm...> - 2013-04-22 16:04:38
|
I was doing a test install of PGXC version 1.0.2. I basically did as such for GTM and GTM_Proxy on same server: /usr/local/pgsql/bin/initgtm -Z gtm -D /var/lib/postgresql/pgxc/data_gtm /usr/local/pgsql/bin/gtm_ctl -Z gtm -D /var/lib/postgresql/pgxc/data_gtm -l /var/log/postgresql/pgxc/gtm.log start which starts. I left the config file as is for now. Then /usr/local/pgsql/bin/initgtm -Z gtm_proxy -D /var/lib/postgresql/pgxc/data_gtm_proxy and i left the config file alone as well and ran: /usr/local/pgsql/bin/gtm_ctl -Z gtm_proxy -D /var/lib/postgresql/pgxc/data_gtm_proxy -l /var/log/postgresql/pgxc/gtm_proxy.log start And I get an error of server starting postgres@drupalgtm:/var/lib/postgresql/pgxc/data_gtm_proxy$ 1:139986171815680:2013-04-22 12:02:43.098 EDT -FATAL: can not connect to GTM LOCATION: ConnectGTM, proxy_main.c:3356 and it's stuck. Doing a ps -ef i see the gtm running but not the proxy. Am i missing something? I followed the instructions from the wiki and still nothing. Any help here would be really appreciated. -Anson |
43 messages has been excluded from this view by a project administrator.
