postgres-xc-general Mailing List for Postgres-XC (Page 69)

Hi folks!

First of all, many thanks for the great work! Postgres-XC is exactly what
I've been looking for: a real cluster version of PostgreSQL. We have a
project where I would like to implement it (with time, we will need this
type of performance).

There is one question for which I do not have an answer (the only possible
show-stopper): if a node fails temporarily (or goes down for maintenance),
how does it synchronize with the other servers once it comes back up? We
need 24/7 operation so this is critical to us.

Thanks in advance
--
Olivier Gautherot
ol...@ga...
http://www.linkedin.com/in/ogautherot

On Fri, Sep 30, 2011 at 8:18 PM, Michael Paquier
<mic...@gm...> wrote:
>
> This may be a problem, but I'm not sure.
> Have you tried to set LD_LIBRARY_PATH to connect refer to the path where you
> installed postgres-xc libs?
> Regarding your settings, it should be in /usr/local/pgsql/lib.

An interesting point. I'll check it on Monday when I get back to the office.

ChrisA

On Fri, Sep 30, 2011 at 4:51 PM, Chris Angelico <ro...@gm...> wrote:

> On Fri, Sep 30, 2011 at 4:48 PM, Michael Paquier
> <mic...@gm...> wrote:
> > Retry to start GTM with with command, it may make a difference:
> > gtm -x 1000 -D /usr/local/pgsql/data/gtm/ -p 6667 -l gtm.log &
> > ...
> > This was perhaps autovacuum transactions that were committing on nodes
> with
> > the same transaction ID as a past transaction. A transaction ID being
> > committed twice on a node makes it crash with a FATAL error.
> >
>
> Made that change, thanks. Doesn't appear to have changed anything from
> the state as at my second post; Unix socket isn't working but TCP
> socket is. It's worth being correct, anyhow.
>
> Is it a consideration that I'm using libpqxx?
>
This may be a problem, but I'm not sure.
Have you tried to set LD_LIBRARY_PATH to connect refer to the path where you
installed postgres-xc libs?
Regarding your settings, it should be in /usr/local/pgsql/lib.
-- 
Michael Paquier
http://michael.otacoo.com

On Fri, Sep 30, 2011 at 4:48 PM, Michael Paquier
<mic...@gm...> wrote:
> Retry to start GTM with with command, it may make a difference:
> gtm -x 1000 -D /usr/local/pgsql/data/gtm/ -p 6667 -l gtm.log &
> ...
> This was perhaps autovacuum transactions that were committing on nodes with
> the same transaction ID as a past transaction. A transaction ID being
> committed twice on a node makes it crash with a FATAL error.
>

Made that change, thanks. Doesn't appear to have changed anything from
the state as at my second post; Unix socket isn't working but TCP
socket is. It's worth being correct, anyhow.

Is it a consideration that I'm using libpqxx?

Chris Angelico

Hi,

On Fri, Sep 30, 2011 at 2:49 PM, Chris Angelico <ro...@gm...> wrote:

> $ gtm -D /usr/local/pgsql/data/gtm/ -p 6667 -l gtm.log &
>
You should start gtm with a minimum transaction ID value sufficient to avoid
conflicts by what has been created with initdb, which creates all the
default objects of a database in a data folder. initdb itself uses
transaction IDs up to 708 if I recall, so you need to start GTM with a value
superior than that.

Retry to start GTM with with command, it may make a difference:
gtm* -x 1000* -D /usr/local/pgsql/data/gtm/ -p 6667 -l gtm.log &

x option on GTM specifies from which transaction ID value GTM should begin
to feed all the cluster nodes.

$ tail -F /usr/local/pgsql/data/gtm/gtm.log
> $ postgres -D /usr/local/pgsql/data -p 15432 -i -X
> $ postgres -D /usr/local/pgsql/data1 -i -C
>
This looks correct.

>
> The command lines were partly derived from
> http://sourceforge.net/mailarchive/message.php?msg_id=26377904 but I
> don't really know what port numbers I should use. In
> /usr/local/pgsql/data/postgresql.conf are a number of references to
> 'port', most of them commented; the only one that's active is:
> gtm_port 6667
>
> When I tried to fire up gtm on port 6666, it immediately terminated
> without message. I assume this means that it couldn't bind to the
> port, but I have no way to confirm this.
>
It think 6666 is the default port value is by pooler on Coordinator,
therefore it may not work.

>
> There's a similar conf file in .../data1/ which I have now also edited
> to put gtm_port to 6667. Without this change, I was getting total
> inability to connect; now, I still get a lack of pooled commections.
>
> $ sudo su postgres
> $ psql
> psql (9.1beta2)
> Type "help" for help.
> postgres=# create user chris with password 'chris';
> ERROR:  Failed to get pooled connections
> PANIC:  cannot abort transaction 45, it was already committed
> server closed the connection unexpectedly
>        This probably means the server terminated abnormally
>        before or while processing the request.
> The connection to the server was lost. Attempting reset: Failed.
> !>
>
> Meanwhile, the other logs show odd messages; mainly, complaining that
> the database was improperly shut down.
>
This was perhaps autovacuum transactions that were committing on nodes with
the same transaction ID as a past transaction. A transaction ID being
committed twice on a node makes it crash with a FATAL error.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Update! It's now almost all working - I'd mucked up the config files
(and then in subsequent editing, didn't realise the errors as I
mentally assumed that commented-out entries were just restating the
defaults). There's just one small issue: Unix sockets support. If I
use "host=::1" in the connection string, my program can connect no
trouble; leaving it off throws an error "No such file or directory"
attempting to connect to "/var/run/postgresql/.s.PGSQL.5432".
Unfortunately I haven't found where that's configured... it's probably
something really obvious that I've missed!

Sorry to bother you over nothing. :)

Chris Angelico

Greetings!

I'm attempting to set up a test "cluster" on a single computer, for
testing purposes. It's the first experience I have with PostgreSQL-XC,
and I know I've mucked stuff up, but I'm hoping someone can help me
sort out the mess I've got myself into.

So far, I've cloned the source tree and configured it --prefix=/usr
(this may very well have been an unwise choice). I then created two
data directories at /usr/local/pgsql/data and /usr/local/pgsql/data1,
and attempted to launch the three parts of the server, in separate
windows. All three sessions are running on the same computer (a Dell
laptop running 64-bit Ubuntu 10.10), and are invoked from 'sudo su
postgres' to get to the expected user.

$ gtm -D /usr/local/pgsql/data/gtm/ -p 6667 -l gtm.log &
$ tail -F /usr/local/pgsql/data/gtm/gtm.log

$ postgres -D /usr/local/pgsql/data -p 15432 -i -X

$ postgres -D /usr/local/pgsql/data1 -i -C

The command lines were partly derived from
http://sourceforge.net/mailarchive/message.php?msg_id=26377904 but I
don't really know what port numbers I should use. In
/usr/local/pgsql/data/postgresql.conf are a number of references to
'port', most of them commented; the only one that's active is:
gtm_port 6667

When I tried to fire up gtm on port 6666, it immediately terminated
without message. I assume this means that it couldn't bind to the
port, but I have no way to confirm this.

There's a similar conf file in .../data1/ which I have now also edited
to put gtm_port to 6667. Without this change, I was getting total
inability to connect; now, I still get a lack of pooled commections.

$ sudo su postgres
$ psql
psql (9.1beta2)
Type "help" for help.
postgres=# create user chris with password 'chris';
ERROR:  Failed to get pooled connections
PANIC:  cannot abort transaction 45, it was already committed
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

Meanwhile, the other logs show odd messages; mainly, complaining that
the database was improperly shut down.

Any assistance would be much appreciated!

Chris Angelico

Hi,

Sorry for the late reply on this topic.
Here is a little bit of feedback about your problem.

After looking closer at what was currently missing in Postgres-XC to support
SERIAL, I found that some basic mechanism to feed values on Coordinator for
stable and volatile functions is still missing.
For example, the lack of support for SERIAL is not only limited to the query
itself, it is a more fundamental error that happens for all the queries of
the type:
select * from table where column_1 = volatile_function('one_value');
By that I mean SELECT queries using functions in their WHERE clause that
cannot be evaluated on Datanodes and values have to be taken from
Coordinator to insure data consistency among the cluster. This is also the
case for INSERT as values are not determined correctly on Coordinator. This
is also the case for DELETE and UPDATE clauses using non-immutable functions
in their WHERE clause.

For example, a simple INSERT query symbolizing this problem would be:
create table aa (int serial) distribute by replication;
insert into aa values (nextval('sequ'));
insert into aa values (DEFAULT);
In those cases we need to calculate the value on Coordinator and then
distribute the same value among nodes.

>From August, we will begin to investigate those problems. But it needs some
fundamental work and may need a bit of time.
However, once such a mechanism will be in place, supporting serial tables is
a piece of cake.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Op 15-07-11 13:27, Michael Paquier schreef:
> On Fri, Jul 15, 2011 at 8:01 PM, David Hartveld
> <dav...@me... <mailto:dav...@me...>> wrote:
>
>     After playing around with postgres-xc 0.9.5, I wanted to give my actual
>     application a try. I get the following error message when setting up the
>     base schema: "ERROR: Postgres-XC does not support DEFAULT with
>     non-immutable functions yet". This happens when setting up a column as
>     follows (this is btw copied verbatim from the postgresql documentation,
>     explaining how a serial type works):
>
>     CREATE SEQUENCE tablename_colname_seq;
>     CREATE TABLE tablename (
>     colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
>     );
>     ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
>
>     I've read that (global) sequences and serial are not yet supported. I
>     checked the roadmap and couldn't find mention of this explicitly. I was
>     wondering if support is planned, and if so, when? (And if not, why :-).
>
>
> Supporting serial and default values with nextval is pretty complicated
> because sequences are only created on Coordinators. It is not currently
> taken into account in the development plans.
> However, I am working currently on the support of this feature because
> me too I believe it is important to support it.
> My patch is half-way done, and needs some rework on the planner side.
> I may have some results within the next couple of weeks.

I understand the problem, however, the application for which I'm 
experimenting does need the functionality (currently). I might be 
interested in testing your patches, do let me know if you need some 
testing done!

Greetings,
David

On Fri, Jul 15, 2011 at 8:01 PM, David Hartveld
<dav...@me...>wrote:

> After playing around with postgres-xc 0.9.5, I wanted to give my actual
> application a try. I get the following error message when setting up the
> base schema: "ERROR:  Postgres-XC does not support DEFAULT with
> non-immutable functions yet". This happens when setting up a column as
> follows (this is btw copied verbatim from the postgresql documentation,
> explaining how a serial type works):
>
> CREATE SEQUENCE tablename_colname_seq;
> CREATE TABLE tablename (
>     colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
> );
> ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
>
> I've read that (global) sequences and serial are not yet supported. I
> checked the roadmap and couldn't find mention of this explicitly. I was
> wondering if support is planned, and if so, when? (And if not, why :-).
>

Supporting serial and default values with nextval is pretty complicated
because sequences are only created on Coordinators. It is not currently
taken into account in the development plans.
However, I am working currently on the support of this feature because me
too I believe it is important to support it.
My patch is half-way done, and needs some rework on the planner side.
I may have some results within the next couple of weeks.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

After playing around with postgres-xc 0.9.5, I wanted to give my actual 
application a try. I get the following error message when setting up the 
base schema: "ERROR:  Postgres-XC does not support DEFAULT with 
non-immutable functions yet". This happens when setting up a column as 
follows (this is btw copied verbatim from the postgresql documentation, 
explaining how a serial type works):

CREATE SEQUENCE tablename_colname_seq;
CREATE TABLE tablename (
     colname integer NOT NULL DEFAULT nextval('tablename_colname_seq')
);
ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;

I've read that (global) sequences and serial are not yet supported. I 
checked the roadmap and couldn't find mention of this explicitly. I was 
wondering if support is planned, and if so, when? (And if not, why :-).

Thanks,
David

On Mon, Jul 11, 2011 at 6:22 PM, David Hartveld
<dav...@me...>wrote:

> Op 09-07-11 04:13, Michael Paquier schreef:
>
>> Hi
>>
>> On Fri, Jul 8, 2011 at 9:04 PM, David Hartveld
>> <dav...@me... <mailto:david.hartveld@mendix.**com<dav...@me...>>>
>> wrote:
>>
>>    Hi all,
>>
>>    I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze,
>>    with a build from the 0.9.5 sources. I think I've set up a cluster
>>    with three hw systems, two nodes (data node/coordinator each) and a
>>    gtm node, properly, following the installation manual.
>>
>>    However, the data nodes complain when they are started, every so
>>    many seconds: "WARNING: Do not have a GTM snapshot available".
>>
>> This bug is related with autovacuum launcher which is not able to get a
>> valid snapshot from GTM. I am studying this test case and may have a fix
>> for that next week.
>>
>
> Is it a symptom of the nodes not being able to connect to the
> coordinators?
>
Snapshots and GXIDs are fed from GTM. So it is not directly related.
It is a problem with the internal process autovacuum is using to try to get
a snapshot from GTM, and is not related at all with your setting problems.


>
>
>
>>    And when I connect to a coordinator and try to execute 'CREATE
>>    DATABASE mydb;', I get the message: "ERROR: Failed to get pooled
>>    connections". I've attached the configuration files that I am using
>>    for coordinator and datanode (the difference between nodes 1 and 2
>>    being only that pgxc_node_id is 1 and 2 respectively). The GTM is
>>    started with "$GTM -x 628 -p 5000 -D $DATADIR -l $LOGFILE &".
>>
>> Your configuration files look OK.
>> However, a datanode is a node that do not interact with other nodes
>> through a connection pooling, so you do not need to set data_node_hosts,
>> data_node_ports, pooler_port in its configuration file. num_data_nodes
>> also in unnecessary
>>
>>
>>    Any suggestions what I should do or try?
>>
>> Have you started your nodes with -i option so as they accept TCP-IP
>> connections?
>>
>
> They already were, but I actually configured the wrong hostnames. I've now
> reconfigured with the correct IPs.

OK, that's cool if it works.

-- 
Michael Paquier
http://michael.otacoo.com

Op 11-07-11 11:22, David Hartveld schreef:
> Op 11-07-11 07:17, Abbas Butt schreef:
>> I noticed that you have given 2 as pgxc_node_id in both the files you
>> have attached. Please make sure you have given different numbers to all
>> the 4 nodes.
>
> I need a little clarification for this point. In the installation manual
> is suggested that pgxc_node_id is the index into the coordinator_hosts
> and coordinator_ports arrays, as well as into datanode_hosts /
> datanode_ports. But you suggest i assign unique numbers to each node. If
> it should be the latter, this could be improved in the installation manual.

Following up on my own post, coordinators and data nodes each have their 
own sequence of index numbers, all starting at 1, up to n (the number of 
coordinators and data nodes respectively).

After pointing the correct indices to the correct hosts (in the 
coordinator_hosts and data_node_hosts arrays), I've gotten the 
distributed cluster to create a database on one node, and to connect to 
it to the other. I'm now going to try to run a more extensive test, and 
will let you know the results if anything interesting happens ;-)

Thank you all for your help,
David Hartveld

Op 09-07-11 04:13, Michael Paquier schreef:
> Hi
>
> On Fri, Jul 8, 2011 at 9:04 PM, David Hartveld
> <dav...@me... <mailto:dav...@me...>> wrote:
>
>     Hi all,
>
>     I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze,
>     with a build from the 0.9.5 sources. I think I've set up a cluster
>     with three hw systems, two nodes (data node/coordinator each) and a
>     gtm node, properly, following the installation manual.
>
>     However, the data nodes complain when they are started, every so
>     many seconds: "WARNING: Do not have a GTM snapshot available".
>
> This bug is related with autovacuum launcher which is not able to get a
> valid snapshot from GTM. I am studying this test case and may have a fix
> for that next week.

Is it a symptom of the nodes not being able to connect to the coordinators?

>
>     And when I connect to a coordinator and try to execute 'CREATE
>     DATABASE mydb;', I get the message: "ERROR: Failed to get pooled
>     connections". I've attached the configuration files that I am using
>     for coordinator and datanode (the difference between nodes 1 and 2
>     being only that pgxc_node_id is 1 and 2 respectively). The GTM is
>     started with "$GTM -x 628 -p 5000 -D $DATADIR -l $LOGFILE &".
>
> Your configuration files look OK.
> However, a datanode is a node that do not interact with other nodes
> through a connection pooling, so you do not need to set data_node_hosts,
> data_node_ports, pooler_port in its configuration file. num_data_nodes
> also in unnecessary
>
>
>     Any suggestions what I should do or try?
>
> Have you started your nodes with -i option so as they accept TCP-IP
> connections?

They already were, but I actually configured the wrong hostnames. I've 
now reconfigured with the correct IPs.

> As Lionel said, pg_hba.conf setting is full of traps. You should check
> that also.

"host all all samenet trust" should do the trick, I guess?

Thanks for your input!
David

Op 11-07-11 07:17, Abbas Butt schreef:
> I noticed that you have given 2 as pgxc_node_id in both the files you
> have attached. Please make sure you have given different numbers to all
> the 4 nodes.

I need a little clarification for this point. In the installation manual 
is suggested that pgxc_node_id is the index into the coordinator_hosts 
and coordinator_ports arrays, as well as into datanode_hosts / 
datanode_ports. But you suggest i assign unique numbers to each node. If 
it should be the latter, this could be improved in the installation manual.

> Another suggestion from my side will be to give IP address of GTM
> instead of host name pg-db-01.

Done. I'm not sure this is necessary, as the log file of the GTM shows 
incoming connections on startup of each of the four nodes.

David

Hi,
I noticed that you have given 2 as pgxc_node_id in both the files you have
attached. Please make sure you have given different numbers to all the 4
nodes.

Another suggestion from my side will be to give IP address of GTM instead of
host name pg-db-01.

Regards
Abbas

On Fri, Jul 8, 2011 at 5:04 PM, David Hartveld <dav...@me...>wrote:

> Hi all,
>
> I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze, with a
> build from the 0.9.5 sources. I think I've set up a cluster with three hw
> systems, two nodes (data node/coordinator each) and a gtm node, properly,
> following the installation manual.
>
> However, the data nodes complain when they are started, every so many
> seconds: "WARNING:  Do not have a GTM snapshot available". And when I
> connect to a coordinator and try to execute 'CREATE DATABASE mydb;', I get
> the message: "ERROR:  Failed to get pooled connections". I've attached the
> configuration files that I am using for coordinator and datanode (the
> difference between nodes 1 and 2 being only that pgxc_node_id is 1 and 2
> respectively). The GTM is started with "$GTM -x 628 -p 5000 -D $DATADIR -l
> $LOGFILE &".
>
> Any suggestions what I should do or try?
>
> Greetings,
> David
>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

Hi

On Fri, Jul 8, 2011 at 9:04 PM, David Hartveld <dav...@me...>wrote:

> Hi all,
>
> I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze, with a
> build from the 0.9.5 sources. I think I've set up a cluster with three hw
> systems, two nodes (data node/coordinator each) and a gtm node, properly,
> following the installation manual.
>
> However, the data nodes complain when they are started, every so many
> seconds: "WARNING:  Do not have a GTM snapshot available".
>
This bug is related with autovacuum launcher which is not able to get a
valid snapshot from GTM. I am studying this test case and may have a fix for
that next week.

> And when I connect to a coordinator and try to execute 'CREATE DATABASE
> mydb;', I get the message: "ERROR:  Failed to get pooled connections". I've
> attached the configuration files that I am using for coordinator and
> datanode (the difference between nodes 1 and 2 being only that pgxc_node_id
> is 1 and 2 respectively). The GTM is started with "$GTM -x 628 -p 5000 -D
> $DATADIR -l $LOGFILE &".
>
Your configuration files look OK.
However, a datanode is a node that do not interact with other nodes through
a connection pooling, so you do not need to set data_node_hosts,
data_node_ports, pooler_port in its configuration file. num_data_nodes also
in unnecessary


> Any suggestions what I should do or try?
>
Have you started your nodes with -i option so as they accept TCP-IP
connections?
As Lionel said, pg_hba.conf setting is full of traps. You should check that
also.

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Hello,

(from my own experience) ensure that nothing is blocking incoming
connections (like tcpwrappers => /etc/hosts.allow, /etc/hosts.deny, or
firewall). ACL of pg_ha.conf could be part of the solution there.

You could ensure everything is connected correctly by issuing 'netstat -apn'
on each node, and looking for ESTABLISHED connections from/to GTM port and
coordinator ports.

Regards

Lionel Frachon

2011/7/8 David Hartveld <dav...@me...>

> Hi all,
>
> I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze, with a
> build from the 0.9.5 sources. I think I've set up a cluster with three hw
> systems, two nodes (data node/coordinator each) and a gtm node, properly,
> following the installation manual.
>
> However, the data nodes complain when they are started, every so many
> seconds: "WARNING:  Do not have a GTM snapshot available". And when I
> connect to a coordinator and try to execute 'CREATE DATABASE mydb;', I get
> the message: "ERROR:  Failed to get pooled connections". I've attached the
> configuration files that I am using for coordinator and datanode (the
> difference between nodes 1 and 2 being only that pgxc_node_id is 1 and 2
> respectively). The GTM is started with "$GTM -x 628 -p 5000 -D $DATADIR -l
> $LOGFILE &".
>
> Any suggestions what I should do or try?
>
> Greetings,
> David
>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>

Hi all,

I'm experimenting a bit with postgres-xc 0.9.5 on debian squeeze, with a 
build from the 0.9.5 sources. I think I've set up a cluster with three 
hw systems, two nodes (data node/coordinator each) and a gtm node, 
properly, following the installation manual.

However, the data nodes complain when they are started, every so many 
seconds: "WARNING:  Do not have a GTM snapshot available". And when I 
connect to a coordinator and try to execute 'CREATE DATABASE mydb;', I 
get the message: "ERROR:  Failed to get pooled connections". I've 
attached the configuration files that I am using for coordinator and 
datanode (the difference between nodes 1 and 2 being only that 
pgxc_node_id is 1 and 2 respectively). The GTM is started with "$GTM -x 
628 -p 5000 -D $DATADIR -l $LOGFILE &".

Any suggestions what I should do or try?

Greetings,
David

On Tue, Jun 7, 2011 at 11:35 PM, Lionel Frachon <lio...@gm...>wrote:

> Hi,
>
> Vacuum did not solve the problem.
>
I looks to be a deeper problem than I expected related to prepared
transactions in JDBC.

>
> I did a workaround for the problem by loading directly files through "copy
> <table> from <file.csv>" from coordinator, problem did not appear again (and
> data is distributed correctly imho).
>
> Should I enter a bug anyway regarding jdbc bulk/quick inserts ?
>
Yes. If you could fill in a bug report in the bug tracker of the project, it
is definitely helpful.
Just don't forget to add the tests you used, the steps you made to reproduce
the problem and what are the problems.
-- 
Michael Paquier
http://michael.otacoo.com

Hi,

Vacuum did not solve the problem.

I did a workaround for the problem by loading directly files through "copy
<table> from <file.csv>" from coordinator, problem did not appear again (and
data is distributed correctly imho).

Should I enter a bug anyway regarding jdbc bulk/quick inserts ?

Thx for your help

Lionel F.


2011/6/7 Lionel Frachon <lio...@gm...>

> Hello,
>
> ran gtm with -x 1025, the same problem appears.
> (ERROR:  prepared transaction with identifier "T1530" does not exist
> STATEMENT:  COMMIT PREPARED 'T1530')
>
> I'm shutting down autovacuum on nodes to see if problem persists (and
> re-enable debug1 tracing)
>
> Regards
>
> Lionel F.
>
>
>
> 2011/6/7 Michael Paquier <mic...@gm...>
>
>>
>>
>> On Mon, Jun 6, 2011 at 9:39 PM, Lionel Frachon <lio...@gm...>wrote:
>>
>>> Hello,
>>>
>>> looking at the debug1 mode log on datanode3, I found some interesting
>>> points hereafter (vacuum on, max_prepared_transactions=5000):
>>>
>>> (with normal inserts)
>>> [....]
>>> DEBUG:  unset snapshot info
>>> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  [re]setting xid = 0, old_value = 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 102
>>> DEBUG:  [re]setting xid = 102, old_value = 0
>>> DEBUG:  TransactionId = 102
>>> DEBUG:  xid (102) does not follow ShmemVariableCache->nextXid (665)
>>> DEBUG:  Record transaction commit 101
>>> DEBUG:  Record transaction commit 102
>>> DEBUG:  [re]setting xid = 0, old_value = 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 103
>>> DEBUG:  [re]setting xid = 103, old_value = 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
>>> DEBUG:  TransactionId = 103
>>> DEBUG:  xid (103) does not follow ShmemVariableCache->nextXid (665)
>>> DEBUG:  unset snapshot info
>>> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
>>>
>>> While inserting with dsitrubuted hashed keys :
>>>
>>> [...]
>>> DEBUG:  [re]setting xid = 0, old_value = 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 522
>>> DEBUG:  [re]setting xid = 522, old_value = 0
>>> DEBUG:  TransactionId = 522
>>> DEBUG:  xid (522) does not follow ShmemVariableCache->nextXid (665)
>>> DEBUG:  Record transaction commit 521
>>> DEBUG:  Record transaction commit 522
>>> DEBUG:  [re]setting xid = 0, old_value = 0
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 524
>>> DEBUG:  [re]setting xid = 524, old_value = 0
>>> ERROR:  prepared transaction with identifier "T523" does not exist
>>> STATEMENT:  COMMIT PREPARED 'T523'
>>> DEBUG:  [re]setting xid = 0, old_value = 524
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 526
>>> DEBUG:  [re]setting xid = 526, old_value = 0
>>> ERROR:  prepared transaction with identifier "T525" does not exist
>>> STATEMENT:  COMMIT PREPARED 'T525'
>>> DEBUG:  [re]setting xid = 0, old_value = 526
>>> DEBUG:  unset snapshot info
>>> DEBUG:  Received new gxid 528
>>> DEBUG:  [re]setting xid = 528, old_value = 0
>>> ERROR:  prepared transaction with identifier "T527" does not exist
>>> [...]
>>>
>> You are right.
>> But this log:
>>
>> DEBUG:  Received new gxid 103
>> means that GTM is feeding cluster in transaction ID to a very low value.
>> This may lead to visibility problems.
>> You should start GTM with an option like -x 1000 to be sure that it
>> doesn't feed transaction IDs lower than 628.
>> --
>> Michael Paquier
>> http://michael.otacoo.com
>>
>
>

Hello,

ran gtm with -x 1025, the same problem appears.
(ERROR:  prepared transaction with identifier "T1530" does not exist
STATEMENT:  COMMIT PREPARED 'T1530')

I'm shutting down autovacuum on nodes to see if problem persists (and
re-enable debug1 tracing)

Regards

Lionel F.


2011/6/7 Michael Paquier <mic...@gm...>

>
>
> On Mon, Jun 6, 2011 at 9:39 PM, Lionel Frachon <lio...@gm...>wrote:
>
>> Hello,
>>
>> looking at the debug1 mode log on datanode3, I found some interesting
>> points hereafter (vacuum on, max_prepared_transactions=5000):
>>
>> (with normal inserts)
>> [....]
>> DEBUG:  unset snapshot info
>> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
>> DEBUG:  unset snapshot info
>> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
>> DEBUG:  unset snapshot info
>> DEBUG:  [re]setting xid = 0, old_value = 0
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 102
>> DEBUG:  [re]setting xid = 102, old_value = 0
>> DEBUG:  TransactionId = 102
>> DEBUG:  xid (102) does not follow ShmemVariableCache->nextXid (665)
>> DEBUG:  Record transaction commit 101
>> DEBUG:  Record transaction commit 102
>> DEBUG:  [re]setting xid = 0, old_value = 0
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 103
>> DEBUG:  [re]setting xid = 103, old_value = 0
>> DEBUG:  unset snapshot info
>> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
>> DEBUG:  TransactionId = 103
>> DEBUG:  xid (103) does not follow ShmemVariableCache->nextXid (665)
>> DEBUG:  unset snapshot info
>> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
>>
>> While inserting with dsitrubuted hashed keys :
>>
>> [...]
>> DEBUG:  [re]setting xid = 0, old_value = 0
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 522
>> DEBUG:  [re]setting xid = 522, old_value = 0
>> DEBUG:  TransactionId = 522
>> DEBUG:  xid (522) does not follow ShmemVariableCache->nextXid (665)
>> DEBUG:  Record transaction commit 521
>> DEBUG:  Record transaction commit 522
>> DEBUG:  [re]setting xid = 0, old_value = 0
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 524
>> DEBUG:  [re]setting xid = 524, old_value = 0
>> ERROR:  prepared transaction with identifier "T523" does not exist
>> STATEMENT:  COMMIT PREPARED 'T523'
>> DEBUG:  [re]setting xid = 0, old_value = 524
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 526
>> DEBUG:  [re]setting xid = 526, old_value = 0
>> ERROR:  prepared transaction with identifier "T525" does not exist
>> STATEMENT:  COMMIT PREPARED 'T525'
>> DEBUG:  [re]setting xid = 0, old_value = 526
>> DEBUG:  unset snapshot info
>> DEBUG:  Received new gxid 528
>> DEBUG:  [re]setting xid = 528, old_value = 0
>> ERROR:  prepared transaction with identifier "T527" does not exist
>> [...]
>>
> You are right.
> But this log:
>
> DEBUG:  Received new gxid 103
> means that GTM is feeding cluster in transaction ID to a very low value.
> This may lead to visibility problems.
> You should start GTM with an option like -x 1000 to be sure that it doesn't
> feed transaction IDs lower than 628.
> --
> Michael Paquier
> http://michael.otacoo.com
>

On Mon, Jun 6, 2011 at 9:39 PM, Lionel Frachon <lio...@gm...>wrote:

> Hello,
>
> looking at the debug1 mode log on datanode3, I found some interesting
> points hereafter (vacuum on, max_prepared_transactions=5000):
>
> (with normal inserts)
> [....]
> DEBUG:  unset snapshot info
> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
> DEBUG:  unset snapshot info
> DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
> DEBUG:  unset snapshot info
> DEBUG:  [re]setting xid = 0, old_value = 0
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 102
> DEBUG:  [re]setting xid = 102, old_value = 0
> DEBUG:  TransactionId = 102
> DEBUG:  xid (102) does not follow ShmemVariableCache->nextXid (665)
> DEBUG:  Record transaction commit 101
> DEBUG:  Record transaction commit 102
> DEBUG:  [re]setting xid = 0, old_value = 0
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 103
> DEBUG:  [re]setting xid = 103, old_value = 0
> DEBUG:  unset snapshot info
> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
> DEBUG:  TransactionId = 103
> DEBUG:  xid (103) does not follow ShmemVariableCache->nextXid (665)
> DEBUG:  unset snapshot info
> DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
>
> While inserting with dsitrubuted hashed keys :
>
> [...]
> DEBUG:  [re]setting xid = 0, old_value = 0
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 522
> DEBUG:  [re]setting xid = 522, old_value = 0
> DEBUG:  TransactionId = 522
> DEBUG:  xid (522) does not follow ShmemVariableCache->nextXid (665)
> DEBUG:  Record transaction commit 521
> DEBUG:  Record transaction commit 522
> DEBUG:  [re]setting xid = 0, old_value = 0
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 524
> DEBUG:  [re]setting xid = 524, old_value = 0
> ERROR:  prepared transaction with identifier "T523" does not exist
> STATEMENT:  COMMIT PREPARED 'T523'
> DEBUG:  [re]setting xid = 0, old_value = 524
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 526
> DEBUG:  [re]setting xid = 526, old_value = 0
> ERROR:  prepared transaction with identifier "T525" does not exist
> STATEMENT:  COMMIT PREPARED 'T525'
> DEBUG:  [re]setting xid = 0, old_value = 526
> DEBUG:  unset snapshot info
> DEBUG:  Received new gxid 528
> DEBUG:  [re]setting xid = 528, old_value = 0
> ERROR:  prepared transaction with identifier "T527" does not exist
> [...]
>
You are right.
But this log:
DEBUG:  Received new gxid 103
means that GTM is feeding cluster in transaction ID to a very low value.
This may lead to visibility problems.
You should start GTM with an option like -x 1000 to be sure that it doesn't
feed transaction IDs lower than 628.
-- 
Michael Paquier
http://michael.otacoo.com

Hello,

looking at the debug1 mode log on datanode3, I found some interesting points
hereafter (vacuum on, max_prepared_transactions=5000):

(with normal inserts)
[....]
DEBUG:  unset snapshot info
DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
DEBUG:  unset snapshot info
DEBUG:  global snapshot info: gxmin: 101, gxmax: 101, gxcnt: 0
DEBUG:  unset snapshot info
DEBUG:  [re]setting xid = 0, old_value = 0
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 102
DEBUG:  [re]setting xid = 102, old_value = 0
DEBUG:  TransactionId = 102
DEBUG:  xid (102) does not follow ShmemVariableCache->nextXid (665)
DEBUG:  Record transaction commit 101
DEBUG:  Record transaction commit 102
DEBUG:  [re]setting xid = 0, old_value = 0
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 103
DEBUG:  [re]setting xid = 103, old_value = 0
DEBUG:  unset snapshot info
DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0
DEBUG:  TransactionId = 103
DEBUG:  xid (103) does not follow ShmemVariableCache->nextXid (665)
DEBUG:  unset snapshot info
DEBUG:  global snapshot info: gxmin: 103, gxmax: 103, gxcnt: 0

While inserting with dsitrubuted hashed keys :

[...]
DEBUG:  [re]setting xid = 0, old_value = 0
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 522
DEBUG:  [re]setting xid = 522, old_value = 0
DEBUG:  TransactionId = 522
DEBUG:  xid (522) does not follow ShmemVariableCache->nextXid (665)
DEBUG:  Record transaction commit 521
DEBUG:  Record transaction commit 522
DEBUG:  [re]setting xid = 0, old_value = 0
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 524
DEBUG:  [re]setting xid = 524, old_value = 0
ERROR:  prepared transaction with identifier "T523" does not exist
STATEMENT:  COMMIT PREPARED 'T523'
DEBUG:  [re]setting xid = 0, old_value = 524
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 526
DEBUG:  [re]setting xid = 526, old_value = 0
ERROR:  prepared transaction with identifier "T525" does not exist
STATEMENT:  COMMIT PREPARED 'T525'
DEBUG:  [re]setting xid = 0, old_value = 526
DEBUG:  unset snapshot info
DEBUG:  Received new gxid 528
DEBUG:  [re]setting xid = 528, old_value = 0
ERROR:  prepared transaction with identifier "T527" does not exist
[...]

No special info on the gtm node regarding the same transactions, though.

Hope this can help

Regards

Lionel F.


2011/6/2 Michael Paquier <mic...@gm...>

> The problem you are facing with the pooler may be related to this bug that
> has been found recently:
>
> https://sourceforge.net/tracker/?func=detail&aid=3310399&group_id=311227&atid=1310232
>
> It looks that datanode is not able to manage efficiently autovacuum commit.
> This problem may cause problems in data consistency, making a node to crash
> in the worst scenario.
>
> This could explain why you cannot begin a transaction correctly on nodes,
> connections to backends being closed by a crash or a consistency problem.
> Can you provide some backtrace or give hints about the problem you have?
> Some tips in node logs perhaps?
>
>
> On Wed, Jun 1, 2011 at 8:12 PM, Lionel Frachon <lio...@gm...>wrote:
>
>> Hello,
>>
>> I was forced to distribute data by replication and not by hash, as I'm
>> constantly getting "ERROR: Could not commit prepared transaction
>> implicitely" on other tables than Warehouse (w_id), using 10
>> warehouses (this error appears both on data loading, when using hash,
>> and when performing distributed queries).
>>
>> I used slightly different setup :
>> - 1 GTM-only node
>> - 1 Coordinator-only node
>> - 3 Datanodes
>>
>> Coordinator has 256MB RAM, Datanodes having 768. They did not reach at
>> any moment the full usage of dedicated RAM.
>>
>> However, running benchmark more than a few minutes (2 or 3) drives to
>> the following errors
>>
>> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> Message:   ERROR: Could not begin transaction on data nodes.
>> SQLState:  XX000
>> ErrorCode: 0
>>
>> Then a bit later
>> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>>
>> Message:   ERROR: Failed to get pooled connections
>> SQLState:  53000
>> ErrorCode: 0
>>
>> then (and I assume they are linked)
>> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> Message:   ERROR: Could not begin transaction on data nodes.
>> SQLState:  XX000
>> ErrorCode: 0
>>
>> additionnally, the test end with many
>> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> Message:   This connection has been closed.
>> SQLState:  08003
>> ErrorCode: 0
>>
>> I'm using 10 terminals, using 10 warehouses.
>>
>> Any clue for this error, (and for distribution by hash, I understand
>> they're probably linked...)
>>
>> Lionel F.
>>
>>
>>
>> 2011/5/31 Lionel Frachon <lio...@gm...>:
>> > Hi,
>> >
>> > yes, persistent_datanode_connections is now set to off - it may not be
>> > related to the issues I have.
>> >
>> > What amount of memory do you have on your datanodes & coordinator ?
>> >
>> > Here are my settings :
>> > datanode : shared_buffers = 512MB
>> > coordinator=256MB (now, was 96MB)
>> >
>> > I still get for some distributed tables (by hash)
>> > "ERROR: Could not commit prepared transaction implicitely"
>> >
>> > For distribution syntax, yes, I found your webpage talking about
>> > regression tests
>> >
>> >> You also have to know that it is important to set a limit of
>> connections on
>> >> datanodes equal to the sum of max connections on all coordinators.
>> >> For example, if your cluster is using 2 coordinator with 20 max
>> connections
>> >> each, you may have a maximum of 40 connections to datanodes.
>> >
>> > Ok, tweaking this today and launching the tests again...
>> >
>> >
>> > Lionel F.
>> >
>> >
>> >
>> > 2011/5/31 Michael Paquier <mic...@gm...>:
>> >>
>> >>
>> >> On Mon, May 30, 2011 at 7:34 PM, Lionel Frachon <
>> lio...@gm...>
>> >> wrote:
>> >>>
>> >>> Hi again,
>> >>>
>> >>> I turned off connection pooling on coordinator (dunno why it sayed
>> >>> on), raised the shared_buffers of coordinator, allowed 1000
>> >>> connections and the error disappeared.
>> >>
>> >> I am not really sure I get the meaning of this, but how did you turn
>> off
>> >> pooler on coordinator.
>> >> Did you use the parameter persistent_connections?
>> >> Connection pooling from coordinator is an automatic feature and you
>> have to
>> >> use it if you want to connect from a remote coordinator to backend XC
>> nodes.
>> >>
>> >> You also have to know that it is important to set a limit of
>> connections on
>> >> datanodes equal to the sum of max connections on all coordinators.
>> >> For example, if your cluster is using 2 coordinator with 20 max
>> connections
>> >> each, you may have a maximum of 40 connections to datanodes.
>> >> This uses a lot of shared buffer on a node, but typically this maximum
>> >> number of connections is never reached thanks to the connection
>> pooling.
>> >>
>> >> Please node also that number of Coordinator <-> Coordinator connections
>> may
>> >> also increase if DDL are used from several coordinators.
>> >>
>> >>> However, all data is still going on one node (and whatever I could
>> >>> choose as primary datanode), with 40 warehouses... any specific syntax
>> >>> to load balance warehouses over nodes ?
>> >>
>> >> CREATE TABLE foo (column_key type, other_column int) DISTRIBUTE BY
>> >> HASH(column_key);
>> >> --
>> >> Michael Paquier
>> >> http://michael.otacoo.com
>> >>
>> >
>>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>

Hi again,

done the test (with 3 initial warehouses, distributed by hash on their ID).
Expected behaviour is they've distributed amongst nodes, but (connected
through coordinator):

testperfs=# EXECUTE DIRECT ON NODE 3 'select * from warehouse';
 w_id | w_ytd | w_tax | w_name | w_street_1 | w_street_2 | w_city | w_state
| w_zip
------+-------+-------+--------+------------+------------+--------+---------+-------
(0 rows)

testperfs=# EXECUTE DIRECT ON NODE 2 'select * from warehouse';
 w_id | w_ytd | w_tax | w_name | w_street_1 | w_street_2 | w_city | w_state
| w_zip
------+-------+-------+--------+------------+------------+--------+---------+-------
(0 rows)

testperfs=# EXECUTE DIRECT ON NODE 1 'select * from warehouse';
 w_id |   w_ytd   | w_tax  |  w_name  |    w_street_1     |  w_street_2
|       w_city        | w_state |   w_zip
------+-----------+--------+----------+-------------------+--------------+---------------------+---------+-----------
    1 | 300000.00 | 0.0253 | awmmmaRe | sKsjzyBoATkSdQCKv | gzWxflQdxagP |
kEcZGWmkZRQuPTEnJYq | HA      | 123456789
(1 row)



Lionel F.


2011/6/3 Michael Paquier <mic...@gm...>

> I am also wondering if the status of your connections is OK. It is not
> really normal that you get error messages:
>
> ERROR: Could not begin transaction on data nodes.
> ERROR:  prepared transaction with identifier "T711" does not exist
>
> Do you know the existence of EXECUTE DIRECT?
>
> With a query like that:
> EXECUTE DIRECT ON NODE 1 'select * from a';
> you can check the results that are only on node 1.
>
> It could be worth checking once with a psql terminal that data is loaded
> correctly.
> If execute direct returns an error it would mean that something is missing
> in your settings.
> If there are no errors, something with JDBC does not work correctly.
>
> Also I have something else in mind, do you start up GTM with a first GXID
> more than 628?
> There may be visibility issues as initdb uses transaction ID lower than
> those ones for initialization.
>
>
> On Thu, Jun 2, 2011 at 8:46 PM, Mason <ma...@us...>wrote:
>
>> On Wed, Jun 1, 2011 at 9:09 PM, Michael Paquier
>> <mic...@gm...> wrote:
>> > The problem you are facing with the pooler may be related to this bug
>> that
>> > has been found recently:
>> >
>> https://sourceforge.net/tracker/?func=detail&aid=3310399&group_id=311227&atid=1310232
>> >
>> > It looks that datanode is not able to manage efficiently autovacuum
>> commit.
>> > This problem may cause problems in data consistency, making a node to
>> crash
>> > in the worst scenario.
>> >
>> > This could explain why you cannot begin a transaction correctly on
>> nodes,
>> > connections to backends being closed by a crash or a consistency
>> problem.
>> > Can you provide some backtrace or give hints about the problem you have?
>> > Some tips in node logs perhaps?
>>
>> To see if it is autovacuum, Lionel, you could temporarily disable it
>> and try to reproduce the error.
>>
>> Mason
>>
>> >
>> > On Wed, Jun 1, 2011 at 8:12 PM, Lionel Frachon <
>> lio...@gm...>
>> > wrote:
>> >>
>> >> Hello,
>> >>
>> >> I was forced to distribute data by replication and not by hash, as I'm
>> >> constantly getting "ERROR: Could not commit prepared transaction
>> >> implicitely" on other tables than Warehouse (w_id), using 10
>> >> warehouses (this error appears both on data loading, when using hash,
>> >> and when performing distributed queries).
>> >>
>> >> I used slightly different setup :
>> >> - 1 GTM-only node
>> >> - 1 Coordinator-only node
>> >> - 3 Datanodes
>> >>
>> >> Coordinator has 256MB RAM, Datanodes having 768. They did not reach at
>> >> any moment the full usage of dedicated RAM.
>> >>
>> >> However, running benchmark more than a few minutes (2 or 3) drives to
>> >> the following errors
>> >>
>> >> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> >> Message:   ERROR: Could not begin transaction on data nodes.
>> >> SQLState:  XX000
>> >> ErrorCode: 0
>> >>
>> >> Then a bit later
>> >> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> >>
>> >> Message:   ERROR: Failed to get pooled connections
>> >> SQLState:  53000
>> >> ErrorCode: 0
>> >>
>> >> then (and I assume they are linked)
>> >> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> >> Message:   ERROR: Could not begin transaction on data nodes.
>> >> SQLState:  XX000
>> >> ErrorCode: 0
>> >>
>> >> additionnally, the test end with many
>> >> --- Unexpected SQLException caught in NEW-ORDER Txn ---
>> >> Message:   This connection has been closed.
>> >> SQLState:  08003
>> >> ErrorCode: 0
>> >>
>> >> I'm using 10 terminals, using 10 warehouses.
>> >>
>> >> Any clue for this error, (and for distribution by hash, I understand
>> >> they're probably linked...)
>> >>
>> >> Lionel F.
>> >>
>> >>
>> >>
>> >> 2011/5/31 Lionel Frachon <lio...@gm...>:
>> >> > Hi,
>> >> >
>> >> > yes, persistent_datanode_connections is now set to off - it may not
>> be
>> >> > related to the issues I have.
>> >> >
>> >> > What amount of memory do you have on your datanodes & coordinator ?
>> >> >
>> >> > Here are my settings :
>> >> > datanode : shared_buffers = 512MB
>> >> > coordinator=256MB (now, was 96MB)
>> >> >
>> >> > I still get for some distributed tables (by hash)
>> >> > "ERROR: Could not commit prepared transaction implicitely"
>> >> >
>> >> > For distribution syntax, yes, I found your webpage talking about
>> >> > regression tests
>> >> >
>> >> >> You also have to know that it is important to set a limit of
>> >> >> connections on
>> >> >> datanodes equal to the sum of max connections on all coordinators.
>> >> >> For example, if your cluster is using 2 coordinator with 20 max
>> >> >> connections
>> >> >> each, you may have a maximum of 40 connections to datanodes.
>> >> >
>> >> > Ok, tweaking this today and launching the tests again...
>> >> >
>> >> >
>> >> > Lionel F.
>> >> >
>> >> >
>> >> >
>> >> > 2011/5/31 Michael Paquier <mic...@gm...>:
>> >> >>
>> >> >>
>> >> >> On Mon, May 30, 2011 at 7:34 PM, Lionel Frachon
>> >> >> <lio...@gm...>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi again,
>> >> >>>
>> >> >>> I turned off connection pooling on coordinator (dunno why it sayed
>> >> >>> on), raised the shared_buffers of coordinator, allowed 1000
>> >> >>> connections and the error disappeared.
>> >> >>
>> >> >> I am not really sure I get the meaning of this, but how did you turn
>> >> >> off
>> >> >> pooler on coordinator.
>> >> >> Did you use the parameter persistent_connections?
>> >> >> Connection pooling from coordinator is an automatic feature and you
>> >> >> have to
>> >> >> use it if you want to connect from a remote coordinator to backend
>> XC
>> >> >> nodes.
>> >> >>
>> >> >> You also have to know that it is important to set a limit of
>> >> >> connections on
>> >> >> datanodes equal to the sum of max connections on all coordinators.
>> >> >> For example, if your cluster is using 2 coordinator with 20 max
>> >> >> connections
>> >> >> each, you may have a maximum of 40 connections to datanodes.
>> >> >> This uses a lot of shared buffer on a node, but typically this
>> maximum
>> >> >> number of connections is never reached thanks to the connection
>> >> >> pooling.
>> >> >>
>> >> >> Please node also that number of Coordinator <-> Coordinator
>> connections
>> >> >> may
>> >> >> also increase if DDL are used from several coordinators.
>> >> >>
>> >> >>> However, all data is still going on one node (and whatever I could
>> >> >>> choose as primary datanode), with 40 warehouses... any specific
>> syntax
>> >> >>> to load balance warehouses over nodes ?
>> >> >>
>> >> >> CREATE TABLE foo (column_key type, other_column int) DISTRIBUTE BY
>> >> >> HASH(column_key);
>> >> >> --
>> >> >> Michael Paquier
>> >> >> http://michael.otacoo.com
>> >> >>
>> >> >
>> >
>> >
>> >
>> > --
>> > Michael Paquier
>> > http://michael.otacoo.com
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Simplify data backup and recovery for your virtual environment with
>> vRanger.
>> > Installation's a snap, and flexible recovery options mean your data is
>> safe,
>> > secure and there when you need it. Data protection magic?
>> > Nope - It's vRanger. Get your free trial download today.
>> > http://p.sf.net/sfu/quest-sfdev2dev
>> > _______________________________________________
>> > Postgres-xc-general mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>> >
>> >
>>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>