Menu
▾
▴
Re: [Postgres-xc-general] "invalid memory alloc request size ..." error
|
From: Koichi S. <koi...@gm...> - 2013-08-21 02:19:50
|
Thanks Eiji for the effort. Looking forward to your outcome. Regards; --- Koichi Suzuki 2013/8/21 Eiji UWAIZUMI <uwa...@as...> > I think I'll try to output a detailed log of this matter. > I will output a detailed log with the following settings. > > Postgres-XC > log_min_messages = debug5 > > Server > iostat and vmstat > > Some time it may take, but I will report the results. > > Regards, > uwaizumi > > (2013/08/16 14:28), Nikhil Sontakke wrote: > > What I think is happening here is that the COPY process is trying to send > > data to the datanodes, but they are not consuming it at the same rate. > > Hence the send_some call sends only a portion of the data and the rest of > > the data has to remain buffered. This builds up eventually to more than > > half GB and the subsequent repalloc call to double the buffer fails > because > > 1GB is the max limit. This is evident from the below COPY error message > as > > well. > > > > What we should do is maybe if the data outbuffer size crosses a > pre-defined > > limit like 16MB or so, then we should use pgxc_node_flush to ensure that > > all of that has reached the datanode. After some point even the socket > will > > not be handle direct large send_some calls and we will increase the > buffer > > size unnecessarily. > > > > Regards, > > Nikhils > > > > > > On Fri, Aug 16, 2013 at 8:37 AM, Nikhil Sontakke <ni...@st... > >wrote: > > > >> Thanks Eiji san, > >> > >> > >> > >>> > >>> At the request of Hemmi-san, > >>> By applying the patch to the Postgres-XC v1.1, I have conducted a > DBT-1. > >>> However, the results did not change unfortunately. > >>> > >>> [results] > >>> $ psql -p 5422 -c "copy address from '/tmp/address.data' delimiter > '>';" > >>> ERROR: invalid memory alloc request size 1073741824 > >>> CONTEXT: COPY address, line 30305382: > >>> > "30305382>TXnEREEwqOLV11zEFGDXV4AXEDEeqLn9JDDwGxHw>TCyFEgXDNVOCihBdmdExEBAAvnXdvEEBDEE>TGEgxF5RdiuAEz..." > >>> > >>> > >> Yeah, as I slept over it I realized that what I am trying to solve is a > >> "COPY TO" issue and what you guys must be facing is a "COPY FROM" > error. I > >> will try to look at this after when I can make more progress on the > "COPY > >> TO" issue. > >> > >> Regards, > >> Nikhils > >> > >>> Regards, > >>> uwaizumi > >>> > >>> > >>> (2013/08/15 18:06), Hitoshi HEMMI wrote: > >>>> Hi Nikhil, > >>>> > >>>> Thank you for the patch. > >>>> I will be off from tomorrow, but I ask our team to test it > >>>> and post the result. > >>>> > >>>> Best, > >>>> -hemmi > >>>> > >>>> > >>>> Nikhil Sontakke さんは書きました: > >>>>> Hi Hitoshi-san, > >>>>> > >>>>> PFA, patch which should help to use less memory while running COPY > >>>>> commands. This is against REL 1.1. Please do let me know if this > helps > >>>>> in your case. > >>>>> > >>>>> Regards, > >>>>> Nikhils > >>>>> > >>>>> > >>>>> On Wed, Aug 14, 2013 at 3:38 PM, Hitoshi HEMMI > >>>>> <hem...@la... <mailto:hem...@la...>> > >>> wrote: > >>>>> > >>>>> Thank you, Nikhil. > >>>>> We will wait until a patch is available. > >>>>> > >>>>> Best, > >>>>> -hemmi > >>>>> > >>>>> > >>>>> Nikhil Sontakke さんは書きました: > >>>>> > Hi Hitoshi-san, > >>>>> > > >>>>> > I am working on a similar issue coincidentally. Basically > what's > >>>>> > happening is that the COPY command is creating tuples which > >>> will get > >>>>> > freed up only at the end when the command will finish, so the > >>> RAM > >>>>> > memory ends up being consumed and can lead to the error that > you > >>>>> have > >>>>> > provided. > >>>>> > > >>>>> > See the discussion around RemoteQueryNext on another thread. > >>>>> > > >>>>> > Regards, > >>>>> > Nikhils > >>>>> > > >>>>> > On Wed, Aug 14, 2013 at 1:56 PM, Hitoshi HEMMI > >>>>> > <hem...@la... > >>>>> <mailto:hem...@la...> > >>>>> <mailto:hem...@la... > >>>>> <mailto:hem...@la...>>> wrote: > >>>>> > > >>>>> > Hi, > >>>>> > > >>>>> > We encountered "invalid memory alloc request size ..." error > of > >>>>> > Postgres-XC, when preparing DBT-1 benchmark, and need > >>>>> > someones help. > >>>>> > (First of all, what does the error message mean?) > >>>>> > > >>>>> > ------------------ > >>>>> > Version of XC: v1.1.beta > >>>>> > > >>>>> > [process to cause an error] > >>>>> > - Data generation > >>>>> > cd <DBT-1 dir>/datagen/ > >>>>> > ./datagen -i 10000 -u 15000 -p <data dir> > >>>>> > > >>>>> > - Prepare XC cluster > >>>>> > We are using following HA configuration. > >>>>> > We are not sure this configuration is essential for the > error, > >>>>> > but we have never encountered the error without XC slave > nodes. > >>>>> > > >>>>> > ServerA: GTM (master) > >>>>> > ServerB: GTM (slave) > >>>>> > ServerC: GTM-Proxy, Coordinator1 (master), Datanode1 > (master), > >>>>> > Coordinator2 (slave), Datanode2 (slave) > >>>>> > ServerD: GTM-Proxy, Coordinator2 (master), Datanode2 > (master), > >>>>> > Coordinator1 (slave), Datanode1 (slave) > >>>>> > > >>>>> > - Data loading > >>>>> > $ createdb -E SQL_ASCII -T template0 -p 5422 DBT1 > >>>>> > $ psql -a -p 5422 -f <DBT-1 > dir>/scripts/pgsql/create_tables.sql > >>>>> > $ psql -p 5422 -c "copy address from '/tmp/address.data' > >>> delimiter > >>>>> > '>';" > >>>>> > > >>>>> > [the error] > >>>>> > ERROR: invalid memory alloc request size 1073741824 > >>>>> > CONTEXT: COPY address, line 33191219: > >>>>> > "33191219>EDpcFnEEAe954E5Tg6XbkiBVGLAcuHBhFAEElEE7>zxPzRj > >>>>> > > >>>>> > [Other information] > >>>>> > It seems small sized data can be copied: > >>>>> > > >>>>> > - DBT-1 tables with size (item 10000 ,user 15000) > >>>>> > address 4.7G ERROR > >>>>> > author 894K OK > >>>>> > cc_xacts 4.4G ERROR > >>>>> > customer 20G ERROR > >>>>> > item 5.0M OK > >>>>> > order_line 11G ERROR > >>>>> > orders 4.1G ERROR > >>>>> > stock 78K OK > >>>>> > -------------------- > >>>>> > > >>>>> > Best regards, > >>>>> > > >>>>> > -hemmi > >>>>> > > >>>>> > -- > >>>>> > Hitoshi HEMMI > >>>>> > NTT Open Source Software Center > >>>>> > hem...@la... <mailto: > hem...@la... > >>>> > >>>>> <mailto:hem...@la... > >>>>> <mailto:hem...@la...>> > >>>>> > Tel:(03)5860-5115 > >>>>> > Fax:(03)5463-5490 > >>>>> > > >>>>> > > >>>>> > > >>>>> > >>> > ------------------------------------------------------------------------------ > >>>>> > Get 100% visibility into Java/.NET code with AppDynamics > Lite! > >>>>> > It's a free troubleshooting tool designed for production. > >>>>> > Get down to code-level detail for bottlenecks, with <2% > >>> overhead. > >>>>> > Download for free and get started troubleshooting in minutes. > >>>>> > > >>>>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>>> < > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>> > >>>>> > > >>>>> < > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>>> < > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>>> > >>>>> > _______________________________________________ > >>>>> > Postgres-xc-general mailing list > >>>>> > Pos...@li... > >>>>> <mailto:Pos...@li...> > >>>>> > <mailto:Pos...@li... > >>>>> <mailto:Pos...@li...>> > >>>>> > > >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > -- > >>>>> > StormDB - http://www.stormdb.com > >>>>> > The Database Cloud > >>>>> > >>>>> > >>>>> -- > >>>>> Hitoshi HEMMI > >>>>> NTT Open Source Software Center > >>>>> hem...@la... <mailto: > hem...@la...> > >>>>> Tel:(03)5860-5115 > >>>>> Fax:(03)5463-5490 > >>>>> > >>>>> > >>>>> > >>> > ------------------------------------------------------------------------------ > >>>>> Get 100% visibility into Java/.NET code with AppDynamics Lite! > >>>>> It's a free troubleshooting tool designed for production. > >>>>> Get down to code-level detail for bottlenecks, with <2% > overhead. > >>>>> Download for free and get started troubleshooting in minutes. > >>>>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>>> < > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>>> > >>>>> _______________________________________________ > >>>>> Postgres-xc-general mailing list > >>>>> Pos...@li... > >>>>> <mailto:Pos...@li...> > >>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> StormDB - http://www.stormdb.com > >>>>> The Database Cloud > >>>> > >>>> > >>> > >>> > >>> -- > >>> ----------------------------------- > >>> 上泉 英二 > >>> uwa...@as... > >>> ----------------------------------- > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Get 100% visibility into Java/.NET code with AppDynamics Lite! > >>> It's a free troubleshooting tool designed for production. > >>> Get down to code-level detail for bottlenecks, with <2% overhead. > >>> Download for free and get started troubleshooting in minutes. > >>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Postgres-xc-general mailing list > >>> Pos...@li... > >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > >>> > >> > >> > >> > >> -- > >> StormDB - http://www.stormdb.com > >> The Database Cloud > >> > > > > > > > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > |
