Menu
▾
▴
postgres-xc-general — General info and messages
You can subscribe to this list here.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
(6) |
Sep
|
Oct
(19) |
Nov
(1) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(12) |
Feb
(1) |
Mar
(4) |
Apr
(4) |
May
(32) |
Jun
(12) |
Jul
(11) |
Aug
(1) |
Sep
(6) |
Oct
(3) |
Nov
|
Dec
(10) |
| 2012 |
Jan
(11) |
Feb
(1) |
Mar
(3) |
Apr
(25) |
May
(53) |
Jun
(38) |
Jul
(103) |
Aug
(54) |
Sep
(31) |
Oct
(66) |
Nov
(77) |
Dec
(20) |
| 2013 |
Jan
(91) |
Feb
(86) |
Mar
(103) |
Apr
(107) |
May
(25) |
Jun
(37) |
Jul
(17) |
Aug
(59) |
Sep
(38) |
Oct
(78) |
Nov
(29) |
Dec
(15) |
| 2014 |
Jan
(23) |
Feb
(82) |
Mar
(118) |
Apr
(101) |
May
(103) |
Jun
(45) |
Jul
(6) |
Aug
(10) |
Sep
|
Oct
(32) |
Nov
|
Dec
(9) |
| 2015 |
Jan
(3) |
Feb
(5) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
(9) |
Aug
(4) |
Sep
(3) |
Oct
|
Nov
|
Dec
|
| 2016 |
Jan
(3) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2018 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(4) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
1
(2) |
2
(2) |
3
(4) |
4
(5) |
5
(17) |
6
(4) |
|
7
(7) |
8
(12) |
9
(1) |
10
(1) |
11
(6) |
12
(7) |
13
|
|
14
(1) |
15
(3) |
16
(2) |
17
(1) |
18
(2) |
19
(8) |
20
|
|
21
(4) |
22
(5) |
23
(3) |
24
|
25
(1) |
26
(3) |
27
(2) |
|
28
|
29
(1) |
30
(3) |
|
|
|
|
|
From: Rene R. B. <ren...@gm...> - 2013-04-19 14:04:42
|
Ok, I'll set a battery of tests between three machines for a longer time (5+mins) ,with a greater scaling factor (10+), and report back. Regards! 2013/4/19 Mason Sharp <ma...@st...> > Hi Rene, > > How many servers did you use? > > XC adds a lot more latency than standard PostgreSQL because you have to go > through the coordinator layer, which also adds additional hops to > communicate with GTM. > > You start to see a benefit with XC under high concurrency workloads, when > on a single server it cannot cache so much anymore and/or IO cannot > effectively keep up with the write workload. > > > > On Fri, Apr 19, 2013 at 12:22 AM, Rene Romero Benavides < > ren...@gm...> wrote: > >> Thanks for your kind replies. >> The benchmark was the standard pgbench benchmark (select, insert, delete, >> update). I'll look into the table distribution issue. I'm not complaining >> at all, I really like your work, I just wan't to understand what's going >> on, an someday maybe help you make tests. >> >> Good night. >> >> >> 2013/4/18 Ashutosh Bapat <ash...@en...> >> >>> >>> >>> >>> On Fri, Apr 19, 2013 at 9:38 AM, 鈴木 幸市 <ko...@in...> wrote: >>> >>>> Unfortunately, read-only benchmark does not scale well. >>>> Write-intensive benchmark will scale better. >>>> >>> >>> Not necessarily Suzuki-san. It all boils down to how the tables are >>> distributed. For read-only benchmarks, one of the options is to have all >>> the data replicated and use multiple coordinators with different preferred >>> node. >>> >>> >>>> --- >>>> Koichi Suzuki >>>> >>>> >>>> >>>> On 2013/04/19, at 12:49, Ashutosh Bapat < >>>> ash...@en...> wrote: >>>> >>>> Hi Rene, >>>> Please make sure that the distributions of the tables involved in the >>>> benchmark are as per guidelines. You will find some material on this topic >>>> at >>>> http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf<http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012..pdf> >>>> . >>>> >>>> >>>> On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < >>>> ren...@gm...> wrote: >>>> >>>>> Hello. using version 1.0.2 >>>>> I followed the short version of the installation process and ran the >>>>> following benchmark: >>>>> >>>>> pgbench -T 60 -c 4 -j 4 benchdb >>>>> starting vacuum...end. >>>>> transaction type: TPC-B (sort of) >>>>> scaling factor: 4 >>>>> query mode: simple >>>>> number of clients: 4 >>>>> number of threads: 4 >>>>> duration: 60 s >>>>> number of transactions actually processed: 1631 >>>>> tps = 27.108444 (including connections establishing) >>>>> tps = 27.111498 (excluding connections establishing) >>>>> >>>>> This is the related sar and iotop info: >>>>> http://img32.imageshack.us/img32/3990/highio.png >>>>> >>>>> Same test on a single server (no postgres-xc) with the same >>>>> configuration: >>>>> >>>>> pgbench -T 60 -c 4 -j 4 -p 5433 pgbench >>>>> starting vacuum...end. >>>>> transaction type: TPC-B (sort of) >>>>> scaling factor: 4 >>>>> query mode: simple >>>>> number of clients: 4 >>>>> number of threads: 4 >>>>> duration: 60 s >>>>> number of transactions actually processed: 6179 >>>>> tps = 102.775810 (including connections establishing) >>>>> tps = 102.788523 (excluding connections establishing) >>>>> >>>>> sar and iotop output: >>>>> >>>>> http://img823.imageshack.us/img823/8775/highio11.png >>>>> >>>>> Is this high IO activity expected? I know that prepare transaction / >>>>> commit prepared are expensive operations but this is too much IMO. >>>>> >>>>> Both cases using default server configuration. >>>>> >>>>> Is there any tuning that'd mitigate this IO bottleneck? >>>>> >>>>> -- >>>>> El genio es 1% inspiración y 99% transpiración. >>>>> Thomas Alva Edison >>>>> http://pglearn.blogspot.mx/ >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Precog is a next-generation analytics platform capable of advanced >>>>> analytics on semi-structured data. The platform includes APIs for >>>>> building >>>>> apps and a phenomenal toolset for data science. Developers can use >>>>> our toolset for easy data analysis & visualization. Get a free account! >>>>> http://www2.precog.com/precogplatform/slashdotnewsletter >>>>> _______________________________________________ >>>>> Postgres-xc-general mailing list >>>>> Pos...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best Wishes, >>>> Ashutosh Bapat >>>> EntepriseDB Corporation >>>> The Enterprise Postgres Company >>>> ------------------------------------------------------------------------------ >>>> Precog is a next-generation analytics platform capable of advanced >>>> analytics on semi-structured data. The platform includes APIs for >>>> building >>>> apps and a phenomenal toolset for data science. Developers can use >>>> our toolset for easy data analysis & visualization. Get a free account! >>>> >>>> http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________ >>>> Postgres-xc-general mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>>> >>>> >>>> >>> >>> >>> -- >>> Best Wishes, >>> Ashutosh Bapat >>> EntepriseDB Corporation >>> The Enterprise Postgres Company >>> >> >> >> >> -- >> El genio es 1% inspiración y 99% transpiración. >> Thomas Alva Edison >> http://pglearn.blogspot.mx/ >> >> >> >> ------------------------------------------------------------------------------ >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> http://www2.precog.com/precogplatform/slashdotnewsletter >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> > > > -- > Mason Sharp > > > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Services > -- El genio es 1% inspiración y 99% transpiración. Thomas Alva Edison http://pglearn.blogspot.mx/ |
|
From: Mason S. <ma...@st...> - 2013-04-19 10:23:38
|
Hi Rene, How many servers did you use? XC adds a lot more latency than standard PostgreSQL because you have to go through the coordinator layer, which also adds additional hops to communicate with GTM. You start to see a benefit with XC under high concurrency workloads, when on a single server it cannot cache so much anymore and/or IO cannot effectively keep up with the write workload. On Fri, Apr 19, 2013 at 12:22 AM, Rene Romero Benavides < ren...@gm...> wrote: > Thanks for your kind replies. > The benchmark was the standard pgbench benchmark (select, insert, delete, > update). I'll look into the table distribution issue. I'm not complaining > at all, I really like your work, I just wan't to understand what's going > on, an someday maybe help you make tests. > > Good night. > > > 2013/4/18 Ashutosh Bapat <ash...@en...> > >> >> >> >> On Fri, Apr 19, 2013 at 9:38 AM, 鈴木 幸市 <ko...@in...> wrote: >> >>> Unfortunately, read-only benchmark does not scale well. >>> Write-intensive benchmark will scale better. >>> >> >> Not necessarily Suzuki-san. It all boils down to how the tables are >> distributed. For read-only benchmarks, one of the options is to have all >> the data replicated and use multiple coordinators with different preferred >> node. >> >> >>> --- >>> Koichi Suzuki >>> >>> >>> >>> On 2013/04/19, at 12:49, Ashutosh Bapat <ash...@en...> >>> wrote: >>> >>> Hi Rene, >>> Please make sure that the distributions of the tables involved in the >>> benchmark are as per guidelines. You will find some material on this topic >>> at >>> http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf<http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012..pdf> >>> . >>> >>> >>> On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < >>> ren...@gm...> wrote: >>> >>>> Hello. using version 1.0.2 >>>> I followed the short version of the installation process and ran the >>>> following benchmark: >>>> >>>> pgbench -T 60 -c 4 -j 4 benchdb >>>> starting vacuum...end. >>>> transaction type: TPC-B (sort of) >>>> scaling factor: 4 >>>> query mode: simple >>>> number of clients: 4 >>>> number of threads: 4 >>>> duration: 60 s >>>> number of transactions actually processed: 1631 >>>> tps = 27.108444 (including connections establishing) >>>> tps = 27.111498 (excluding connections establishing) >>>> >>>> This is the related sar and iotop info: >>>> http://img32.imageshack.us/img32/3990/highio.png >>>> >>>> Same test on a single server (no postgres-xc) with the same >>>> configuration: >>>> >>>> pgbench -T 60 -c 4 -j 4 -p 5433 pgbench >>>> starting vacuum...end. >>>> transaction type: TPC-B (sort of) >>>> scaling factor: 4 >>>> query mode: simple >>>> number of clients: 4 >>>> number of threads: 4 >>>> duration: 60 s >>>> number of transactions actually processed: 6179 >>>> tps = 102.775810 (including connections establishing) >>>> tps = 102.788523 (excluding connections establishing) >>>> >>>> sar and iotop output: >>>> >>>> http://img823.imageshack.us/img823/8775/highio11.png >>>> >>>> Is this high IO activity expected? I know that prepare transaction / >>>> commit prepared are expensive operations but this is too much IMO. >>>> >>>> Both cases using default server configuration. >>>> >>>> Is there any tuning that'd mitigate this IO bottleneck? >>>> >>>> -- >>>> El genio es 1% inspiración y 99% transpiración. >>>> Thomas Alva Edison >>>> http://pglearn.blogspot.mx/ >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Precog is a next-generation analytics platform capable of advanced >>>> analytics on semi-structured data. The platform includes APIs for >>>> building >>>> apps and a phenomenal toolset for data science. Developers can use >>>> our toolset for easy data analysis & visualization. Get a free account! >>>> http://www2.precog.com/precogplatform/slashdotnewsletter >>>> _______________________________________________ >>>> Postgres-xc-general mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>>> >>>> >>> >>> >>> -- >>> Best Wishes, >>> Ashutosh Bapat >>> EntepriseDB Corporation >>> The Enterprise Postgres Company >>> ------------------------------------------------------------------------------ >>> Precog is a next-generation analytics platform capable of advanced >>> analytics on semi-structured data. The platform includes APIs for >>> building >>> apps and a phenomenal toolset for data science. Developers can use >>> our toolset for easy data analysis & visualization. Get a free account! >>> >>> http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >>> >>> >> >> >> -- >> Best Wishes, >> Ashutosh Bapat >> EntepriseDB Corporation >> The Enterprise Postgres Company >> > > > > -- > El genio es 1% inspiración y 99% transpiración. > Thomas Alva Edison > http://pglearn.blogspot.mx/ > > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- Mason Sharp StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Services |
|
From: Nikhil S. <ni...@st...> - 2013-04-19 06:05:47
|
Hi Rene, I see that you have run pgbench for just one minute? To get any reasonable results make sure to run pgbench for atleast 5+ minutes. Anything below that can be attributed to noise. Also a scaling factor of 4 is a bit too low. At that scale factor the entire database might just fit in memory. Please experiment with bigger scale factors and more transactions. Regards, Nikhils On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < ren...@gm...> wrote: > Hello. using version 1.0.2 > I followed the short version of the installation process and ran the > following benchmark: > > pgbench -T 60 -c 4 -j 4 benchdb > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 1631 > tps = 27.108444 (including connections establishing) > tps = 27.111498 (excluding connections establishing) > > This is the related sar and iotop info: > http://img32.imageshack.us/img32/3990/highio.png > > Same test on a single server (no postgres-xc) with the same configuration: > > pgbench -T 60 -c 4 -j 4 -p 5433 pgbench > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 6179 > tps = 102.775810 (including connections establishing) > tps = 102.788523 (excluding connections establishing) > > sar and iotop output: > > http://img823.imageshack.us/img823/8775/highio11.png > > Is this high IO activity expected? I know that prepare transaction / > commit prepared are expensive operations but this is too much IMO. > > Both cases using default server configuration. > > Is there any tuning that'd mitigate this IO bottleneck? > > -- > El genio es 1% inspiración y 99% transpiración. > Thomas Alva Edison > http://pglearn.blogspot.mx/ > > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- StormDB - http://www.stormdb.com The Database Cloud |
|
From: Rene R. B. <ren...@gm...> - 2013-04-19 04:22:31
|
Thanks for your kind replies. The benchmark was the standard pgbench benchmark (select, insert, delete, update). I'll look into the table distribution issue. I'm not complaining at all, I really like your work, I just wan't to understand what's going on, an someday maybe help you make tests. Good night. 2013/4/18 Ashutosh Bapat <ash...@en...> > > > > On Fri, Apr 19, 2013 at 9:38 AM, 鈴木 幸市 <ko...@in...> wrote: > >> Unfortunately, read-only benchmark does not scale well. Write-intensive >> benchmark will scale better. >> > > Not necessarily Suzuki-san. It all boils down to how the tables are > distributed. For read-only benchmarks, one of the options is to have all > the data replicated and use multiple coordinators with different preferred > node. > > >> --- >> Koichi Suzuki >> >> >> >> On 2013/04/19, at 12:49, Ashutosh Bapat <ash...@en...> >> wrote: >> >> Hi Rene, >> Please make sure that the distributions of the tables involved in the >> benchmark are as per guidelines. You will find some material on this topic >> at http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf<http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012..pdf> >> . >> >> >> On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < >> ren...@gm...> wrote: >> >>> Hello. using version 1.0.2 >>> I followed the short version of the installation process and ran the >>> following benchmark: >>> >>> pgbench -T 60 -c 4 -j 4 benchdb >>> starting vacuum...end. >>> transaction type: TPC-B (sort of) >>> scaling factor: 4 >>> query mode: simple >>> number of clients: 4 >>> number of threads: 4 >>> duration: 60 s >>> number of transactions actually processed: 1631 >>> tps = 27.108444 (including connections establishing) >>> tps = 27.111498 (excluding connections establishing) >>> >>> This is the related sar and iotop info: >>> http://img32.imageshack.us/img32/3990/highio.png >>> >>> Same test on a single server (no postgres-xc) with the same >>> configuration: >>> >>> pgbench -T 60 -c 4 -j 4 -p 5433 pgbench >>> starting vacuum...end. >>> transaction type: TPC-B (sort of) >>> scaling factor: 4 >>> query mode: simple >>> number of clients: 4 >>> number of threads: 4 >>> duration: 60 s >>> number of transactions actually processed: 6179 >>> tps = 102.775810 (including connections establishing) >>> tps = 102.788523 (excluding connections establishing) >>> >>> sar and iotop output: >>> >>> http://img823.imageshack.us/img823/8775/highio11.png >>> >>> Is this high IO activity expected? I know that prepare transaction / >>> commit prepared are expensive operations but this is too much IMO. >>> >>> Both cases using default server configuration. >>> >>> Is there any tuning that'd mitigate this IO bottleneck? >>> >>> -- >>> El genio es 1% inspiración y 99% transpiración. >>> Thomas Alva Edison >>> http://pglearn.blogspot.mx/ >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Precog is a next-generation analytics platform capable of advanced >>> analytics on semi-structured data. The platform includes APIs for >>> building >>> apps and a phenomenal toolset for data science. Developers can use >>> our toolset for easy data analysis & visualization. Get a free account! >>> http://www2.precog.com/precogplatform/slashdotnewsletter >>> _______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >>> >> >> >> -- >> Best Wishes, >> Ashutosh Bapat >> EntepriseDB Corporation >> The Enterprise Postgres Company >> ------------------------------------------------------------------------------ >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> >> http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> >> > > > -- > Best Wishes, > Ashutosh Bapat > EntepriseDB Corporation > The Enterprise Postgres Company > -- El genio es 1% inspiración y 99% transpiración. Thomas Alva Edison http://pglearn.blogspot.mx/ |
|
From: Ashutosh B. <ash...@en...> - 2013-04-19 04:12:47
|
On Fri, Apr 19, 2013 at 9:38 AM, 鈴木 幸市 <ko...@in...> wrote: > Unfortunately, read-only benchmark does not scale well. Write-intensive > benchmark will scale better. > Not necessarily Suzuki-san. It all boils down to how the tables are distributed. For read-only benchmarks, one of the options is to have all the data replicated and use multiple coordinators with different preferred node. > --- > Koichi Suzuki > > > > On 2013/04/19, at 12:49, Ashutosh Bapat <ash...@en...> > wrote: > > Hi Rene, > Please make sure that the distributions of the tables involved in the > benchmark are as per guidelines. You will find some material on this topic > at http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf<http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012..pdf> > . > > > On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < > ren...@gm...> wrote: > >> Hello. using version 1.0.2 >> I followed the short version of the installation process and ran the >> following benchmark: >> >> pgbench -T 60 -c 4 -j 4 benchdb >> starting vacuum...end. >> transaction type: TPC-B (sort of) >> scaling factor: 4 >> query mode: simple >> number of clients: 4 >> number of threads: 4 >> duration: 60 s >> number of transactions actually processed: 1631 >> tps = 27.108444 (including connections establishing) >> tps = 27.111498 (excluding connections establishing) >> >> This is the related sar and iotop info: >> http://img32.imageshack.us/img32/3990/highio.png >> >> Same test on a single server (no postgres-xc) with the same configuration: >> >> pgbench -T 60 -c 4 -j 4 -p 5433 pgbench >> starting vacuum...end. >> transaction type: TPC-B (sort of) >> scaling factor: 4 >> query mode: simple >> number of clients: 4 >> number of threads: 4 >> duration: 60 s >> number of transactions actually processed: 6179 >> tps = 102.775810 (including connections establishing) >> tps = 102.788523 (excluding connections establishing) >> >> sar and iotop output: >> >> http://img823.imageshack.us/img823/8775/highio11.png >> >> Is this high IO activity expected? I know that prepare transaction / >> commit prepared are expensive operations but this is too much IMO. >> >> Both cases using default server configuration. >> >> Is there any tuning that'd mitigate this IO bottleneck? >> >> -- >> El genio es 1% inspiración y 99% transpiración. >> Thomas Alva Edison >> http://pglearn.blogspot.mx/ >> >> >> >> ------------------------------------------------------------------------------ >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> http://www2.precog.com/precogplatform/slashdotnewsletter >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> > > > -- > Best Wishes, > Ashutosh Bapat > EntepriseDB Corporation > The Enterprise Postgres Company > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > > http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Enterprise Postgres Company |
|
From: 鈴木 幸市 <ko...@in...> - 2013-04-19 04:08:23
|
Unfortunately, read-only benchmark does not scale well. Write-intensive benchmark will scale better. --- Koichi Suzuki On 2013/04/19, at 12:49, Ashutosh Bapat <ash...@en...> wrote: > Hi Rene, > Please make sure that the distributions of the tables involved in the benchmark are as per guidelines. You will find some material on this topic at http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf. > > > On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides <ren...@gm...> wrote: > Hello. using version 1.0.2 > I followed the short version of the installation process and ran the following benchmark: > > pgbench -T 60 -c 4 -j 4 benchdb > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 1631 > tps = 27.108444 (including connections establishing) > tps = 27.111498 (excluding connections establishing) > > This is the related sar and iotop info: > http://img32.imageshack.us/img32/3990/highio.png > > Same test on a single server (no postgres-xc) with the same configuration: > > pgbench -T 60 -c 4 -j 4 -p 5433 pgbench > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 6179 > tps = 102.775810 (including connections establishing) > tps = 102.788523 (excluding connections establishing) > > sar and iotop output: > > http://img823.imageshack.us/img823/8775/highio11.png > > Is this high IO activity expected? I know that prepare transaction / commit prepared are expensive operations but this is too much IMO. > > Both cases using default server configuration. > > Is there any tuning that'd mitigate this IO bottleneck? > > -- > El genio es 1% inspiración y 99% transpiración. > Thomas Alva Edison > http://pglearn.blogspot.mx/ > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > > > -- > Best Wishes, > Ashutosh Bapat > EntepriseDB Corporation > The Enterprise Postgres Company > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general |
|
From: Ashutosh B. <ash...@en...> - 2013-04-19 03:50:05
|
Hi Rene, Please make sure that the distributions of the tables involved in the benchmark are as per guidelines. You will find some material on this topic at http://wiki.postgresql.org/images/f/f6/PGXC_Scalability_PGOpen2012.pdf. On Fri, Apr 19, 2013 at 4:37 AM, Rene Romero Benavides < ren...@gm...> wrote: > Hello. using version 1.0.2 > I followed the short version of the installation process and ran the > following benchmark: > > pgbench -T 60 -c 4 -j 4 benchdb > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 1631 > tps = 27.108444 (including connections establishing) > tps = 27.111498 (excluding connections establishing) > > This is the related sar and iotop info: > http://img32.imageshack.us/img32/3990/highio.png > > Same test on a single server (no postgres-xc) with the same configuration: > > pgbench -T 60 -c 4 -j 4 -p 5433 pgbench > starting vacuum...end. > transaction type: TPC-B (sort of) > scaling factor: 4 > query mode: simple > number of clients: 4 > number of threads: 4 > duration: 60 s > number of transactions actually processed: 6179 > tps = 102.775810 (including connections establishing) > tps = 102.788523 (excluding connections establishing) > > sar and iotop output: > > http://img823.imageshack.us/img823/8775/highio11.png > > Is this high IO activity expected? I know that prepare transaction / > commit prepared are expensive operations but this is too much IMO. > > Both cases using default server configuration. > > Is there any tuning that'd mitigate this IO bottleneck? > > -- > El genio es 1% inspiración y 99% transpiración. > Thomas Alva Edison > http://pglearn.blogspot.mx/ > > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Enterprise Postgres Company |
|
From: Ashutosh B. <ash...@en...> - 2013-04-19 03:46:54
|
Hi Cronje, In the current release and upcoming release, we haven't address utilising replicated tables as stand-bys for each other. Theoretically, and also architecture-wise, yes, if someone has replicated the tables, the corresponding nodes should be able to provide the required data in case, one of them fails. But that's not implemented yet. On Thu, Apr 18, 2013 at 2:29 PM, Cronje Fourie <cr...@ab...> wrote: > Morning all…**** > > ** ** > > I’ve been reading a fair bit about pgsql-xc and its shared nothing > architecture as well as some of its HA setups.**** > > ** ** > > For my part I’m in the process of researching an RDBMS that we can use for > a HA system. Based on my initial reading of the docs etc it looked like > pgsql-xc is a perfect fit for our situation since we can configure 2 or > more systems and have data replicated across all the nodes.**** > > ** ** > > My understanding is that this should provide us with a shared nothing > solution in which any node can go down at any time without affecting the > solution.**** > > ** ** > > But the docs I’ve seen always refer to a HA setups using master slave / > synching with HA tools used to fail over between the master and slave etc… > **** > > ** ** > > Are these configurations required only in the case of distributed tables? > Or does this remain a requirement for replicated tables?**** > > ** ** > > In short my desired configuration would be**** > > ** ** > > SERVERA – GTM, COORDA, DATANODEA**** > > SERVERB – GTM-STANDBY, COORDB, DATANODEB**** > > ** ** > > LB1 – Load balancing between SERVERA, SERVERB**** > > ** ** > > All tables are replicated across all nodes.**** > > In theory we should then be able to add**** > > SERVERC – GTM-STANDBY, COORDC, DATANODEC**** > > ** ** > > Data is replicated from SERVERA or SERVERB to SERVERC and once ready we > can activate on LB1.**** > > ** ** > > Also if SERVERA or SERVERB goes down, the GTM-STANDBY config will handle > that failover while the LB1 then just needs to feed all requests to the > remaining COORD.**** > > ** ** > > This should provide us with a theoretical 100% uptime without the need for > master / slave hot standby replication.**** > > ** ** > > Yet I don’t see this in any examples? Why? Is it so simple to do that no > one has bothered setting up such an example? Or is it in fact not possible > with pgsql-xc?**** > > ** ** > > Thanks in advance > Cronje**** > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Enterprise Postgres Company |
