Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, > it's a lot of work and right now the only people > who've done that work aren't giving it away for free - or not in any form > that can be integrated into PostgreSQL without removing other capabilities > other users need. One MPP vendor implemented columnar store in roughly six months -- lo

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Craig Ringer
On 11/08/2011 06:36 PM, Ondrej Ivanič wrote: Are you saying there's too much random I/O? Is it possible it'd benefit from a column store? When you're using Greenplum are you using "Polymorphic Data Storage" column storage "WITH (orientation=column)" ? yes, exactly. Column store and compression

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, On 9 November 2011 04:53, John R Pierce wrote: > On 11/08/11 1:49 AM, Ondrej Ivanič wrote: >> >> Greenplum or Postgres + Fusion IO can deliver this performance for us. > > then, thats your answer!   it ain't free, oh well. FusionIO is little bit problematic: smaller card (2.4TB) has serious

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread John R Pierce
On 11/08/11 2:36 AM, Ondrej Ivanič wrote: Yeah, I know about those.. I like iotop but enterprise distributions do not ship fresh kernels... I need something which can I "safely" (slightly worse performance is acceptable but machine must survie) run in production for several hours and then cross r

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread John R Pierce
On 11/08/11 1:49 AM, Ondrej Ivanič wrote: Greenplum or Postgres + Fusion IO can deliver this performance for us. then, thats your answer! it ain't free, oh well. -- john r pierceN 37, W 122 santa cruz ca mid-left coast -- Sent via pgsq

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, 2011/11/8 Tomas Vondra : > Sure you did - you've stated that "mostly heavy read > workloads but OLTP performance is required (like run query over 100m+ > dataset in 15 sec)." That clearly mentions OLTP  ... Whatever :) Let's make it clear: I need to run aggregates/roll ups/drill downs on larg

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, 2011/11/8 Craig Ringer : > "Spreads reads too much" ? > > Are you saying there's too much random I/O? Is it possible it'd benefit from > a column store? > When you're using Greenplum are you using "Polymorphic Data Storage" column > storage "WITH (orientation=column)" ? yes, exactly. Column s

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Tomas Vondra
On 8 Listopad 2011, 10:49, Ondrej Ivanič wrote: > Hi, >>>  mostly heavy read >>> workloads but OLTP performance is required (like run query over 100m+ >>> dataset in 15 sec) >> >> that isn't OLTP, its OLAP.  Online Analytic Processing rather than >> Online >> Transaction Processing   large comp

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-08 Thread Ondrej Ivanič
Hi, >>  mostly heavy read >> workloads but OLTP performance is required (like run query over 100m+ >> dataset in 15 sec) > > that isn't OLTP, its OLAP.  Online Analytic Processing rather than Online > Transaction Processing   large complex reporting queries that have to > aggregate many rows is

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread Craig Ringer
On 11/08/2011 02:49 PM, Ondrej Ivanič wrote: Usually CPU is not bottleneck but I it was when I put Pustgres on FusionIO. The problem is that PG spreads reads too much . iostat reports very low drive utilisation and very low queue size. "Spreads reads too much" ? Are you saying there's too muc

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread John R Pierce
On 11/07/11 10:49 PM, Ondrej Ivanič wrote: mostly heavy read workloads but OLTP performance is required (like run query over 100m+ dataset in 15 sec) that isn't OLTP, its OLAP. Online Analytic Processing rather than Online Transaction Processing large complex reporting queries that h

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread Ondrej Ivanič
Hi, On 8 November 2011 16:58, Craig Ringer wrote: > Which one(s) are you referring to? In what kind of workloads? > > Are you talking about Greenplum or similar? Yes, mainly Geenplum and nCluster (AsterData). I haven't played with gridSQL and pgpool-II's parallel query mode too much. Queries are

Re: [GENERAL] Postgres vs other Postgres based MPP implementations

2011-11-07 Thread Craig Ringer
On 11/08/2011 09:28 AM, Ondrej Ivanič wrote: I have simple question (I think which is not easy to answer): why Postgres is so slow comparing to other Postgres based MPP products Which one(s) are you referring to? In what kind of workloads? Are you talking about Greenplum or similar? Another