Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Suvankar Roy
Hi Robert, Thanks much for your valuable inputs This spaces and tabs problem is killing me in a way, it is pretty cumbersome to say the least Regards, Suvankar Roy "Robert Mah" Sent by: Robert Mah 08/02/2009 10:52 PM To "'Suvankar Roy'" , cc Subject RE: [PERFORM] Greenplum Ma

Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Suvankar Roy
Hi Richard, I sincerely regret the inconvenience caused. %YAML 1.1 --- VERSION: 1.0.0.1 DATABASE: test_db1 USER: gpadmin DEFINE: - INPUT: #** This the line which is causing the error **# NAME: doc TABLE: documents - INPUT:

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread PFC
lzo is much, much, (much) faster than zlib. Note, I've tried several decompression speed is even more awesome... times to contact the author to get clarification on licensing terms and have been unable to get a response. lzop and the LZO library are distributed under the terms of the GNU

Re: [PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
Server has 32G memory and it's a dedicated to run PG and no other application is sharing this database. I have checked checkpoints and they don't occur during those slow query runtimes. Checkpoint_segments is set 128. here is quick snap from vmstat. # vmstat 5 5 kthr memorypage

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Merlin Moncure
On Mon, Aug 3, 2009 at 2:56 PM, Tom Lane wrote: > I don't see anything very contradictory here.  What you're demonstrating > is that it's nice to be able to throw a third CPU at the compression > part of the problem.  That's likely to remain true if we shift to a > different compression algorithm.

Re: [PERFORM] Query help

2009-08-03 Thread Kevin Grittner
"Subbiah Stalin-XCGF84" wrote: > Shared buffer=8G, effective cache size=4G. That is odd; if your shared buffers are at 8G, you must have more than 4G of cache. How much RAM is used for cache at the OS level? Normally you would add that to the shared buffers to get your effective cache size,

Re: [PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
Sure I can provide those details. I have seen this query running 5+ minutes for different values for doaminID too. Its just that it happens at random and gets fixed within few mins. Shared buffer=8G, effective cache size=4G. Optimizer/autovaccum settings are defaults relname

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread PFC
I get very different (contradictory) behavior. Server with fast RAID, 32GB RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 8.3.6 That's a very different serup from my (much less powerful) box, so that would explain it... No disk wait time during any test. One test beforehand was

Re: [PERFORM] Query help

2009-08-03 Thread Kevin Grittner
"Subbiah Stalin-XCGF84" wrote: > Not sure what's wrong in below execution plan but at times the query > runs for 5 minutes to complete and after a while it runs within a > second or two. The plan doesn't look entirely unreasonable for the given query, although it's hard to be sure of that wit

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Scott Carey
On 8/3/09 11:56 AM, "Tom Lane" wrote: > Scott Carey writes: >> I get very different (contradictory) behavior. Server with fast RAID, 32GB >> RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 >> 8.3.6 >> No disk wait time during any test. One test beforehand was used to prime >> the disk c

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Tom Lane
Scott Carey writes: > I get very different (contradictory) behavior. Server with fast RAID, 32GB > RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 > 8.3.6 > No disk wait time during any test. One test beforehand was used to prime > the disk cache. > 100% CPU in the below means one core fully

[PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
All, Not sure what's wrong in below execution plan but at times the query runs for 5 minutes to complete and after a while it runs within a second or two. Here is explain analyze out of the query. SELECT OBJECTS.ID,OBJECTS.NAME,OBJECTS.TYPE,OBJECTS.STATUS,OBJECTS.ALTNAME,OBJE CTS.DOMAINID,OBJ

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Scott Carey
On 7/31/09 4:01 PM, "PFC" wrote: > On Fri, 31 Jul 2009 19:04:52 +0200, Tom Lane wrote: > >> Greg Stark writes: >>> On Thu, Jul 30, 2009 at 11:30 PM, Tom Lane wrote: I did some tracing and verified that pg_dump passes data to deflate() one table row at a time.  I'm not sure about the

Re: [PERFORM] Why is PostgreSQL so slow on Windows ( Postgres 8.3.7) version

2009-08-03 Thread Marc Cousin
The few 'obvious' things I see : ID and POLLID aren't of the same type (numeric vs bigint) TTIME isn't indexed. And as a general matter, you should stick to native datatypes if you don't need numeric. But as said in the other answer, maybe you should redo this schema and use more consistent d

Re: [PERFORM] Why is PostgreSQL so slow on Windows ( Postgres 8.3.7) version

2009-08-03 Thread Grzegorz Jaśkiewicz
how about normalizing the schema for start ? by the looks of it, you have huge table,with plenty of varchars, that smells like bad design of db. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/p

Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Richard Huxton
Suvankar Roy wrote: Hi Richard, I sincerely regret the inconvenience caused. No big inconvenience, but the lists can be very busy sometimes and the easier you make it for people to answer your questions the better the answers you will get. %YAML 1.1 --- VERSION: 1.0.0.1 DATABASE: tes

Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Richard Huxton
Suvankar Roy wrote: Hi all, Has anybody worked on Greenplum MapReduce programming ? I am facing a problem while trying to execute the below Greenplum Mapreduce program written in YAML (in blue). The other poster suggested contacting Greenplum and I can only agree. The error is thrown in t