Re: [PERFORM] Linux I/O tuning: CFQ vs. deadline

2010-02-09 Thread Scott Marlowe
On Tue, Feb 9, 2010 at 11:37 PM, Greg Smith wrote: > Jeff wrote: >> >> I'd done some testing a while ago on the schedulers and at the time >> deadline or noop smashed cfq.  Now, it is 100% possible since then that >> they've made vast improvements to cfq and or the VM to get better or similar >> p

Re: [PERFORM] Linux I/O tuning: CFQ vs. deadline

2010-02-09 Thread Greg Smith
Jeff wrote: I'd done some testing a while ago on the schedulers and at the time deadline or noop smashed cfq. Now, it is 100% possible since then that they've made vast improvements to cfq and or the VM to get better or similar performance. I recall a vintage of 2.6 where they severely messe

Re: [PERFORM] [GENERAL] PostgreSQL - case studies

2010-02-09 Thread Scott Marlowe
Quick note, please stick to text formatted email for the mailing list, it's the preferred format. On Tue, Feb 9, 2010 at 9:09 PM, Jayadevan M wrote: > > Hello all, > Apologies for the long mail. > I work for a company that is provides solutions mostly on a Java/Oracle > platform. Recently we mov

[PERFORM] How exactly PostgreSQL allocates memory for its needs?

2010-02-09 Thread Anton Maksimenkov
Can anybody briefly explain me how each postgres process allocate memory for it needs? I mean, what is the biggest size of malloc() it may want? How many such chunks? What is the average size of allocations? I think that at first it allocates contiguous piece of shared memory for "shared buffers"

Re: [PERFORM] PostgreSQL - case studies

2010-02-09 Thread Amitabh Kant
On Wed, Feb 10, 2010 at 9:39 AM, Jayadevan M wrote: > Any feedback - a few sentences with the db size, tps, h/w necessary to > support that, and acceptable down-time, type of application etc will be > greatly appreciated. > Our products are not of the blog/social networking type, but more of > o

[PERFORM] 512,600ms query becomes 7500ms... but why? Postgres 8.3 query planner quirk?

2010-02-09 Thread Bryce Nesbitt
I've got a very slow query, which I can make faster by doing something seemingly trivial. The query has been trouble for years (always slow, sometimes taking hours): 512,600ms Original, filter on articles.indexed (622 results) 7,500ms Remove "AND articles.indexed" (726 results, undesirable).

[PERFORM] PostgreSQL - case studies

2010-02-09 Thread Jayadevan M
Hello all, Apologies for the long mail. I work for a company that is provides solutions mostly on a Java/Oracle platform. Recently we moved on of our products to PostgreSQL. The main reason was PostgreSQL's GIS capabilities and the inability of government departments (especially road/traffic) to

Re: [PERFORM] DISTINCT vs. GROUP BY

2010-02-09 Thread Dimi Paun
On Tue, 2010-02-09 at 17:38 -0500, Tom Lane wrote: > The results certainly ought to be the same (although perhaps not with > the same ordering) --- if they aren't, please provide a reproducible > test case. The results are the same, this is not a problem. > As for efficiency, though, 8.1 didn't u

Re: [PERFORM] DISTINCT vs. GROUP BY

2010-02-09 Thread Tom Lane
Dimi Paun writes: >> From what I've read on the net, these should be very similar, > and should generate equivalent plans, in such cases: > SELECT DISTINCT x FROM mytable > SELECT x FROM mytable GROUP BY x > However, in my case (postgresql-server-8.1.18-2.el5_4.1), > they generated different res

Re: [PERFORM] DISTINCT vs. GROUP BY

2010-02-09 Thread Thom Brown
On 9 February 2010 21:46, Dimi Paun wrote: > >From what I've read on the net, these should be very similar, > and should generate equivalent plans, in such cases: > > SELECT DISTINCT x FROM mytable > SELECT x FROM mytable GROUP BY x > > However, in my case (postgresql-server-8.1.18-2.el5_4.1), > t

[PERFORM] DISTINCT vs. GROUP BY

2010-02-09 Thread Dimi Paun
>From what I've read on the net, these should be very similar, and should generate equivalent plans, in such cases: SELECT DISTINCT x FROM mytable SELECT x FROM mytable GROUP BY x However, in my case (postgresql-server-8.1.18-2.el5_4.1), they generated different results with quite different execu

Re: [PERFORM] Linux I/O tuning: CFQ vs. deadline

2010-02-09 Thread Jeff
On Feb 8, 2010, at 11:35 PM, da...@lang.hm wrote: And, yes, the whole I/O scheduling approach in Linux was just completely redesigned for a very recent kernel update. So even what we think we know is already obsolete in some respects. I'd done some testing a while ago on the schedulers

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Amitabh Kant wrote: > Just curious if you have a 16 physical CPU's or 16 cores on 4 > CPU/8 cores over 2 CPU with HT. Four quad core CPUs: vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU X7350 @ 2.93GHz stepping

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Amitabh Kant
On Tue, Feb 9, 2010 at 10:03 PM, Kevin Grittner wrote: > Jesper Krogh wrote: > File system is xfs noatime,nobarrier for all data; OS is on ext3. I > *think* the pg_xlog mirrored pair is hanging off the same > BBU-writeback controller as the big RAID, but I'd have to track down > the hardware te

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Jesper Krogh wrote: > Sorry if it is obvious.. but what filesystem/OS are you using and > do you have BBU-writeback on the main data catalog also? Sorry for not providing more context. ATHENA:/var/pgsql/data # uname -a Linux ATHENA 2.6.16.60-0.39.3-smp #1 SMP Mon May 11 11:46:34 UTC 2009 x86

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Jesper Krogh
> > Frankly, I was quite surprised by this, since some of the benchmarks > people have published on the effects of using a separate RAID for > the WAL files have only shown a one or two percent difference when > using a hardware RAID controller with BBU cache configured for > write-back. Hi Kevin.

[PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-09 Thread Kevin Grittner
Due to an error during an update to the system kernel on a database server backing a web application, we ran for a month (mid-December to mid-January) with the WAL files in the pg_xlog subdirectory on the same 40-spindle array as the data -- only the OS was on a separate mirrored pair of drives. W

Re: [PERFORM] foreign key constraint lock behavour in postgresql

2010-02-09 Thread Albe Laurenz
I wrote: > > One idea that occurs to me is that it might be possible to add to PG > > some tuple lock modes that are intended to cover updates that don't > > touch indexed columns. So, say: > > > > SHARED NONINDEX - conflicts only with EXCLUSIVE locks > > SHARED - conflicts with EXCLUSIVE or EXCL

Re: [PERFORM] [GENERAL] index is not using

2010-02-09 Thread Scott Marlowe
I just answered this less than an hour ago... And please don't cross post to multiple mailing lists. On Tue, Feb 9, 2010 at 12:43 AM, AI Rumman wrote: > I have created a index > create index leadaddress_phone_idx on > leadaddress(regexp_replace((phone)::text, '[^0-9]*'::text, ''::text, > 'g'::te

Re: [PERFORM] [GENERAL] index is not using

2010-02-09 Thread Guillaume Lelarge
Le 09/02/2010 08:43, AI Rumman a écrit : > I have created a index > create index leadaddress_phone_idx on > leadaddress(regexp_replace((phone)::text, '[^0-9]*'::text, ''::text, > 'g'::text)); > > But the index is not using. > > explain select * from leadaddress where > regexp_replace(phone,'[^0-9

Re: [PERFORM] index is not using?

2010-02-09 Thread Scott Marlowe
On Tue, Feb 9, 2010 at 12:55 AM, AI Rumman wrote: > I have created a index > create index leadaddress_phone_idx on > leadaddress(regexp_replace((phone)::text, '[^0-9]*'::text, ''::text, > 'g'::text)); > > But the index is not using. like '%yada' isn't capable of using an index. If it's left anc

[PERFORM] index is not using?

2010-02-09 Thread AI Rumman
I have created a index create index leadaddress_phone_idx on leadaddress(regexp_replace((phone)::text, '[^0-9]*'::text, ''::text, 'g'::text)); But the index is not using. explain select * from leadaddress where regexp_replace(phone,'[^0-9]*','','g') like '%2159438606';