Re: [PERFORM] Performance across multiple schemas

2007-08-28 Thread Robins
Thanks Tom, Exactly what I did, when I realised that there was an extra Table in the FROM with no conditions set. Well anyway, this did clear my doubts about whether schema affects performance at all. Robins On 8/29/07, Robins Tharakan <[EMAIL PROTECTED]> wrote: > > Thanks Tom, > > Exactly what

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-28 Thread Decibel!
On Tue, Aug 28, 2007 at 10:00:57AM -0500, Erik Jones wrote: > >>It seemed strange to me that our 70%-read db generates so much > >>dirty > >>pages that writing them out takes 4-8 seconds and grabs the full > >>bandwidth. > >>First, I started to tune bgwriter to a more aggressive settings,

Re: [PERFORM] 8.2.4 Chooses Bad Query Plan

2007-08-28 Thread Tom Lane
Pallav Kalva <[EMAIL PROTECTED]> writes: > I have analyzed tables again and also my default_stats_target is set to > 100, still it shows the same plan. >>> -> Index Scan using idx_accountactivity_fkactivityid on >>> accountactivity accountact0_ (cost=0.00..3.94 rows=1 width=16) >>> Index Cond: (

Re: [PERFORM] 8.2.4 Chooses Bad Query Plan

2007-08-28 Thread Pallav Kalva
Hi Tom, Thanks! for the reply, see my comments below Tom Lane wrote: > Pallav Kalva <[EMAIL PROTECTED]> writes: > >> We have recently upgraded our production database from 8.0.12 to >> 8.2.4, We have seen lot of improvements on 8.2.4 side but we are also >> seeing some queries which are

Re: [PERFORM] Performance problem with table containing a lot of text (blog)

2007-08-28 Thread Kari Lavikka
I didn't see your schema, but couldn't these problems be solved by storing the article id, owner id, and blog date in a separate table? It seems that if you don't actually need the content of the blogs, all of those questions could be answered by querying a very simple table with minimal I/O

Re: [PERFORM] Performance problem with table containing a lot of text (blog)

2007-08-28 Thread Dan Harris
Kari Lavikka wrote: Hello! Some background info.. We have a blog table that contains about eight million blog entries. Average length of an entry is 1200 letters. Because each 8k page can accommodate only a few entries, every query that involves several entries causes several random seeks to

[PERFORM] Performance problem with table containing a lot of text (blog)

2007-08-28 Thread Kari Lavikka
Hello! Some background info.. We have a blog table that contains about eight million blog entries. Average length of an entry is 1200 letters. Because each 8k page can accommodate only a few entries, every query that involves several entries causes several random seeks to disk. We are having

Re: Fwd: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Tom Lane
Mark Lewis <[EMAIL PROTECTED]> writes: > Unfortunately there has been a planner regression in 8.2 in some cases > with these forms of queries. This was discussed a few weeks (months?) > ago on this forum. I haven't looked closely enough to confirm that this > is the problem in your case, but it s

Re: [PERFORM] 8.2.4 Chooses Bad Query Plan

2007-08-28 Thread Tom Lane
Pallav Kalva <[EMAIL PROTECTED]> writes: > We have recently upgraded our production database from 8.0.12 to > 8.2.4, We have seen lot of improvements on 8.2.4 side but we are also > seeing some queries which are slow. > Particularly this below query is really bad in 8.2.4 , I can get > onl

Re: [PERFORM] Postgres performance problem

2007-08-28 Thread Anton Melser
Just a random thought/question... Are you running else on the machine? When you say "resource usage", do you mean hd space, memory, processor, ??? What are your values in top? More info... Cheers Anton On 27/08/2007, Bill Moran <[EMAIL PROTECTED]> wrote: > In response to Chris Mair <[EMAIL PROTEC

Re: [PERFORM] index & Bitmap Heap Scan

2007-08-28 Thread Tom Lane
Paul <[EMAIL PROTECTED]> writes: > Why in the first case, pgsql uses the "better" index and if i search > r_service instead of r_numero pgsql does a "Bitmap Heap scan" first ? Given the difference in the number of rows to be fetched, both plan choices look pretty reasonable to me. If you want to

Re: Fwd: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Mark Lewis
It looks like your view is using a left join to look for rows in one table without matching rows in the other, i.e. a SQL construct similar in form to the query below: SELECT ... FROM A LEFT JOIN B ON (...) WHERE B.primary_key IS NULL Unfortunately there has been a planner regression in 8.2 in so

Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Evan Carroll
From: "Evan Carroll" <[EMAIL PROTECTED]> To: "Kevin Grittner" <[EMAIL PROTECTED]>, pgsql-performance@postgresql.org Date: Tue, 28 Aug 2007 11:21:54 -0500 Subject: Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy) On 8/28/07, Kevin Grittner <[EMAIL PROTECTED]> wrote: > >>> On Tue, Aug 28

Fwd: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Evan Carroll
-- Forwarded message -- From: Evan Carroll <[EMAIL PROTECTED]> Date: Aug 28, 2007 11:23 AM Subject: Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy) To: Scott Marlowe <[EMAIL PROTECTED]> On 8/28/07, Scott Marlowe <[EMAIL PROTECTED]> wrote: > I looked through your quer

Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Kevin Grittner
>>> On Tue, Aug 28, 2007 at 10:22 AM, in message <[EMAIL PROTECTED]>, "Evan Carroll" <[EMAIL PROTECTED]> wrote: > Yes, I ran vacuum full after loading both dbs. Have you run VACUUM ANALYZE or ANALYZE? -Kevin ---(end of broadcast)--- TIP 2: D

Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gregory Stark wrote: > "Evan Carroll" <[EMAIL PROTECTED]> writes: > > "Evan Carroll" <[EMAIL PROTECTED]> writes: > >> Dearest dragon hunters and mortal wanna-bes, >> >> I recently upgraded a system from Apache2/mod_perl2 to >> Lighttpd/fastcgi. The u

Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Gregory Stark
"Evan Carroll" <[EMAIL PROTECTED]> writes: "Evan Carroll" <[EMAIL PROTECTED]> writes: > Dearest dragon hunters and mortal wanna-bes, > > I recently upgraded a system from Apache2/mod_perl2 to > Lighttpd/fastcgi. The upgrade went about as rough as can be. While in > the midst of a bad day, I decid

Re: [PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Scott Marlowe
On 8/28/07, Evan Carroll <[EMAIL PROTECTED]> wrote: > the midst of a bad day, I decided to make it worse, and upgrade Pg 8.1 > to 8.2. Most people I talk to seem to think 8.1 was a lemon release; > not I. 8.0 was the release that had more issues for me, as it was the first version with all the ba

[PERFORM] 8.2 Query 10 times slower than 8.1 (view-heavy)

2007-08-28 Thread Evan Carroll
Dearest dragon hunters and mortal wanna-bes, I recently upgraded a system from Apache2/mod_perl2 to Lighttpd/fastcgi. The upgrade went about as rough as can be. While in the midst of a bad day, I decided to make it worse, and upgrade Pg 8.1 to 8.2. Most people I talk to seem to think 8.1 was a lem

[PERFORM] 8.2.4 Chooses Bad Query Plan

2007-08-28 Thread Pallav Kalva
Hi, We have recently upgraded our production database from 8.0.12 to 8.2.4, We have seen lot of improvements on 8.2.4 side but we are also seeing some queries which are slow. Particularly this below query is really bad in 8.2.4 , I can get only the explain on this as explain analyze never

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-28 Thread Erik Jones
On Aug 22, 2007, at 10:57 AM, Kenneth Marshall wrote: On Wed, Aug 22, 2007 at 07:33:35PM +0400, Dmitry Potapov wrote: Hello! We run a large (~66Gb) web-backend database on Postgresql 8.2.4 on Linux. The hardware is Dual Xeon 5130 with 16Gb ram, LSI Megaraid U320-2x scsi c

[PERFORM] Bitmap Heap Scan before using index

2007-08-28 Thread GOERGLER Paul
Hello, I have a table (stats.tickets) with 2288965 rows (51 columns) and indexes like: ind_ti_stats_numero btree (tday, tmonth, tyear, r_cat, r_numero) ind_ti_stats_service btree (tday, tmonth, tyear, r_cat, r_service) ind_ti_stats_tmp_service btree (r_service, tyear, tmonth) ind_ti_stats_tmp_ser

[PERFORM] Performance issue

2007-08-28 Thread Willo van der Merwe
Hi Guys, I have something odd. I have Gallery2 running on PostgreSQL 8.1, and recently I upgraded to 8.1.9-1.el4s1.1 (64bit). The issue here really is how do I get PostgreSQL to work with their horrible code. The queries they generate look something like : SELECT blah, blah FROM table1, table2

Re: [PERFORM] Optimising "in" queries

2007-08-28 Thread Stephen Davies
I thought that I had but I screwed up the addresses. Here they are: benparts=# explain select count(rdate),rdate from reading where sensor_id in (1137,1138,1139,1140) group by rdate order by rdate desc limit 1; QUERY PLAN --

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-28 Thread Kenneth Marshall
On Wed, Aug 22, 2007 at 07:33:35PM +0400, Dmitry Potapov wrote: > Hello! > > We run a large (~66Gb) web-backend database on Postgresql 8.2.4 on > Linux. The hardware is Dual Xeon 5130 with 16Gb ram, LSI Megaraid U320-2x > scsi controller w/512Mb writeback cache and a BBU. Storage

Re: [PERFORM] Optimising "in" queries

2007-08-28 Thread Stephen Davies
array processing??? There are no arrays. What made you think there might be? The table definition is: benparts=# \d reading Table "public.reading" Column |Type | Modifiers ---+---

Re: [PERFORM] Autovacuum is running forever

2007-08-28 Thread Sachchida Ojha
Thanks to all of you. I have changed the settings and reloaded the config. Let me run this system overnight. I will update this forum if new settings works for me. I am also asking management to upgrade the hardware. Thanks a lot. Regards Sachchida -Original Message- From: Scott Marlow

Re: [PERFORM] server performance issues - suggestions for tuning

2007-08-28 Thread Scott Marlowe
On 8/27/07, Kevin Kempter <[EMAIL PROTECTED]> wrote: > Hi List; > > I've just inherited multiple postgres database servers in multiple data > centers across the US and Europe via a new contract I've just started. What pg version are you working with, and on what OS / OS version? > Each night duri

Re: [PERFORM] server performance issues - suggestions for tuning

2007-08-28 Thread Andrew Sullivan
On Tue, Aug 28, 2007 at 08:12:06AM -0500, Kevin Grittner wrote: > > Is there any way to queue up these queries and limit how many are running at > a time? Sure: limit the number of connections to the database, and put a pool in front. It can indeed help. If you have a lot of bloat due to lar

Re: [PERFORM] Performance across multiple schemas

2007-08-28 Thread Tom Lane
Robins <[EMAIL PROTECTED]> writes: > Could someone confirm as to whether a query across multiple schemas is known > to have any kind of a degraded performance ? Schemas are utterly, utterly irrelevant to performance. I'm guessing you missed analyzing one of the tables, or forgot an index, or some

Re: [PERFORM] Performance across multiple schemas

2007-08-28 Thread Robins
Oops! Guess I shot myself in the foot there. It seems to be an SQL issue and not really a PG problem... Sorry for bothering you all. However, now that we are here, could anyone tell if you would advise for multiple schemas (in PG) while designing the database structure ? Thanks Robins Tharakan

Re: [PERFORM] server performance issues - suggestions for tuning

2007-08-28 Thread Kevin Grittner
>>> On Mon, Aug 27, 2007 at 11:13 PM, in message <[EMAIL PROTECTED]>, Kevin Kempter <[EMAIL PROTECTED]> wrote: > Each night during the nightly batch processing several of the servers (2 in > particular) slow to a crawl - they are dedicated postgres database servers. > There is a lot of database

[PERFORM] Performance across multiple schemas

2007-08-28 Thread Robins
Hi, I have just reorganized a relatively decent sized query such that its constituent functions / tables are now spread over 3-4 schemas. However, the query has for some reason now become very slow (earlier used to take about 20 seconds, now takes about 500 seconds). The explain analyse (given be

Re: [PERFORM] partitioned table and ORDER BY indexed_field DESC LIMIT 1

2007-08-28 Thread Heikki Linnakangas
Bruce, would you please add this to the 8.4 patch queue so we remember to look at this later? It didn't occur to me that we can do that in the degenerate case when there's just a single node below the Append. A more general solution would be to check if the pathkeys of all the child nodes match, a

Re: [PERFORM] Performance issue

2007-08-28 Thread Willo van der Merwe
Hi Guys, Following Tom Lane's advice I upgraded to 8.2, and that solved all my problems. :D Thank you so much for your input, I really appreciate it. Kind regards Willo van der Merwe ---(end of broadcast)--- TIP 6: explain analyze is your fr

[PERFORM] index & Bitmap Heap Scan

2007-08-28 Thread Paul
Hello, I have a table (stats.tickets) with 2288965 rows (51 columns) and indexes like: ind_ti_stats_numero btree (tday, tmonth, tyear, r_cat, r_numero) ind_ti_stats_service btree (tday, tmonth, tyear, r_cat, r_service) ind_ti_stats_tmp_service btree (r_service, tyear, tmonth) ind_ti_stats_tmp_se

Re: [PERFORM] server performance issues - suggestions for tuning

2007-08-28 Thread Richard Huxton
Kevin Kempter wrote: Hi List; I've just inherited multiple postgres database servers in multiple data centers across the US and Europe via a new contract I've just started. Each night during the nightly batch processing several of the servers (2 in particular) slow to a crawl - they are dedi