Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread Mark Kirkwood
On 07/05/13 18:10, Simon Riggs wrote: On 7 May 2013 01:23, wrote: I'm thinking that a variant of (2) might be simpler to inplement: (I think Matt C essentially beat me to this suggestion - he originally discovered this issue). It is probably good enough for only *new* plans to react to the i

Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread Simon Riggs
On 7 May 2013 01:23, wrote: > I'm thinking that a variant of (2) might be simpler to inplement: > > (I think Matt C essentially beat me to this suggestion - he originally > discovered this issue). It is probably good enough for only *new* plans to > react to the increased/increasing number of in

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Igor Neyman
From: Anne Rosset [aros...@collab.net] Sent: Monday, May 06, 2013 5:51 PM To: Igor Neyman; k...@rice.edu Cc: pgsql-performance@postgresql.org Subject: RE: [PERFORM] Deterioration in performance when query executed in multi threads Hi Igor, Result with en

Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread mark . kirkwood
> Simon Riggs wrote: > > Patch works and improves things, but we're still swamped by the block > accesses via the index. Which *might* be enough to stop it making the server go unresponsive, we'll look at the effect of this in the next few days, nice work! > > Which brings me back to Mark's origi

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Gavin Flower
Anne, please read the comment at the bottom of this post! On 07/05/13 09:46, Anne Rosset wrote: Hi Thomas, It is not a dedicated box (we have Jboss running too). cpu_tuple_cost | 0.01 seq_page_cost | 1 random_page_cost| 4 effective_cache_size

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Anne Rosset
Hi Igor, Result with enable_nestloop off: --- Hash Join (cost=49946.4

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Anne Rosset
Hi Thomas, It is not a dedicated box (we have Jboss running too). cpu_tuple_cost | 0.01 seq_page_cost | 1 random_page_cost| 4 effective_cache_size| 512MB We have the data directory on nfs (rw,intr,hard,tcp,rsize=32768,wsize=32768,nfs

Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread Matt Clarkson
> > (2) seems fairly hard generically, since we'd have to keep track of > the tids returned from the IndexScan to allow us to switch to a > different plan and avoid re-issuing rows that we've already returned. > But maybe if we adapted the IndexScan plan type so that it adopted a > more page orie

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Igor Neyman
> -Original Message- > From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] On Behalf Of Thomas Kellerer > Sent: Monday, May 06, 2013 1:12 PM > To: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] Deterioration in performance when query

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Igor Neyman
> -Original Message- > From: Anne Rosset [mailto:aros...@collab.net] > Sent: Monday, May 06, 2013 1:01 PM > To: Igor Neyman; k...@rice.edu > Cc: pgsql-performance@postgresql.org > Subject: RE: [PERFORM] Deterioration in performance when query executed > in multi threads > > Hi Igor, > Th

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Thomas Kellerer
Anne Rosset, 06.05.2013 19:00: Postgres version: 9.0.13 Work_mem is set to 64MB Shared_buffer to 240MB Segment_size is 1GB Wal_buffer is 10MB Artifact table: 251831 rows Field_value table: 77378 rows Mntr_subscription: 929071 rows Relationship: 270478 row Folder: 280356 rows Item: 716465 rows

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Anne Rosset
Hi Igor, The explain analyze is from when there was no load. Artifact table: 251831 rows Field_value table: 77378 rows Mntr_subscription: 929071 rows Relationship: 270478 row Folder: 280356 rows Item: 716465 rows Sfuser: 5733 rows Project: 1817 rows 8CPUs RAM: 8GB Postgres version: 9.0.13 And

Re: [PERFORM] RT3.4 query needed a lot more tuning with 9.2 than it did with 8.1

2013-05-06 Thread Tom Lane
Christoph Berg writes: > We recently (finally) migrated an Request Tracker 3.4 database running > on 8.1.19 to 9.2.4. The queries used by rt3.4 are sometimes weird, but > 8.1 coped without too much tuning. The schema looks like this: The newer rowcount estimates are much further away from reality

[PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Anne Rosset
Hi all, We are running a stress test that executes one select query with multiple threads. The query executes very fast (10ms). It returns 100 rows. I see deterioration in the performance when we have multiple threads executing the query. With 100 threads, the query takes between 3s and 8s. I

Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread Gavin Flower
On 03/05/13 00:27, Simon Riggs wrote: On 2 May 2013 01:49, Mark Kirkwood wrote: On 02/05/13 02:06, Tom Lane wrote: Mark Kirkwood writes: I am concerned that the deafening lack of any replies to my original message is a result of folk glancing at your original quick reply and thinking... inco

[PERFORM] RT3.4 query needed a lot more tuning with 9.2 than it did with 8.1

2013-05-06 Thread Christoph Berg
Hi, this is more of a report than a question, because we thought this would be interesting to share. We recently (finally) migrated an Request Tracker 3.4 database running on 8.1.19 to 9.2.4. The queries used by rt3.4 are sometimes weird, but 8.1 coped without too much tuning. The schema looks li

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Merlin Moncure
On Fri, May 3, 2013 at 3:52 PM, Anne Rosset wrote: > We saw a little bit improvement by increasing the min_pool_size but again I > see a bigvariation in the time the query is executed. Here is the query: > > srdb=> explain analyze SELECT > psrdb->artifact.id AS id, > psrdb->artifa

Re: [PERFORM] Deterioration in performance when query executed in multi threads

2013-05-06 Thread Igor Neyman
> -Original Message- > From: Anne Rosset [mailto:aros...@collab.net] > Sent: Friday, May 03, 2013 4:52 PM > To: Igor Neyman; k...@rice.edu > Cc: pgsql-performance@postgresql.org > Subject: RE: [PERFORM] Deterioration in performance when query executed > in multi threads > > We saw a litt

Re: [PERFORM] In progress INSERT wrecks plans on table

2013-05-06 Thread Simon Riggs
On 6 May 2013 02:51, Mark Kirkwood wrote: > On 05/05/13 00:49, Simon Riggs wrote: >> >> On 3 May 2013 13:41, Simon Riggs wrote: >> >>> (3) to make the check on TransactionIdIsInProgress() into a heuristic, >>> since we don't *need* to check that, so if we keep checking the same >>> xid repeatedly