[PERFORM] pgbench output

2006-02-01 Thread Pradeep Parmar
Hi,I'm fairly new to PostgreSQL. I was trying pgbench , but could not understand the output . Can anyone help me out to understand the output of pgbenchPradeep

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Luke Lonergan
Jeffrey, On 2/1/06 12:25 AM, "Jeffrey W. Baker" <[EMAIL PROTECTED]> wrote: > Ah, but someday Pg will be able to concurrently read from two > datastreams to complete a single query. And that day will be glorious > and fine, and you'll want as much disk concurrency as you can get your > hands on.

Re: [PERFORM] Index Usage using IN

2006-02-01 Thread Hari Warrier
Select ID from TableA where not exists ( Select ID from Table B where ID = TableA.ID) might give you index scan. Of course, that is only useful is TableA is very small table. Not appropriate for 250k rows on 2/1/2006 12:12 PM Ralph Mason said the following: Hi, I have 2 tables both have an

Re: [PERFORM] Default autovacuum settings too conservative

2006-02-01 Thread Matthew T. O'Connor
Jim C. Nasby wrote: Small tables are most likely to have either very few updates (ie: a 'lookup table') or very frequent updates (ie: a table implementing a queue). In the former, even with vacuum_threshold = 0 vacuum will be a very rare occurance. In the later case, a high threshold is likely to

[PERFORM] Default autovacuum settings too conservative

2006-02-01 Thread Jim C. Nasby
As I recall, the idea behind vacuum_threshold was to prevent too-frequent vacuuming of small tables. I'm beginning to question this reasoning: Small tables vacuum very, very quickly, so 'extra' vacuuming is very unlikely to hurt system performance. Small tables are most likely to have either very

Re: [PERFORM] Index Usage using IN

2006-02-01 Thread Bruno Wolff III
On Thu, Feb 02, 2006 at 09:12:59 +1300, Ralph Mason <[EMAIL PROTECTED]> wrote: > Hi, > > I have 2 tables both have an index on ID (both ID columns are an oid). > > I want to find only only rows in one and not the other. > > Select ID from TableA where ID not IN ( Select ID from Table B) > > T

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > Since you were so confident it couldn't be the outer join, I went > looking for what else I changed at the same time. I eliminated the code > referencing that table, which contained an OR. I've seen ORs cause > nasty problems with optimizers in the p

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Kevin Grittner
>>> On Wed, Feb 1, 2006 at 2:43 pm, in message <[EMAIL PROTECTED]>, "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > > I took out the OR in the > where clause, without eliminating that last outer join, and it optimized > fine. FYI, with both sides of the OR separated: explain analyze SELECT "C".*

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Kevin Grittner
>>> On Wed, Feb 1, 2006 at 2:36 pm, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> Tom Lane <[EMAIL PROTECTED]> wrote: >>> I'm interested to poke at this ... are you in a position to provide a >>> test case? > >> I can't sup

Re: [PERFORM] Index Usage using IN

2006-02-01 Thread Tom Lane
"Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > On Thu, 2006-02-02 at 09:12 +1300, Ralph Mason wrote: >> Select ID from TableA where ID not IN ( Select ID from Table B) > Have you considered this: > SELECT ID from TableA EXCEPT Select ID from Table B Also, increasing work_mem might persuade the

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> wrote: >> I'm interested to poke at this ... are you in a position to provide a >> test case? > I can't supply the original data, since many of the tables have > millions of rows, with some of the data (related to juvenil

Re: [PERFORM] Index Usage using IN

2006-02-01 Thread Jeffrey W. Baker
On Wed, 2006-02-01 at 12:22 -0800, Jeffrey W. Baker wrote: > On Thu, 2006-02-02 at 09:12 +1300, Ralph Mason wrote: > > Hi, > > > > I have 2 tables both have an index on ID (both ID columns are an oid). > > > > I want to find only only rows in one and not the other. > > > > Select ID from TableA

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Kevin Grittner
>>> On Wed, Feb 1, 2006 at 2:14 pm, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> Tom Lane <[EMAIL PROTECTED]> wrote: >>> ... expected an equivalent IN clause to work better. In fact, I'm not >>> clear why the planner isn't

Re: [PERFORM] Index Usage using IN

2006-02-01 Thread Jeffrey W. Baker
On Thu, 2006-02-02 at 09:12 +1300, Ralph Mason wrote: > Hi, > > I have 2 tables both have an index on ID (both ID columns are an oid). > > I want to find only only rows in one and not the other. > > Select ID from TableA where ID not IN ( Select ID from Table B) Have you considered this: SELEC

[PERFORM] Index Usage using IN

2006-02-01 Thread Ralph Mason
Hi, I have 2 tables both have an index on ID (both ID columns are an oid). I want to find only only rows in one and not the other. Select ID from TableA where ID not IN ( Select ID from Table B) This always generates sequential scans. Table A has about 250,000 rows. Table B has about 250,00

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> wrote: >> ... expected an equivalent IN clause to work better. In fact, I'm not >> clear why the planner isn't finding the cheapest plan (which it does >> estimate as cheapest) from the IN version you posted. > All I kno

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Kevin Grittner
>>> On Wed, Feb 1, 2006 at 1:34 pm, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> We do have a few queries where PostgreSQL is several orders of >> magnitude slower. It appears that the reason it is choosing a bad plan >> is

Re: [PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > We do have a few queries where PostgreSQL is several orders of > magnitude slower. It appears that the reason it is choosing a bad plan > is that it is reluctant to start from a subquery when there is an outer > join in the FROM clause. AFAICT this c

[PERFORM] Planner reluctant to start from subquery

2006-02-01 Thread Kevin Grittner
We're converting from a commercial database product to PostgreSQL, and generally things are going well. While the licensing agreement with the commercial vendor prohibits publication of benchmarks without their written consent, I'll just say that on almost everything, PostgreSQL is faster. We do

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Mike Rylander
On 2/1/06, Luke Lonergan <[EMAIL PROTECTED]> wrote: [snip] > This is actually interesting overall - I think what this might be showing is > that the Linux SW RAID1 is alternating I/Os to the mirror disks from > different processes (LWP or HWP both maybe?), but not within one process. I can confirm

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Steinar H. Gunderson
On Wed, Feb 01, 2006 at 09:42:12AM -0800, Luke Lonergan wrote: > This is actually interesting overall - I think what this might be showing is > that the Linux SW RAID1 is alternating I/Os to the mirror disks from > different processes (LWP or HWP both maybe?), but not within one process. Having re

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Luke Lonergan
PFC, On 2/1/06 1:01 AM, "PFC" <[EMAIL PROTECTED]> wrote: > 3- Read one block of A, then one block of B, then one block of A, etc. > Essentially this is the same as the threaded case, except there's only one > thread. > 53 seconds total (with heavy seeking noise from the h

Re: [PERFORM] execution plan : Oracle vs PostgreSQL

2006-02-01 Thread FERREIRA, William (VALTECH)
my first implementation was in pl/pgsql but when i query the children of a node, i need to store them into an array because i need to iterate over all the children and for each child, I test the type of it. if it's a PI or a TEXT, i write it into a file, but if it's an element, i call the same

Re: [PERFORM] execution plan : Oracle vs PostgreSQL

2006-02-01 Thread Tom Lane
"FERREIRA, William (VALTECH)" <[EMAIL PROTECTED]> writes: > My test document has 115000 nodes. > the export of the document(extracting all informations from database and > writing XML file on disk) takes 30s with Oracle and 5mn with Postgresql. > The Oracle stored procedure is written in pl/sql an

Re: [PERFORM] partitioning and locking problems

2006-02-01 Thread Tom Lane
"Marc Morin" <[EMAIL PROTECTED]> writes: > Do you mean it would be impossible to change the code so that existing > selects continue to use the pre-truncated table until they commit? Yes, because that table won't exist any more (as in the file's been unlinked) once the TRUNCATE commits. > The upd

Re: [PERFORM] partitioning and locking problems

2006-02-01 Thread Marc Morin
Tom, Do you mean it would be impossible to change the code so that existing selects continue to use the pre-truncated table until they commit? Or just require a more extensive change? The update/insert rule change appears to be more more doable? No? > -Original Message- > From: Tom Lane

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Michael Stone
On Tue, Jan 31, 2006 at 08:09:40PM -0800, Jeffrey W. Baker wrote: I think I've proved my point. Software RAID1 read balancing provides 0%, 300%, 100%, and 100% speedup on 1, 2, 4, and 8 threads, respectively. In the presence of random I/O, the results are even better. Umm, the point *was* abo

[PERFORM] execution plan : Oracle vs PostgreSQL

2006-02-01 Thread FERREIRA, William (VALTECH)
hi, i have a database storing XML documents. The main table contains the nodes of the document, the other tables contain data for each node (depending on the node's type : ELE, Text, PI, ...) My test document has 115000 nodes. the export of the document(extracting all informations from database

Re: [PERFORM] partitioning and locking problems

2006-02-01 Thread Richard Huxton
Marc Morin wrote: Under both these circumstances (truncate and create / replace rule) the locking behaviour of these commands can cause locking problems for us. The scenario is best illustrated as a series of steps: 1- long running report is running on view 2- continuous inser

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread PFC
I did a little test on soft raid1 : I have two 800 Mbytes files, say A and B. (RAM is 512Mbytes). Test 1 : 1- Read A, then read B : 19 seconds per file 2- Read A and B simultaneously using two threads : 22 seconds total (r

Re: [PERFORM] Huge Data sets, simple queries

2006-02-01 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 21:53 -0800, Luke Lonergan wrote: > Jeffrey, > > On 1/31/06 8:09 PM, "Jeffrey W. Baker" <[EMAIL PROTECTED]> wrote: > >> ... Prove it. > > I think I've proved my point. Software RAID1 read balancing provides > > 0%, 300%, 100%, and 100% speedup on 1, 2, 4, and 8 threads, > >