Re: [PERFORM] Planner doing seqscan before indexed join

2007-03-29 Thread Marc Mamin
You may try to change the planner's opinion using sub queries. Something like: select * from eventactivity, (select * from keyword_incidents, eventmain, eventgeo where e

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-03-29 Thread Marc Mamin
Hello Peter, If you are dealing with timed data or similar, you may consider to partition your table(s). In order to deal with large data, I've built a "logical" partition system, whereas the target partition is defined by the date of my data (the date is part of the filenames that I import...).

Re: [PERFORM] Weird performance drop

2007-03-29 Thread Vincenzo Romano
On Friday 30 March 2007 01:12 Dave Dutcher wrote: > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of > > Vincenzo Romano > > > > I thought that the query planner usually did a bad job on > > function bodies > > because they'd appear opaque to it. > > In this case it seems to me

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Erik Jones
On Mar 29, 2007, at 5:15 PM, Dimitri wrote: >> > Erik, > > using 'forcedirectio' simply brings your write operations to the > *real* volume - means while you need to write 10 bytes you'll write 10 > bytes (instead of UFS block size (8K)). So it explains me why your > write volume became slowe

[PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-03-29 Thread Peter Schuller
Hello, I am looking to use PostgreSQL for storing some very simple flat data mostly in a single table. The amount of data will be in the hundreds of gigabytes range. Each row is on the order of 100-300 bytes in size; in other words, small enough that I am expecting disk I/O to be seek bound (even

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Greg Smith
On Thu, 29 Mar 2007, Erik Jones wrote: As far as the procs responsible for the writes go, we were unable to see that from the OS level as the guy we had as a systems admin last year totally screwed us with the way he set up the SunCluster on the boxes and we have been unable to run Dtrace whic

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Erik Jones
On Mar 29, 2007, at 7:23 PM, Josh Berkus wrote: Erik, Wow, thanks for the post. We've just started testing the option of sizing shared_buffers bigger than the database, and using forcedirectio in benchmarks at Sun. So far, our experience has been *equal* performance in that configuratio

Re: [PERFORM] Sunfire X4500 recommendations

2007-03-29 Thread david
On Thu, 29 Mar 2007, Matt Smiley wrote: Hi David, Thanks for your feedback! I'm rather a newbie at this, and I do appreciate the critique. First, let me correct myself: The formulas for the risk of loosing data when you loose 2 and 3 disks shouldn't have included the first term (g/n). I'll

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Josh Berkus
Erik, Wow, thanks for the post. We've just started testing the option of sizing shared_buffers bigger than the database, and using forcedirectio in benchmarks at Sun. So far, our experience has been *equal* performance in that configuration, so it's *very* interesting to see you're getting a

Re: [PERFORM] Planner doing seqscan before indexed join

2007-03-29 Thread Dave Dutcher
> From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Dan Harris > > After some digging, I've found that the planner is choosing > to apply a necessary seq scan to the table. Unfortunately, > it's scanning the whole table, when it seems that it could > have joined it to a small

Re: [PERFORM] Weird performance drop

2007-03-29 Thread Dave Dutcher
> From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Vincenzo Romano > > I thought that the query planner usually did a bad job on > function bodies > because they'd appear opaque to it. > In this case it seems to me that the body is opaque only if I > use the "like" > operator.

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Dimitri
>> > Erik, > > using 'forcedirectio' simply brings your write operations to the > *real* volume - means while you need to write 10 bytes you'll write 10 > bytes (instead of UFS block size (8K)). So it explains me why your > write volume became slower. I men 'lower' (not slower) Sorry, that's

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Erik Jones
On Mar 29, 2007, at 2:19 PM, Tom Lane wrote: Erik Jones <[EMAIL PROTECTED]> writes: One very important thing to note here is that the number, or rather rate, of disk writes has not changed. It's the volume of data in those writes that has dropped, along with those transaction mysterious counts

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Tom Lane
Erik Jones <[EMAIL PROTECTED]> writes: > One very important thing to note here is that the number, or rather > rate, of disk writes has not changed. It's the volume of data in > those writes that has dropped, along with those transaction > mysterious counts. Hmm. I'm suddenly thinking abou

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Erik Jones
On Mar 29, 2007, at 12:41 PM, dimitri k wrote: On 3/29/07, Erik Jones <[EMAIL PROTECTED]> wrote: On Mar 29, 2007, at 11:16 AM, Tom Lane wrote: > Erik Jones <[EMAIL PROTECTED]> writes: >> We've recently made a couple changes to our system that have resulted >> in a drastic increase in perform

[PERFORM] Weird performance drop

2007-03-29 Thread Vincenzo Romano
Hi all. I'm running PostgreSQL v8.1.8 (under Linux Ubuntu). A function body is written as "language sql stable". There is just a select for a search in a view with two arguments to do the search. The search is done with equality comparisons. Both the function call and the select alone run very fa

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread dimitri k
Erik, using 'forcedirectio' simply brings your write operations to the *real* volume - means while you need to write 10 bytes you'll write 10 bytes (instead of UFS block size (8K)). So it explains me why your write volume became slower. Now, why TX number is reduced - is a small mystery :) Opti

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Erik Jones
On Mar 29, 2007, at 11:16 AM, Tom Lane wrote: Erik Jones <[EMAIL PROTECTED]> writes: We've recently made a couple changes to our system that have resulted in a drastic increase in performance as well as some very confusing changes to the database statistics, specifically pg_stat_database.xact_c

Re: [PERFORM] Shared buffers, db transactions commited, and write IO on Solaris

2007-03-29 Thread Tom Lane
Erik Jones <[EMAIL PROTECTED]> writes: > We've recently made a couple changes to our system that have resulted > in a drastic increase in performance as well as some very confusing > changes to the database statistics, specifically > pg_stat_database.xact_commit. Here's the details: I'm kin

Re: [PERFORM] Sunfire X4500 recommendations

2007-03-29 Thread Matt Smiley
Hi David, Thanks for your feedback! I'm rather a newbie at this, and I do appreciate the critique. First, let me correct myself: The formulas for the risk of loosing data when you loose 2 and 3 disks shouldn't have included the first term (g/n). I'll give the corrected formulas and tables at