Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-11 Thread Aidan Van Dyk
Seems like this didn't make it through to the list the first time... * Aidan Van Dyk <[EMAIL PROTECTED]> [081106 22:19]: > * David Rees <[EMAIL PROTECTED]> [081106 21:22]: > > > 2. A write barrier instructs the lower level hardware that commands > > issued before the barrier must be written to d

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-11 Thread Matthew Wakeling
On Thu, 6 Nov 2008, Scott Marlowe wrote: Without write barriers, the second we call an fsync it returns true. But that's assuming write barriers work as I understand them. Write barriers do not work as you understand them. Calling fsync always blocks until all the data has made it to safe st

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread David Rees
On Thu, Nov 6, 2008 at 4:03 PM, Scott Marlowe <[EMAIL PROTECTED]> wrote: > On Thu, Nov 6, 2008 at 4:04 PM, Kevin Grittner <[EMAIL PROTECTED]> wrote: >> "Scott Marlowe" <[EMAIL PROTECTED]> wrote: >>> Without write barriers in my file system an fsync request will >>> be immediately returned true, cor

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Scott Marlowe
On Thu, Nov 6, 2008 at 4:04 PM, Kevin Grittner <[EMAIL PROTECTED]> wrote: "Scott Marlowe" <[EMAIL PROTECTED]> wrote: >> On Thu, Nov 6, 2008 at 3:33 PM, Kevin Grittner >> <[EMAIL PROTECTED]> wrote: >> "Scott Marlowe" <[EMAIL PROTECTED]> wrote: I am pretty sure that with no write barrie

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Kevin Grittner
>>> "Scott Marlowe" <[EMAIL PROTECTED]> wrote: > On Thu, Nov 6, 2008 at 3:33 PM, Kevin Grittner > <[EMAIL PROTECTED]> wrote: > "Scott Marlowe" <[EMAIL PROTECTED]> wrote: >>> I am pretty sure that with no write barriers that even a BBU >> hardware >>> caching raid controller cannot guarantee yo

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Scott Marlowe
On Thu, Nov 6, 2008 at 3:33 PM, Kevin Grittner <[EMAIL PROTECTED]> wrote: "Scott Marlowe" <[EMAIL PROTECTED]> wrote: >> I am pretty sure that with no write barriers that even a BBU > hardware >> caching raid controller cannot guarantee your data. > > That seems at odds with this: > > http://os

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Kevin Grittner
>>> "Scott Marlowe" <[EMAIL PROTECTED]> wrote: > I am pretty sure that with no write barriers that even a BBU hardware > caching raid controller cannot guarantee your data. That seems at odds with this: http://oss.sgi.com/projects/xfs/faq.html#wcache_persistent What evidence to you have that

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Scott Marlowe
On Thu, Nov 6, 2008 at 2:05 PM, Scott Carey <[EMAIL PROTECTED]> wrote: > To others that may stumble upon this thread: > Note that Write Barriers can be very important for data integrity when power > loss or hardware failure are a concern. Only disable them if you know the > consequences are mitiga

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Kevin Grittner
>>> "Scott Carey" <[EMAIL PROTECTED]> wrote: > Note that Write Barriers can be very important for data integrity when power > loss or hardware failure are a concern. Only disable them if you know the > consequences are mitigated by other factors (such as a BBU + db using the > WAL log with sync w

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Scott Carey
To others that may stumble upon this thread: Note that Write Barriers can be very important for data integrity when power loss or hardware failure are a concern. Only disable them if you know the consequences are mitigated by other factors (such as a BBU + db using the WAL log with sync writes), o

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Kevin Grittner
>>> "Joshua D. Drake" <[EMAIL PROTECTED]> wrote: > On Thu, 2008-11-06 at 13:02 -0600, Kevin Grittner wrote: >> the new kernel >> defaulted to using write barriers, while the old kernel didn't. Since >> we have a BBU RAID controller, we will add nobarrier to the fstab >> entries. This makes file

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Joshua D. Drake
On Thu, 2008-11-06 at 13:02 -0600, Kevin Grittner wrote: > >>> "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > > If I find a particular tweak to the background writer or some such > is > > particularly beneficial, I'll post again. > > It turns out that it was not the PostgreSQL version which was >

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-06 Thread Kevin Grittner
>>> "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > If I find a particular tweak to the background writer or some such is > particularly beneficial, I'll post again. It turns out that it was not the PostgreSQL version which was primarily responsible for the performance difference. We updated the

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-05 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > I'm trying to quantify the issue, and would appreciate any > suggestions, either for mitigation or collecting useful data to find > the cause of the performance regression. I create a script which > brackets 1000 lines like the following within a sing

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-05 Thread Kevin Grittner
>>> "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > I'm going to get the latest snapshot to see if the issue has changed > for 8.4devel In testing under today's snapshot, it seemed to take 150,000 writes to create and drop 1,000 temporary tables within a database transaction. The numbers for the

Re: [PERFORM] Create and drop temp table in 8.3.4

2008-11-05 Thread Kevin Grittner
>>> "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > We have found one area where jobs are running > much longer and having a greater impact on concurrent jobs -- those > where the programmer creates and drops many temporary tables > (thousands) within a database transaction. I forgot to include th

[PERFORM] Create and drop temp table in 8.3.4

2008-11-05 Thread Kevin Grittner
We recently upgraded the databases for our circuit court applications from PostgreSQL 8.2.5 to 8.3.4. The application software didn't change. Most software runs fine, and our benchmarks prior to the update tended to show a measurable, if not dramatic, performance improvement overall. We have fou