Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-07 Thread Alvaro Herrera
Jim C. Nasby wrote: > On Tue, Aug 01, 2006 at 08:42:23PM -0400, Alvaro Herrera wrote: > > Most likely ext3 was used on the default configuration, which logs data > > operations as well as metadata, which is what XFS logs. I don't think > > I've seen any credible comparison between XFS and ext3 wi

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-04 Thread Jim C. Nasby
On Tue, Aug 01, 2006 at 08:42:23PM -0400, Alvaro Herrera wrote: > J. Andrew Rogers wrote: > > > > On Aug 1, 2006, at 2:49 PM, Milen Kulev wrote: > > >Is anyone using XFS for storing/retrieving relatively large amount > > >of data (~ 200GB)? > > > > > > Yes, we've been using it on Linux since

Re: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-04 Thread Denis Lussier
I agree that OCFS 2.0 is NOT a general purpose PG (or any other) solution.  My recollection is that OCFS gave about 15% performance improvements (same as setting some aggressive switches on ext3).   I assume OCFS has excellent crash safety with its default settings but we did not test this as of ye

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-03 Thread Luke Lonergan
Milen, On 8/3/06 12:44 PM, "Milen Kulev" <[EMAIL PROTECTED]> wrote: > Hi Luke, > That is ~ 50% increase !! Amazing... > How many reader processes did you have to get this results ? Just one - I'll refresh the results sometime and post. - Luke ---(end of broadcast)--

Re: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-03 Thread Chris Browne
[EMAIL PROTECTED] ("Denis Lussier") writes: > I have no personal experience with XFS, but, I've seen numerous > internal edb-postgres test results that show that of all file > systems... OCFS 2.0 seems to be quite good for PG update intensive > apps (especially on 64 bit machines). I have been cur

Re: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-03 Thread Milen Kulev
gust 03, 2006 7:36 AMTo: Luke LonerganCc: Milen Kulev; pgsql-performance@postgresql.orgSubject: Re: [PERFORM] XFS filessystem for Datawarehousing -2I was kinda thinking that making the Block Size configurable at InitDB time would be a nice & simple enhancement for PG 8.3.  My own

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-03 Thread Milen Kulev
-performance@postgresql.org Subject: Re: [PERFORM] XFS filessystem for Datawarehousing Again - the performance difference increases as the disk speed increases. Our experience is that we went from 300MB/s to 475MB/s when moving from ext3 to xfs. - Luke On 8/2/06 4:33 PM, "Michael Stone&quo

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-03 Thread Michael Stone
On Thu, Aug 03, 2006 at 01:10:39AM -0600, Koth, Christian (DWBI) wrote: For what reason are you planning to use a journaling FS? I think using WAL, fsyncing every transaction and using a journaling FS is tautologous. And if you have problems using EXT2 you can just add the journal later without

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-03 Thread Florian Weimer
* Christian Koth: > For what reason are you planning to use a journaling FS? I think > using WAL, fsyncing every transaction and using a journaling FS is > tautologous. The journal is absolutely required to preserve the integrity of the file system's own on-disk data structures after a crash. Ev

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-03 Thread Koth, Christian (DWBI)
Milen, > XFS, EXT3, JFS For what reason are you planning to use a journaling FS? I think using WAL, fsyncing every transaction and using a journaling FS is tautologous. And if you have problems using EXT2 you can just add the journal later without loosing data. My tests using EXT2 showed a per

Re: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-02 Thread Denis Lussier
I was kinda thinking that making the Block Size configurable at InitDB time would be a nice & simple enhancement for PG 8.3.  My own personal rule of thumb for sizing is 8k for OLTP, 16k for mixed use, & 32k for DWH. I have no personal experience with XFS, but, I've seen numerous internal edb-postg

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Luke Lonergan
Again - the performance difference increases as the disk speed increases. Our experience is that we went from 300MB/s to 475MB/s when moving from ext3 to xfs. - Luke On 8/2/06 4:33 PM, "Michael Stone" <[EMAIL PROTECTED]> wrote: > On Wed, Aug 02, 2006 at 02:26:39PM -0700, Steve Poe wrote: >> F

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Michael Stone
On Wed, Aug 02, 2006 at 02:26:39PM -0700, Steve Poe wrote: For the past year, I have been running odbc-bench on a dual-opteron with 4GB of RAM using a 8GB sample data. I found the performance difference between EXT3, JFS, and XFS is +/- 5-8%. That's not surprising when your db is only 2x your

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Chris Browne
[EMAIL PROTECTED] ("Milen Kulev") writes: > I am pretty exited whether XFS will clearly outpertform ETX3 (no > default setups for both are planned !). I am not sure whether is it > worth to include JFS in comparison too ... I did some benchmarking about 2 years ago, and found that JFS was a few p

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Milen Kulev
y, August 02, 2006 4:43 AMTo: Milen Kulev; pgsql-performance@postgresql.orgSubject: Re: [PERFORM] XFS filessystem for Datawarehousing Milen,On 8/1/06 2:49 PM, "Milen Kulev" <[EMAIL PROTECTED]> wrote:> Is anyone using XFS for storing/retrieving relatively larg

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Steve Poe
etups for both are planned !).  I am not surewhether is it worth to include JFS in comparison too ...Best  Regards,Milen Kulev-Original Message-From: Luke Lonergan [mailto: [EMAIL PROTECTED]]Sent: Wednesday, August 02, 2006 4:43 AMTo: Milen Kulev; pgsql-performance@postgresql.orgSubject: Re: [PE

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-02 Thread Milen Kulev
-performance@postgresql.org Subject: Re: [PERFORM] XFS filessystem for Datawarehousing Milen, On 8/1/06 2:49 PM, "Milen Kulev" <[EMAIL PROTECTED]> wrote: > Is anyone using XFS for storing/retrieving relatively large amount of > data (~ 200GB)? I concur with the previou

Re: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-01 Thread Luke Lonergan
Milen, On 8/1/06 3:19 PM, "Milen Kulev" <[EMAIL PROTECTED]> wrote: > Sorry, forgot to ask: > What is the recommended/best PG block size for DWH database? 16k, 32k, 64k > ? > What hsould be the relation between XFS/RAID stripe size and PG block size ? We have found that the page size in PG st

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread Luke Lonergan
Milen, On 8/1/06 2:49 PM, "Milen Kulev" <[EMAIL PROTECTED]> wrote: > Is anyone using XFS for storing/retrieving relatively large amount of data (~ > 200GB)? I concur with the previous poster's experiences with one additional observation: We have had instabilities with XFS with software RAID (m

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread Mark Kirkwood
Milen Kulev wrote: Is anyone using XFS for storing/retrieving relatively large amount of data (~ 200GB)? Yes, but not for that large - only about 40-50 GB of database data. If yes, what about the performance and stability of XFS. I'm pretty happy with the performance, particularly read

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread Alvaro Herrera
J. Andrew Rogers wrote: > > On Aug 1, 2006, at 2:49 PM, Milen Kulev wrote: > >Is anyone using XFS for storing/retrieving relatively large amount > >of data (~ 200GB)? > > > Yes, we've been using it on Linux since v2.4 (currently v2.6) and it > has been rock solid on our database servers (Op

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread J. Andrew Rogers
On Aug 1, 2006, at 2:49 PM, Milen Kulev wrote: Is anyone using XFS for storing/retrieving relatively large amount of data (~ 200GB)? Yes, we've been using it on Linux since v2.4 (currently v2.6) and it has been rock solid on our database servers (Opterons, running in both 32-bit and 64-

Re: [PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread Milen Kulev
, 2006 12:47 AM To: Milen Kulev Cc: Pgsql-Performance ((E-mail)) Subject: Re: [PERFORM] XFS filessystem for Datawarehousing On Aug 1, 2006, at 2:49 PM, Milen Kulev wrote: > Is anyone using XFS for storing/retrieving relatively large amount > of data (~ 200GB)? Yes, we've been using

FW: [PERFORM] XFS filessystem for Datawarehousing -2

2006-08-01 Thread Milen Kulev
Of Milen Kulev Sent: Tuesday, August 01, 2006 11:50 PM To: pgsql-performance@postgresql.org Subject: [PERFORM] XFS filessystem for Datawarehousing I intend to test Postgres/Bizgres for DWH use. I want to use XFS filesystem to get the best possible performance at FS level(correct me if I am

[PERFORM] XFS filessystem for Datawarehousing

2006-08-01 Thread Milen Kulev
I intend to test Postgres/Bizgres for DWH use. I want to use XFS filesystem to get the best possible performance at FS level(correct me if I am wrong !). Is anyone using XFS for storing/retrieving relatively large amount of data (~ 200GB)? If yes, what about the performance and stability of