Re: [PERFORM] big database performance

2008-01-10 Thread Jared Mauch
On Thu, Jan 10, 2008 at 12:08:39PM +0100, Stephane Bailliez wrote: > Jared Mauch wrote: >> I do large databases in Pg, like 300GB/day of new data. > > That's impressive. Would it be possible to have details on your hardware, > schema and configuration and type of usage ? > > I'm sure there's

Re: [PERFORM] big database performance

2008-01-10 Thread Jared Mauch
On Thu, Jan 10, 2008 at 10:57:46AM +0200, Adrian Moisey wrote: > What sort of information do you need from me ? Ratio of read vs write operations (select vs insert/copy). average number of indicies per table average table size. (analyze verbose if you want to get into m

Re: [PERFORM] big database performance

2008-01-10 Thread Stephane Bailliez
Jared Mauch wrote: I do large databases in Pg, like 300GB/day of new data. That's impressive. Would it be possible to have details on your hardware, schema and configuration and type of usage ? I'm sure there's something to learn in there for a lot of people (or at least for me) Ch

Re: [PERFORM] big database performance

2008-01-10 Thread Adrian Moisey
Hi I do large databases in Pg, like 300GB/day of new data. Need a lot more data on what you're having issues with. That is big! What sort of information do you need from me ? Is your problem with performance database reads? writes? (insert/copy?) How many indicies do you have?

Re: [PERFORM] big database performance

2008-01-10 Thread Adrian Moisey
Hi Also, we're running the db on ext3 with noatime. Should I look at changing or getting rid of journaling ? No (unless you like really long fsck times). data=writeback is safe with PostgreSQL, though. I tested that on a dev box, and I didn't notice a difference when using pgbench -- A

Re: [PERFORM] big database performance

2008-01-09 Thread Simon Riggs
On Wed, 2008-01-09 at 10:18 +0200, Adrian Moisey wrote: > We recently converted to postgres (from mssql) and we're having > performance issues. I think you need to say more about what the performance issues actually are, otherwise everybody will just speculate you to death. -- Simon Riggs

Re: [PERFORM] big database performance

2008-01-09 Thread Greg Smith
On Wed, 9 Jan 2008, Guillaume Smet wrote: On Jan 9, 2008 9:27 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: wal_sync_method = open_sync Do you recommend it in every situation or just because data are on a SAN? Do you have any numbers/real cases explaining this choice. Sync writes are faste

Re: [PERFORM] big database performance

2008-01-09 Thread Jared Mauch
On Wed, Jan 09, 2008 at 12:27:33AM -0800, Joshua D. Drake wrote: > Adrian Moisey wrote: >> Hi >> >> We recently converted to postgres (from mssql) and we're having >> performance issues. Not all the issues are related to postgres, but we're >> trying to sort everything out. Hi,

Re: [PERFORM] big database performance

2008-01-09 Thread Guillaume Smet
Hi Joshua, On Jan 9, 2008 9:27 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > wal_sync_method = open_sync Do you recommend it in every situation or just because data are on a SAN? Do you have any numbers/real cases explaining this choice. Thanks. -- Guillaume ---(end

Re: [PERFORM] big database performance

2008-01-09 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 9 Jan 2008 08:16:48 -0800 Alan Hodgson <[EMAIL PROTECTED]> wrote: > On Wednesday 09 January 2008, Adrian Moisey > <[EMAIL PROTECTED]> wrote: > > > > Also, we're running the db on ext3 with noatime. Should I look at > > changing or getting ri

Re: [PERFORM] big database performance

2008-01-09 Thread Alan Hodgson
On Wednesday 09 January 2008, Adrian Moisey <[EMAIL PROTECTED]> wrote: > > Also, we're running the db on ext3 with noatime. Should I look at > changing or getting rid of journaling ? No (unless you like really long fsck times). data=writeback is safe with PostgreSQL, though.

Re: [PERFORM] big database performance

2008-01-09 Thread Adrian Moisey
Hi We recently converted to postgres (from mssql) and we're having performance issues. Not all the issues are related to postgres, but we're trying to sort everything out. The server is running ubuntu Gutsy with the database stored on a IBM SAN. It is a Dell box with dual quad core 2.00GHz

Re: [PERFORM] big database performance

2008-01-09 Thread Frank Habermann
Hi, what segment size do you use for the san partition? This could also be a bottle neck for db servers. Frank ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PERFORM] big database performance

2008-01-09 Thread Joshua D. Drake
Adrian Moisey wrote: Hi We recently converted to postgres (from mssql) and we're having performance issues. Not all the issues are related to postgres, but we're trying to sort everything out. The server is running ubuntu Gutsy with the database stored on a IBM SAN. It is a Dell box with

[PERFORM] big database performance

2008-01-09 Thread Adrian Moisey
Hi We recently converted to postgres (from mssql) and we're having performance issues. Not all the issues are related to postgres, but we're trying to sort everything out. The server is running ubuntu Gutsy with the database stored on a IBM SAN. It is a Dell box with dual quad core 2.00GHz