Re: Fwd: [PERFORM] wal_sync_methods for AIX

2008-02-19 Thread Greg Smith
On Tue, 19 Feb 2008, JP Fletcher wrote: Shared buffers are set to 5, bgwriter settings are as follows: bgwriter_delay = 50 # 10-1 milliseconds between rounds bgwriter_lru_percent = 20.0 # 0-100% of LRU buffers scanned/round bgwriter_lru_maxpages = 300

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Greg Smith
On Tue, 19 Feb 2008, Douglas J Hunley wrote: The db resides on a HP Modular Storage Array 500 G2. 4x72.8Gb 15k rpm disks. 1 raid 6 logical volume. Compaq Smart Array 6404 controller You might consider doing some simple disk tests on the array just to prove it's working well. Reports here sug

Re: [PERFORM] wal_sync_methods for AIX

2008-02-19 Thread Erik Jones
On Feb 19, 2008, at 3:58 PM, Dan Langille wrote: Erik Jones wrote: On Feb 15, 2008, at 3:55 PM, Dan Langille wrote: We're using PostgreSQL 8.1.11 on AIX 5.3 and we've been doing some playing around with various settings. So far, we've (I say we, but it's another guy doing the work) found

Re: [PERFORM] wal_sync_methods for AIX

2008-02-19 Thread Dan Langille
Erik Jones wrote: On Feb 15, 2008, at 3:55 PM, Dan Langille wrote: We're using PostgreSQL 8.1.11 on AIX 5.3 and we've been doing some playing around with various settings. So far, we've (I say we, but it's another guy doing the work) found that open_datasync seems better than fsync. By how

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Erik Jones
On Feb 19, 2008, at 2:55 PM, Douglas J Hunley wrote: On Tuesday 19 February 2008 15:07:30 Jeff wrote: On Feb 19, 2008, at 1:22 PM, Tom Lane wrote: maintenance_work_mem, to be more specific. If that's too small it will definitely cripple restore speed. I'm not sure fsync would make much di

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 15:07:30 Jeff wrote: > On Feb 19, 2008, at 1:22 PM, Tom Lane wrote: > > maintenance_work_mem, to be more specific. If that's too small it > > will > > definitely cripple restore speed. I'm not sure fsync would make much > > difference, but checkpoint_segments would. S

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 15:16:42 Dave Cramer wrote: > On 19-Feb-08, at 2:35 PM, Douglas J Hunley wrote: > > On Tuesday 19 February 2008 14:28:54 Dave Cramer wrote: > >> shared buffers is *way* too small as is effective cache > >> set them to 2G/6G respectively. > >> > >> Dave > > > > pardon my

Re: Fwd: [PERFORM] wal_sync_methods for AIX

2008-02-19 Thread JP Fletcher
Dan Langille wrote: Begin forwarded message: From: "Joshua D. Drake" <[EMAIL PROTECTED]> Date: February 15, 2008 5:04:53 PM EST To: Dan Langille <[EMAIL PROTECTED]> Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] wal_sync_methods for AIX -BEGIN PGP SIGNED MESSAGE- Hash: S

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Dave Cramer
On 19-Feb-08, at 2:35 PM, Douglas J Hunley wrote: On Tuesday 19 February 2008 14:28:54 Dave Cramer wrote: shared buffers is *way* too small as is effective cache set them to 2G/6G respectively. Dave pardon my ignorance, but is this in the context of a restore only? or 'in general'? Thi

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Jeff
On Feb 19, 2008, at 1:22 PM, Tom Lane wrote: maintenance_work_mem, to be more specific. If that's too small it will definitely cripple restore speed. I'm not sure fsync would make much difference, but checkpoint_segments would. See http://www.postgresql.org/docs/8.3/static/populate.html#

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Jeff Davis
On Tue, 2008-02-19 at 14:28 -0500, Dave Cramer wrote: > shared buffers is *way* too small as is effective cache > set them to 2G/6G respectively. They are way too small, but I don't think that explains the index creation time. Effective_cache_size is only used by the planner, and this problem is

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Jeff Davis
On Tue, 2008-02-19 at 14:20 -0500, Douglas J Hunley wrote: > > Keep in mind, if you have several GB worth of indexes, they take up > > basically no space in the logical dump (just the "CREATE INDEX" command, > > and that's it). But they can take a lot of processor time to build up > > again, especi

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 14:28:54 Dave Cramer wrote: > shared buffers is *way* too small as is effective cache > set them to 2G/6G respectively. > > Dave pardon my ignorance, but is this in the context of a restore only? or 'in general'? -- Douglas J Hunley (doug at hunley.homeip.net) - Linu

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Dave Cramer
shared buffers is *way* too small as is effective cache set them to 2G/6G respectively. Dave ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 13:23:23 Jeff Davis wrote: > On Tue, 2008-02-19 at 13:03 -0500, Douglas J Hunley wrote: > > I spent a whopping seven hours restoring a database late Fri nite for a > > client. We stopped the application, ran pg_dump -v -Ft -b -o $db > > > ~/pre_8.3.tar on the 8.2.x db, a

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 13:22:58 Tom Lane wrote: > Richard Huxton <[EMAIL PROTECTED]> writes: > > Douglas J Hunley wrote: > >> I spent a whopping seven hours restoring a database late Fri nite for a > > > > Oh, and have you tweaked the configuration settings for the restore? > > Lots of work_me

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 13:13:37 Richard Huxton wrote: > Douglas J Hunley wrote: > > I spent a whopping seven hours restoring a database late Fri nite for a > > client. We stopped the application, ran pg_dump -v -Ft -b -o $db > > > ~/pre_8.3.tar on the 8.2.x db, and then upgrading the software

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
On Tuesday 19 February 2008 13:12:54 Joshua D. Drake wrote: > > I spent a whopping seven hours restoring a database late Fri nite for > > a client. We stopped the application, ran pg_dump -v -Ft -b -o $db > > > ~/pre_8.3.tar on the 8.2.x db, and then upgrading the software to > > 8.3. I then did a

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Jeff Davis
On Tue, 2008-02-19 at 13:03 -0500, Douglas J Hunley wrote: > I spent a whopping seven hours restoring a database late Fri nite for a > client. We stopped the application, ran pg_dump -v -Ft -b -o $db > > ~/pre_8.3.tar on the 8.2.x db, and then upgrading the software to 8.3. I then > did a pg_res

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Tom Lane
Richard Huxton <[EMAIL PROTECTED]> writes: > Douglas J Hunley wrote: >> I spent a whopping seven hours restoring a database late Fri nite for a > Oh, and have you tweaked the configuration settings for the restore? > Lots of work_mem, turn fsync off, that sort of thing. maintenance_work_mem, to

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Dave Cramer
On 19-Feb-08, at 1:12 PM, Joshua D. Drake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 19 Feb 2008 13:03:58 -0500 Douglas J Hunley <[EMAIL PROTECTED]> wrote: I spent a whopping seven hours restoring a database late Fri nite for a client. We stopped the application, ran pg_dum

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Richard Huxton
Douglas J Hunley wrote: I spent a whopping seven hours restoring a database late Fri nite for a client. We stopped the application, ran pg_dump -v -Ft -b -o $db > ~/pre_8.3.tar on the 8.2.x db, and then upgrading the software to 8.3. I then did a pg_restore -v -d $db ./pre_8.3.tar and watched i

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 19 Feb 2008 13:03:58 -0500 Douglas J Hunley <[EMAIL PROTECTED]> wrote: > I spent a whopping seven hours restoring a database late Fri nite for > a client. We stopped the application, ran pg_dump -v -Ft -b -o $db > > ~/pre_8.3.tar on the 8.2.x

[PERFORM] 7 hrs for a pg_restore?

2008-02-19 Thread Douglas J Hunley
I spent a whopping seven hours restoring a database late Fri nite for a client. We stopped the application, ran pg_dump -v -Ft -b -o $db > ~/pre_8.3.tar on the 8.2.x db, and then upgrading the software to 8.3. I then did a pg_restore -v -d $db ./pre_8.3.tar and watched it positively crawl. I'll

Re: [PERFORM] Disable WAL completely

2008-02-19 Thread Andrew Sullivan
On Tue, Feb 19, 2008 at 02:48:55PM +, Matthew wrote: > If there's not much write traffic, the WAL won't be used much anyway. You still have checkpoints. > If you really don't care much about the integrity, then the best option is > probably to put the WAL on ramfs. Um, that will cause the

[PERFORM] strange plan choice

2008-02-19 Thread Cousin Marc
Hi, This occurs on postgresql 8.2.5. I'm a bit at loss with the plan chosen for a query : The query is this one : SELECT SULY_SAOEN.SAOEN_ID, SULY_SDCEN.SDCEN_REF, SULY_SDCEN.SDCEN_LIB, CSTD_UTI.UTI_NOM, CSTD_UTI.UTI_LIBC, SULY_SAOEN.SAOEN_DTDERNENVOI, SULY_SDCEN.SDCEN_DTLIMAP, SULY_PF

Re: [PERFORM] Disable WAL completely

2008-02-19 Thread Matthew
On Mon, 18 Feb 2008, Tobias Brox wrote: We actually have some postgres databases that are read-only, others that can be rebuilt by a script or from some old backup, and yet others that can be wiped completely without ill effects ... and others where we would prefer to keep all the data, but it wo