Re: [PERFORM] Improve BULK insertion

2004-12-05 Thread Josh Berkus
Rodrigo, > 3. My transaction log configuration are : checkpoint_segments = 3  and > checkpoint_timeout = 300 and my transaction logs are on the same disk . Well, you need to move your transaction logs to another disk, and increase them to a large number ... like 128, which is about 1GB (you'll n

Re: [PERFORM] Partitioned table performance

2004-12-05 Thread Josh Berkus
Stacy, Thanks for the stats! > In some cases we've seen some increased performance in tests by splitting > the table into several smaller tables.  Both 'UNION ALL' views, and the > superclass/subclass scheme work well at pruning down the set of rows a > query uses, but they seem to introduce a la

Re: [PERFORM] Improve BULK insertion

2004-12-05 Thread Gregory S. Williamson
Rodrigo -- You should definitely drop the indexes and any other FK constraints before loading and then rebuild them. Check your logs and see if there are warnings about checkpoint intervals -- only 3 logs seems like it might be small; if you have the disk space I would definitely consider raisi

Re: [PERFORM] Improve BULK insertion

2004-12-05 Thread Rodrigo Carvalhaes
Hi! 1. I am doing the inserts using pg_restore. The dump was created using pg_dump and the standard format (copy statements) 2. See below the table schema. There are only 7 indexes. 3. My transaction log configuration are : checkpoint_segments = 3 and checkpoint_timeout = 300 and my transactio

Re: [PERFORM] pg_restore taking 4 hours!

2004-12-05 Thread Rodrigo Carvalhaes
Hi ! Thanks for the lots of tips that I received on this matter. Some points: 1. I bumped the sort_mem and vaccum_mem to 202800 (200mb each) and the performance was quite the same , the total difference was 10 minutes 2. I made the restore without the index and the total time was 3 hours so, I d