Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Robert Haas
On Fri, Dec 18, 2009 at 10:51 AM, Sigurgeir Gunnarsson wrote: > The intention was never to talk down postgresql but rather trying to get > some explanation of this difference so that I could do the proper changes. > > After having read the link from Euler's post, which I oversaw, I have > managed

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Sigurgeir Gunnarsson
The intention was never to talk down postgresql but rather trying to get some explanation of this difference so that I could do the proper changes. After having read the link from Euler's post, which I oversaw, I have managed to shorten the import time. My problem was with the indexes. I was able

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Robert Haas
On Fri, Dec 18, 2009 at 7:46 AM, Sigurgeir Gunnarsson wrote: > I hope the issue is still open though I haven't replied to it before. > > Euler mentioned that I did not provide any details about my system. I'm > using version 8.3 and with most settings default on an old machine with 2 GB > of mem.

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Sigurgeir Gunnarsson
I hope the issue is still open though I haven't replied to it before. Euler mentioned that I did not provide any details about my system. I'm using version 8.3 and with most settings default on an old machine with 2 GB of mem. The table definition is simple, four columns; id, value, x, y where id

Re: [PERFORM] Issues with \copy from file

2009-10-19 Thread Matthew Wakeling
On Sun, 18 Oct 2009, Scott Marlowe wrote: You can only write data then commit it so fast to one drive, and that speed is usually somewhere in the megabyte per second range. 450+150 in 5 minutes is 120 Megs per second, that's pretty fast, but is likely the max speed of a modern super fast 15k rpm

Re: [PERFORM] Issues with \copy from file

2009-10-17 Thread Scott Marlowe
On Mon, Oct 12, 2009 at 4:05 PM, Sigurgeir Gunnarsson wrote: > I'm doing \copy from file into table. There are two files one with 7 million > lines and the other with around 24 million and the data goes into separate > table. There are only three columns in each file and four in each table (the >

Re: [PERFORM] Issues with \copy from file

2009-10-17 Thread Euler Taveira de Oliveira
Sigurgeir Gunnarsson escreveu: > What I'm wondering about is what parameters to tweak to improve the > operation and shorten the time of the \copy ? I think I have tweaked > most of the available in postgresql.conf, that is shared_buffer, > temp_buffers, work_mem, maintenance_work_mem, max_fsm_page