Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Mariel Cherkassky
Hi Daniel, I already tried to set the destination table to unlogged - it improved the performance slightly. Is there a way to make sure that I/O is the problem ? 2017-08-17 0:46 GMT+03:00 Daniel Blanch Bataller < daniel.blanch.batal...@gmail.com>: > Seems your disks are too slow. On my laptop (no

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Daniel Blanch Bataller
Seems your disks are too slow. On my laptop (nothing special, just one disk) using COPY I can dump 3G in ~ 20 secs, loading takes 120 secs, bare copying 3G takes 10 secs. Similar proportion you had, but much faster. confirm I/O is your bottleneck, and tell us how you solved your problem Anyw

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Mariel Cherkassky
My server is virtual and it have virtual hd from a vnx storage machine. The logs and the data are on the same disk. 2017-08-16 17:04 GMT+03:00 Daniel Blanch Bataller < daniel.blanch.batal...@gmail.com>: > Considering it has to write logs and data at checkpoints I don’t see it > particularly slow

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Daniel Blanch Bataller
Considering it has to write logs and data at checkpoints I don’t see it particularly slow compared to the extract phase. What kind of disks you have SSD or regular disks? Different disks for ltransaction logs and data? > El 16 ago 2017, a las 15:54, Mariel Cherkassky > escribió: > > I run th

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Mariel Cherkassky
I run the copy command via psql to create a local dump of a 3G table and it took me 134059.732ms =~2 minutes. After that I imported the data via copy and it took 458648.677ms =~7 minutes. So the copy command works but pretty slow. 2017-08-16 16:08 GMT+03:00 Daniel Blanch Bataller < daniel.blanch.b

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Daniel Blanch Bataller
See if the copy command is actually working, copy should be very fast from your local disk. > El 16 ago 2017, a las 14:26, Mariel Cherkassky > escribió: > > > After all the changes of the memory parameters the same operation(without the > copy utility) didnt run much faster - it took one m

Re: [PERFORM] performance problem on big tables

2017-08-16 Thread Mariel Cherkassky
After all the changes of the memory parameters the same operation(without the copy utility) didnt run much faster - it took one minute less. I made a test with the copy command (without the 'with binary') and it took 1.5 hours to create the dumpfile in my local postgresql server. Then I tried to r