Hannu, On 2/26/06 12:19 PM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote:
>> On DBT-3 data, I've just run some tests meant to simulate the speed >> differences of compression versus native I/O. My thought is that an >> external use of gzip on a binary dump file should be close to the speed of >> LZW on toasted fields, > > Your basic assumption si probbaly wrong :( > > gzip what ? "compression level" setting of gzip has big effect on both > compression speed and compression rate. And I suspect that even the > fastest level (gzip -1) compresses slower and better than postgresql's > lzcompress. I thought it might form a lower bound on speed. 75MB/s on an Opteron is really super fast compared to what I expected from LZ. And since gzip uses LZ compression, maybe they'll be in the same family of results, behavior-wise. I'd be more concerned about the implementation within the executor and how the "get tuple, decompress tuple, use tuple" cycling might destroy the efficiency of the scanning compared to decompressing large blocks of data. As Jim pointed out, we would need a real test to confirm the behavior, I'm not yet acquainted with the toast compression, so it's harder for me to compose a real test. > these are also somewhat bogus tests, if you would want them to be > comparable with dd below, you should have used 'time gzip -c > supplier.bin > /dev/null' Hmm - the file writing seems to matter on decompress, making decompression even faster: [EMAIL PROTECTED] tmp]# time gunzip tmp.bin.gz real 0m2.254s user 0m1.718s sys 0m0.536s [EMAIL PROTECTED] tmp]# time gzip tmp.bin real 0m12.912s user 0m12.555s sys 0m0.355s [EMAIL PROTECTED] tmp]# time gzip -c supplier.bin > /dev/null real 0m12.582s user 0m12.464s sys 0m0.115s [EMAIL PROTECTED] tmp]# time gunzip -c tmp.bin.gz > /dev/null real 0m1.734s user 0m1.690s sys 0m0.043s - Luke ---------------------------(end of broadcast)--------------------------- TIP 2: Don't 'kill -9' the postmaster
