Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-03-27 Thread Bruce Momjian
Where are we on this patch idea? --- Koichi Suzuki wrote: > Sorry for the late responce; > > Gzip can reduce the archive log size about one fourth. My point is > that it can still be large enough.Removing physical lo

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-12 Thread Koichi Suzuki
Further, because pgbench writes many zero values to fixed length columns, gzip can achieve better compression. There're another suggestion to test with longer checkpoint interval. I will post the result. Thanks. Martijn van Oosterhout wrote: > On Fri, Feb 09, 2007 at 01:00:10PM +0900, Koichi

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-09 Thread Martijn van Oosterhout
On Fri, Feb 09, 2007 at 01:00:10PM +0900, Koichi Suzuki wrote: > Further, we can apply gzip to this archive (2.36GB). Final size is > 0.75GB, less than one sixtieth of the original WAL. Note that if you were compressing on the fly, you'll have to tell gzip to regularly flush its buffers to make

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-09 Thread Koichi Suzuki
Further information about the following evaluation: Pgbench throughput was as follows: Full WAL archiving (full_page_writes=on), 48.3GB archive: 123TPS Gzip WAL compress, 8.8GB archive: 145TPS Physical log removal, 2.36GB archive: 148TPS full_page_writes=off, 2.42GB archive: 161TPS Koichi Suzuki

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-08 Thread Koichi Suzuki
Sorry for the late responce; Gzip can reduce the archive log size about one fourth. My point is that it can still be large enough.Removing physical log record (by replacing them with logical log record) from archive log will achieve will shrink the size of the archive log to one twentiet

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-02 Thread Jim Nasby
I thought the drive behind full_page_writes = off was to reduce the amount of data being written to pg_xlog, not to shrink the size of a PITR log archive. ISTM that if you want to shrink a PITR log archive you'd be able to get good results by (b|g)zip'ing the WAL files in the archive. I q

[HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-01-28 Thread Koichi Suzuki
This is a proposal for archive log compression keeping physical log in WAL. In PotgreSQL 8.2, full-page_writes option came back to cut out physical log both from WAL and archive log. To deal with the partial write during the online backup, physical log is written only during the online backup.