Re: wal_compression=zstd

2022-03-11 Thread Michael Paquier
On Fri, Mar 11, 2022 at 03:49:00PM -0600, Justin Pryzby wrote: > While rebasing, I realized this should have bumped XLOG_PAGE_MAGIC. > > Also, there's a dangling "and". Right. I'll address that a bit later today. Thanks! -- Michael signature.asc Description: PGP signature

Re: wal_compression=zstd

2022-03-11 Thread Justin Pryzby
While rebasing, I realized this should have bumped XLOG_PAGE_MAGIC. Also, there's a dangling "and". +The supported methods are pglz, +lz4 (if PostgreSQL +was compiled with --with-lz4) and +zstd (if PostgreSQL +was compiled with --with-zstd) and +The

Re: wal_compression=zstd

2022-03-10 Thread Justin Pryzby
On Fri, Mar 11, 2022 at 12:23:59PM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 07:14:11AM -0600, Justin Pryzby wrote: > > Anyway there's no compelling reason to not use the default. If we were to > > use > > a non-default default, we'd have to choose between 1 and 2 (or some negative

Re: wal_compression=zstd

2022-03-10 Thread Michael Paquier
On Wed, Mar 09, 2022 at 07:14:11AM -0600, Justin Pryzby wrote: > Anyway there's no compelling reason to not use the default. If we were to use > a non-default default, we'd have to choose between 1 and 2 (or some negative > compression level). My thinking was that zstd-1 would give the lowest-han

Re: wal_compression=zstd

2022-03-09 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 05:44:06AM -0600, Justin Pryzby wrote: > On Fri, Mar 04, 2022 at 04:19:32PM +0900, Michael Paquier wrote: > > On Tue, Feb 22, 2022 at 05:19:48PM -0600, Justin Pryzby wrote: > > > > > As writen, this patch uses zstd level=1 (whereas the ZSTD's default > > > compress > > > l

Re: wal_compression=zstd

2022-03-09 Thread Michael Paquier
On Sat, Mar 05, 2022 at 07:26:39PM +0900, Michael Paquier wrote: > Repeatability and randomness of data counts, we could have for example > one case with a set of 5~7 int attributes, a second with text values > that include random data, up to 10~12 bytes each to count on the tuple > header to be ab

Re: wal_compression=zstd

2022-03-06 Thread Michael Paquier
On Fri, Mar 04, 2022 at 08:10:35AM -0500, Andrew Dunstan wrote: > I don't see why patch 5 shouldn't be applied forthwith. Only applying 0005 would result in a failure in the TAP test for a problem whose fix is attempted in 0006. This is an issue unrelated to this thread. FWIW, I am a bit disturb

Re: wal_compression=zstd

2022-03-05 Thread Michael Paquier
On Fri, Mar 04, 2022 at 08:08:03AM -0500, Robert Haas wrote: > On Fri, Mar 4, 2022 at 6:44 AM Justin Pryzby wrote: >> In my 1-off test, it gets 610/633 = 96% of the benefit at 209/273 = 77% of >> the >> cost. Hmm, it may be good to start afresh and compile numbers in a single chart. I did that

Re: wal_compression=zstd

2022-03-04 Thread Andrew Dunstan
On 2/22/22 18:19, Justin Pryzby wrote: > As writen, this patch uses zstd level=1 (whereas the ZSTD's default compress > level is 6). I think this choice needs to be supported by some benchmarks. > > 0001 adds support for zstd > 0002 makes more efficient our use of bits in the WAL header > 000

Re: wal_compression=zstd

2022-03-04 Thread Robert Haas
On Fri, Mar 4, 2022 at 6:44 AM Justin Pryzby wrote: > > Why? ZSTD using this default has its reasons, no? And it would be > > consistent to do the same for ZSTD as for the other two methods. > > In my 1-off test, it gets 610/633 = 96% of the benefit at 209/273 = 77% of the > cost. I agree with

Re: wal_compression=zstd

2022-03-04 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 04:19:32PM +0900, Michael Paquier wrote: > On Tue, Feb 22, 2022 at 05:19:48PM -0600, Justin Pryzby wrote: > > > As writen, this patch uses zstd level=1 (whereas the ZSTD's default compress > > level is 6). > > Why? ZSTD using this default has its reasons, no? And it woul

Re: wal_compression=zstd

2022-03-03 Thread Michael Paquier
On Tue, Feb 22, 2022 at 05:19:48PM -0600, Justin Pryzby wrote: > I am not claiming that zstd is generally better for WAL. Rather, if we're > going to support alternate compression methods, it's nice to give a couple > options (in addition to pglz). Some use cases would certainly suffer from > slo