Re: Zstandard support for toast compression

2023-05-23 Thread Michael Paquier
On Tue, May 23, 2023 at 05:56:13PM +0300, Aleksander Alekseev wrote: > TWIMC there is an ongoing discussion [1] of making TOAST pointers > extendable since this is a dependency for several patches that are > currently in development. Thanks for the ping. I have seen and read the other thread, and

Re: Zstandard support for toast compression

2023-05-23 Thread Aleksander Alekseev
Hi, > Yeah - I think we had better reserve the fourth bit pattern for > something extensible e.g. another byte or several to specify the > actual method, so that we don't have a hard limit of 4 methods. TWIMC there is an ongoing discussion [1] of making TOAST pointers extendable since this is a d

Re: Zstandard support for toast compression

2022-05-24 Thread Nikolay Shaplov
В письме от пятница, 20 мая 2022 г. 23:17:42 MSK пользователь Stephen Frost написал: > While I generally agree that we want to provide extensibility in this > area, given that we already have zstd as a compile-time option and it > exists in other parts of the system, I don't think it makes sense

Re: Zstandard support for toast compression

2022-05-23 Thread Robert Haas
On Fri, May 20, 2022 at 4:17 PM Stephen Frost wrote: > A thought I've had before is that it'd be nice to specify a particular > compression method on a data type basis. Wasn't the direction that this > was taken, for reasons, but I wonder about perhaps still having a data > type compression metho

Re: Zstandard support for toast compression

2022-05-23 Thread Robert Haas
On Mon, May 23, 2022 at 12:33 AM Michael Paquier wrote: > On Thu, May 19, 2022 at 04:12:01PM -0400, Robert Haas wrote: > > On Thu, May 19, 2022 at 4:20 AM Michael Paquier wrote: > >> Btw, shouldn't we have something a bit more, err, extensible for the > >> design of an extensible varlena header?

Re: Zstandard support for toast compression

2022-05-22 Thread Michael Paquier
On Thu, May 19, 2022 at 04:12:01PM -0400, Robert Haas wrote: > On Thu, May 19, 2022 at 4:20 AM Michael Paquier wrote: >> Btw, shouldn't we have something a bit more, err, extensible for the >> design of an extensible varlena header? If we keep it down to some >> bitwise information, we'd be fine

Re: Zstandard support for toast compression

2022-05-20 Thread Stephen Frost
Greetings, * Nikolay Shaplov (dh...@nataraj.su) wrote: > В письме от вторник, 17 мая 2022 г. 23:01:07 MSK пользователь Tom Lane > написал: > > Hi! I came to this branch looking for a patch to review, but I guess I would > join the discussion instead of reading the code. Seems that's what would

Re: Zstandard support for toast compression

2022-05-19 Thread Robert Haas
On Thu, May 19, 2022 at 4:20 AM Michael Paquier wrote: > Btw, shouldn't we have something a bit more, err, extensible for the > design of an extensible varlena header? If we keep it down to some > bitwise information, we'd be fine for a long time but it would be > annoying to review again an exte

Re: Zstandard support for toast compression

2022-05-19 Thread Nikolay Shaplov
В письме от вторник, 17 мая 2022 г. 23:01:07 MSK пользователь Tom Lane написал: Hi! I came to this branch looking for a patch to review, but I guess I would join the discussion instead of reading the code. > >> Yeah - I think we had better reserve the fourth bit pattern for > >> something exten

Re: Zstandard support for toast compression

2022-05-19 Thread Amit Kapila
On Wed, May 18, 2022 at 9:47 PM Robert Haas wrote: > > I do understand that Zstandard is a good compression algorithm, and if > we had an extensibility mechanism here where one of the four possible > bit patterns then indicates that the next byte (or two or four) stores > the real algorithm type,

Re: Zstandard support for toast compression

2022-05-19 Thread Michael Paquier
On Wed, May 18, 2022 at 12:17:16PM -0400, Robert Haas wrote: > i.e. something like: > > 00 = PGLZ > 01 = LZ4 > 10 = reserved for future emergencies > 11 = extended header with additional type byte (1 of 256 possible > values reserved for Zstandard) Btw, shouldn't we have something a bit more, err

Re: Zstandard support for toast compression

2022-05-18 Thread Peter Geoghegan
On Wed, May 18, 2022 at 9:17 AM Robert Haas wrote: > But I want to point out here that you haven't really offered any kind > of argument in favor of supporting Zstd. You basically seem to just be > arguing that it's dumb to worry about running out of bit space, and I > think that's just obviously

Re: Zstandard support for toast compression

2022-05-18 Thread Justin Pryzby
On Tue, May 17, 2022 at 02:54:28PM -0400, Robert Haas wrote: > I don't particularly have anything against adding Zstandard > compression here, but I wonder whether there's any rush. If we decide > not to add this now, we can always change our minds and add it later, > but if we decide to add it now

Re: Zstandard support for toast compression

2022-05-18 Thread Robert Haas
On Tue, May 17, 2022 at 4:12 PM Stephen Frost wrote: > I'm getting a bit of deja-vu here from when I was first trying to add > TRUNCATE as a GRANT'able option and being told we didn't want to burn > those precious bits. Right, it's the same issue ... although in that case there are a lot more bit

Re: Zstandard support for toast compression

2022-05-17 Thread Michael Paquier
On Tue, May 17, 2022 at 04:12:14PM -0400, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I'm with Robert in that I do not see any urgency to add another method. Okay. >> The fact that Stephen is already questioning whether LZ4 should have >> been added first is not making me any

Re: Zstandard support for toast compression

2022-05-17 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> Yeah - I think we had better reserve the fourth bit pattern for > >> something extensible e.g. another byte or several to specify the > >> actual method, so that we don'

Re: Zstandard support for toast compression

2022-05-17 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> Yeah - I think we had better reserve the fourth bit pattern for >> something extensible e.g. another byte or several to specify the >> actual method, so that we don't have a hard limit of 4 methods. But >> even with such a sys

Re: Zstandard support for toast compression

2022-05-17 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, May 17, 2022 at 12:19 AM Michael Paquier wrote: > > Toast compression is supported for LZ4, and thanks to the refactoring > > work done with compression methods assigned to an attribute, adding > > support for more methods is strai

Re: Zstandard support for toast compression

2022-05-17 Thread Robert Haas
On Tue, May 17, 2022 at 12:19 AM Michael Paquier wrote: > Toast compression is supported for LZ4, and thanks to the refactoring > work done with compression methods assigned to an attribute, adding > support for more methods is straight-forward, as long as we don't > support more than 4 methods as