On Tue, May 17, 2022 at 4:12 PM Stephen Frost <sfr...@snowman.net> wrote: > I'm getting a bit of deja-vu here from when I was first trying to add > TRUNCATE as a GRANT'able option and being told we didn't want to burn > those precious bits.
Right, it's the same issue ... although in that case there are a lot more bits available than we have here. > But, fine, then I'd suggest to Michael that he work on actively solving > the problem we've now got where we have such a limited number of bits, > and then come back and add Zstd after that's done. I disagree that we > should be pushing back so hard on adding Zstd in general, but if we are > going to demand that we have a way to support more than these few > compression options before ever adding any new ones (considering how > long it's taken Zstd to get to the level it is now, we're talking > about close to a *decade* from such a new algorithm showing up and > getting to a similar level of adoption, and then apparently more because > we don't feel it's 'ready' yet), then let's work towards that and not > complain when it shows up that it's not needed yet (as I fear would > happen ... and just leave us unable to make useful progress). It's kind of ridiculous to talk about "pushing back so hard on adding Zstd in general" when there's like 2 emails expressing only moderate skepticism. I clearly said I wasn't 100% against it. But I want to point out here that you haven't really offered any kind of argument in favor of supporting Zstd. You basically seem to just be arguing that it's dumb to worry about running out of bit space, and I think that's just obviously false. PostgreSQL is full of things that are hard to improve because nearly all of the bit space was gobbled up early on, and there's not much left for future features. The heap tuple header format is an excellent example of this. Surely if we were designing that over again today we wouldn't have expended some of those bits on the things we did. I do understand that Zstandard is a good compression algorithm, and if we had an extensibility mechanism here where one of the four possible bit patterns then indicates that the next byte (or two or four) stores the real algorithm type, then what about adding Zstandard that way instead of consuming one of our four primary bit patterns? That way we'd have this option for people who want it, but we'd have more options for the future instead of fewer. i.e. something like: 00 = PGLZ 01 = LZ4 10 = reserved for future emergencies 11 = extended header with additional type byte (1 of 256 possible values reserved for Zstandard) I wouldn't be worried about getting backed into a corner with that approach. -- Robert Haas EDB: http://www.enterprisedb.com
