On 2022-02-18 09:52:52 -0500, Robert Haas wrote:
> plus a second patch to change the occurrences above to use AC_CHECK_HEADER()
> and remove all traces of the corresponding preprocessor symbol.
LGTM. I'm not entirely sure the gssapi one is a real improvement, because we
kind of test for that bran
On Fri, Feb 18, 2022 at 9:52 AM Robert Haas wrote:
> So here's a revised patch for zstd support that uses
> AC_CHECK_HEADER(), plus a second patch to change the occurrences above
> to use AC_CHECK_HEADER() and remove all traces of the corresponding
> preprocessor symbol.
And I've now committed th
On Fri, Feb 18, 2022 at 9:24 AM Robert Haas wrote:
> On Fri, Feb 18, 2022 at 9:02 AM Robert Haas wrote:
> > Oh wait ... you want it the other way. Yeah, that seems harmless to
> > change. I wonder how many others there are that could be changed
> > similarly...
>
> I went through configure.ac loo
On Fri, Feb 18, 2022 at 9:02 AM Robert Haas wrote:
> Oh wait ... you want it the other way. Yeah, that seems harmless to
> change. I wonder how many others there are that could be changed
> similarly...
I went through configure.ac looking for instances of
AC_CHECK_HEADERS() where the correspondin
On Fri, Feb 18, 2022 at 8:48 AM Robert Haas wrote:
> On Thu, Feb 17, 2022 at 8:36 PM Andres Freund wrote:
> > On 2022-02-17 13:34:08 +0900, Michael Paquier wrote:
> > > %define needs to include HAVE_LIBZSTD, HAVE_ZSTD_H and USE_ZSTD, so
> > > this version fails the sanity check between pg_config.
On Thu, Feb 17, 2022 at 8:36 PM Andres Freund wrote:
> On 2022-02-17 13:34:08 +0900, Michael Paquier wrote:
> > %define needs to include HAVE_LIBZSTD, HAVE_ZSTD_H and USE_ZSTD, so
> > this version fails the sanity check between pg_config.h.in and the
> > MSVC scripts checking that all flags exist.
On Thu, Feb 17, 2022 at 8:18 PM Michael Paquier wrote:
> On Thu, Feb 17, 2022 at 09:40:13AM -0500, Robert Haas wrote:
> > Ah, OK, cool. It seems like we have consensus on this being a good
> > direction, so I plan to commit this later today unless objections or
> > additional review comments turn
Hi,
On 2022-02-17 13:34:08 +0900, Michael Paquier wrote:
> %define needs to include HAVE_LIBZSTD, HAVE_ZSTD_H and USE_ZSTD, so
> this version fails the sanity check between pg_config.h.in and the
> MSVC scripts checking that all flags exist.
Do we really need all three defines? How about using AC
On Thu, Feb 17, 2022 at 09:40:13AM -0500, Robert Haas wrote:
> Ah, OK, cool. It seems like we have consensus on this being a good
> direction, so I plan to commit this later today unless objections or
> additional review comments turn up.
So, will there be a part of the system where we'll make use
On Wed, Feb 16, 2022 at 11:34 PM Michael Paquier wrote:
> Thanks. This looks pretty much right, except for two things that I
> have taken the freedom to fix as of the v3 attached.
Ah, OK, cool. It seems like we have consensus on this being a good
direction, so I plan to commit this later today u
> 15 февр. 2022 г., в 23:20, Robert Haas написал(а):
>
> Anyway, those are my thoughts. What are yours?
+1 for adding Zstd everywhere. Quite similar patches are already a part of
"libpq compression" and "zstd for pg_dump" commitfest entries, so pushing this
will simplify those entries IMV.
On Wed, Feb 16, 2022 at 10:40:01AM -0500, Robert Haas wrote:
> On Tue, Feb 15, 2022 at 9:33 PM Michael Paquier wrote:
>> Yes, the patch misses the fact that each ./configure switch is
>> documented. For consistency, I think that you should also add that in
>> the MSVC scripts in the first version
On Tue, Feb 15, 2022 at 06:24:13PM -0800, Andres Freund wrote:
> For backups it's pretty obviously zstd imo. At the lower levels it achieves
> considerably higher compression ratios while still being vastly faster than
> gzip. Without even using the threaded compression support the library has.
No
On Tue, Feb 15, 2022 at 11:54:10AM -0800, Andres Freund wrote:
> > Isn't it an incontrovertible fact that LZ4 is superior to pglz in
> > every way? LZ4 is pretty much its successor. And so it seems totally
> > fine to assume that users will always want to use the clearly better
> > option, and that
On Tue, Feb 15, 2022 at 9:33 PM Michael Paquier wrote:
> On Tue, Feb 15, 2022 at 02:33:32PM -0600, Justin Pryzby wrote:
> > What I mean is that any patches which *use* the zstd support should update
> > those for completeness.
> >
> > doc/src/sgml/config.sgml | 14 +--
Hi,m
On 2022-02-15 20:56:15 -0500, Tom Lane wrote:
> Maybe we have a bit more flexibility for TOAST, not sure.
It'd be nice to at least add it as an option for initdb. Afaics there's no way
to change the default at that point. initdb itself is measurably
faster. Although sadly it's a bigger diffe
On Tue, Feb 15, 2022 at 02:33:32PM -0600, Justin Pryzby wrote:
> What I mean is that any patches which *use* the zstd support should update
> those for completeness.
>
> doc/src/sgml/config.sgml | 14 +-
> doc/src/sgml/install-windows.sgml | 2 +-
> d
Hi,
On 2022-02-16 09:58:44 +0900, Michael Paquier wrote:
> On Tue, Feb 15, 2022 at 01:42:55PM -0800, Andres Freund wrote:
> > I think well before removing stuff we should default to decent compression
> > algorithms. E.g. -Z/--compress should probably not continue to use gzip if
> > better things
Robert Haas writes:
> On Tue, Feb 15, 2022 at 7:09 PM David Steele wrote:
>> This is even more true for zstd since it is not as ubiquitous as lz4. In
>> fact, it is not even included with base RHEL8. You need to install EPEL
>> to get zstd.
> Yeah, I agree. One thing I was thinking about but did
On Tue, Feb 15, 2022 at 7:09 PM David Steele wrote:
> I'm not sure that this is entirely true. lz4 is available on most
> non-EOL distros but you don't need to look to far to find a distro that
> does not have it. So, choosing lz4 by default could impact binary
> portability. Of course, there are
On Tue, Feb 15, 2022 at 01:42:55PM -0800, Andres Freund wrote:
> I think well before removing stuff we should default to decent compression
> algorithms. E.g. -Z/--compress should probably not continue to use gzip if
> better things are available.
Which one of lz4 or zstd would it be though? LZ4
On 2/15/22 16:10, Peter Geoghegan wrote:
On Tue, Feb 15, 2022 at 12:00 PM Robert Haas wrote:
I'm not sure I completely follow this. There are cases where we use
compression algorithms for internal purposes, and we can change the
defaults without users knowing or caring. For example, we could de
On Tue, Feb 15, 2022 at 12:00 PM Robert Haas wrote:
> I'm not sure I completely follow this. There are cases where we use
> compression algorithms for internal purposes, and we can change the
> defaults without users knowing or caring. For example, we could decide
> that LZ4-compressing FPIs in WA
Hi,
On 2022-02-15 15:09:37 -0500, Robert Haas wrote:
> On Tue, Feb 15, 2022 at 2:54 PM Andres Freund wrote:
> > There's a difference between downloading a source tarball of 1.5x the size,
> > and 3x the decompression cost (made up numbers), because the cost of either
> > is
> > not a relevant fa
On Tue, Feb 15, 2022 at 03:23:30PM -0500, Robert Haas wrote:
> On Tue, Feb 15, 2022 at 1:55 PM Justin Pryzby wrote:
> > Regarding the patch:
> >
> > I suppose it should support windows, and whatever patches use zstd should
> > update the install instructions. See 9ca40dcd4, 02a93e7ef, 4035cd5d4.
On Tue, Feb 15, 2022 at 1:55 PM Justin Pryzby wrote:
> Regarding the patch:
>
> I suppose it should support windows, and whatever patches use zstd should
> update the install instructions. See 9ca40dcd4, 02a93e7ef, 4035cd5d4.
Thanks, this is helpful. To me, it looks like we need something
simila
On Tue, Feb 15, 2022 at 2:54 PM Andres Freund wrote:
> There's a difference between downloading a source tarball of 1.5x the size,
> and 3x the decompression cost (made up numbers), because the cost of either is
> not a relevant factor. I think basebackups are a different story.
To be clear, I'm
On Tue, Feb 15, 2022 at 2:40 PM Peter Geoghegan wrote:
> On Tue, Feb 15, 2022 at 10:20 AM Robert Haas wrote:
> > In general, deciding on new compression algorithms can feel a bit like
> > debating the merits of vi vs. emacs, or one political party vs.
> > another.
>
> Really? I don't get that imp
Hi,
On 2022-02-15 11:40:20 -0800, Peter Geoghegan wrote:
> On Tue, Feb 15, 2022 at 10:20 AM Robert Haas wrote:
> > Likewise, I still download the .tar.gz version of anything
> > that gives me that option, basically because I'm familiar with the
> > format and it's easy for me to just carry on usi
On Tue, Feb 15, 2022 at 10:20 AM Robert Haas wrote:
> In general, deciding on new compression algorithms can feel a bit like
> debating the merits of vi vs. emacs, or one political party vs.
> another.
Really? I don't get that impression myself. (That's not important, though.)
> What I imagine i
I think the WAL patch (4035cd5d4) should support zstd if library support is
added. A supplementary patch for that already exists.
https://www.postgresql.org/message-id/ynqwd2gsmrnqw...@paquier.xyz
There's also some in-progress patches:
- Konstantin and Daniil have a patch to add libpq compressi
Hi,
Over on the rather-long "refactoring basebackup.c" thread, there is a
proposal, which I endorse, to add base-backup compression via zstd. To
do that, we'd need to patch configure.ac to create a new --with-zstd
flag and appropriate supporting infrastructure. My colleague Jeevan
Ladhe included t
32 matches
Mail list logo
