On Tue, Jan 25, 2022 at 8:15 PM Michael Paquier wrote:
> Sure, but I don't think that it is a good idea to unify that yet, at
> least not until pg_dump is able to handle LZ4 as an option, as the
> main benefit that we'd gain here is to be able to change the code to a
> switch/case without defaults
On Tue, Jan 25, 2022 at 03:14:13PM -0500, Robert Haas wrote:
> On Sat, Jan 22, 2022 at 12:47 AM Michael Paquier wrote:
>> Also, having this enum in walmethods.h is perhaps not the best place
>> either, even more if you plan to use that in pg_basebackup for the
>> server-side compression. One idea
On Sat, Jan 22, 2022 at 12:47 AM Michael Paquier wrote:
> Also, having this enum in walmethods.h is perhaps not the best place
> either, even more if you plan to use that in pg_basebackup for the
> server-side compression. One idea is to rename this enum to
> DataCompressionMethod, moving it into
On Fri, Jan 21, 2022 at 09:57:41AM -0500, Robert Haas wrote:
> Thanks. One thing I just noticed is that the enum we're using here is
> called WalCompressionMethod. But we're not compressing WAL. We're
> compressing tarfiles of the data directory.
Also, having this enum in walmethods.h is perhaps n
On Thu, Jan 20, 2022 at 9:18 PM Michael Paquier wrote:
> On Thu, Jan 20, 2022 at 10:25:43AM -0500, Robert Haas wrote:
> > You don't need to test for gzip and none in two places each. Just make
> > the block with the "It does not match ..." comment the "else" clause
> > for this last part.
>
> Inde
On Thu, Jan 20, 2022 at 10:25:43AM -0500, Robert Haas wrote:
> You don't need to test for gzip and none in two places each. Just make
> the block with the "It does not match ..." comment the "else" clause
> for this last part.
Indeed, that looks better. I have done an extra pass on this stuff
thi
On Thu, Jan 20, 2022 at 2:03 AM Michael Paquier wrote:
> Well, if no colon is specified, we still need to check if optarg
> is a pure integer if it does not match any of the supported methods,
> as --compress=0 should be backward compatible with no compression and
> --compress=1~9 should imply gzi
On Wed, Jan 19, 2022 at 08:35:23AM -0500, Robert Haas wrote:
> I think that this will reject something like --compress=nonetheless by
> telling you that 't' is not a valid separator. I think it would be
> better to code this so that we first identify the portion preceding
> the first colon, or the
On Wed, Jan 19, 2022 at 12:50:44PM -0300, Alvaro Herrera wrote:
> Note there is an extra [ before the {gzip bit.
Thanks!
--
Michael
signature.asc
Description: PGP signature
On 2022-Jan-19, Michael Paquier wrote:
> + printf(_(" -Z, --compress=[{gzip,none}[:LEVEL] or [LEVEL]\n"
> + " compress tar output with
> given compression method or level\n"));
Note there is an extra [ before the {gzip bit.
--
Álvaro Herrera
On Tue, Jan 18, 2022 at 11:27 PM Michael Paquier wrote:
> WFM. Attached is a patch that extends --compress to handle a method
> with an optional compression level. Some extra tests are added to
> cover all that.
I think that this will reject something like --compress=nonetheless by
telling you
On Tue, Jan 18, 2022 at 10:04:56AM -0500, Robert Haas wrote:
> I think it could make sense for you implement
> --compress=METHOD[:LEVEL], keeping -z and -Z N as synonyms for
> --compress=gzip and --compress=gzip:N, and with --compress=N being
> interpreted as --compress=gzip:N. Then I'll generalize
On Mon, Jan 17, 2022 at 8:36 PM Michael Paquier wrote:
> On Mon, Jan 17, 2022 at 12:48:12PM -0500, Robert Haas wrote:
> > Alvaro's proposal is fine with me. I don't see any value in replacing
> > --compress with --compression. It's longer but not superior in any way
> > that I can see. Having both
On Mon, Jan 17, 2022 at 12:48:12PM -0500, Robert Haas wrote:
> Alvaro's proposal is fine with me. I don't see any value in replacing
> --compress with --compression. It's longer but not superior in any way
> that I can see. Having both seems worst of all -- that's just
> confusing.
Okay, that look
On Mon, Jan 17, 2022 at 11:50 AM David G. Johnston
wrote:
>> I think having a single option where you specify everything is simpler.
>> I propose we accept these forms:
>>
>> --compress=[{server,client}-]method[:level] new in 15
>> --compress=level(accepted by 14)
>> -Z level
On Mon, Jan 17, 2022 at 8:41 AM Alvaro Herrera
wrote:
> On 2022-Jan-17, Robert Haas wrote:
>
> > Of the two
> > alternatives that you propose, I prefer --compress=["server-"]METHOD
> > and --compression-level=NUMBER to having both
> > --client-compression-level and --server-compression-level. To
On Mon, Jan 17, 2022 at 8:17 AM Robert Haas wrote:
> On Mon, Jan 17, 2022 at 9:27 AM Magnus Hagander
> wrote:
> > I mean that I think it would be confusing to have
> > --client-compression=x, --server-compression=y, and
> > compression-level=z as the options. Why, in that scenario, does the
> >
On 2022-Jan-17, Robert Haas wrote:
> Of the two
> alternatives that you propose, I prefer --compress=["server-"]METHOD
> and --compression-level=NUMBER to having both
> --client-compression-level and --server-compression-level. To me,
> that's still a bit more surprising than my proposal, because
On Mon, Jan 17, 2022 at 9:27 AM Magnus Hagander wrote:
> I mean that I think it would be confusing to have
> --client-compression=x, --server-compression=y, and
> compression-level=z as the options. Why, in that scenario, does the
> "compression" part get two parameters, but the "compression level
On Mon, Jan 17, 2022 at 4:56 AM Michael Paquier wrote:
>
> On Sat, Jan 15, 2022 at 04:15:26PM +0100, Magnus Hagander wrote:
> > I think having --client-compress and --server-compress separately but
> > having --compression-level *not* being separate would be confusing and
> > I *think* that's what
On Mon, Jan 17, 2022 at 3:18 PM Robert Haas wrote:
>
> On Sat, Jan 15, 2022 at 10:15 AM Magnus Hagander wrote:
> > It never makes sense to compress *both* in server and client, right?
>
> Yeah.
>
> > One argument in that case for using --compress would be that we could
> > have that one take opti
On Sat, Jan 15, 2022 at 10:15 AM Magnus Hagander wrote:
> It never makes sense to compress *both* in server and client, right?
Yeah.
> One argument in that case for using --compress would be that we could
> have that one take options like --compress=gzip (use gzip in the
> client) and --compress
On Fri, Jan 14, 2022 at 9:54 PM Michael Paquier wrote:
> Okay. So, based on this feedback, I guess that something like the
> attached would be what we are looking for. I have maximized the
> amount of code removed with the removal of -z/-Z, but I won't fight
> hard if the consensus is to keep t
On Sat, Jan 15, 2022 at 04:15:26PM +0100, Magnus Hagander wrote:
> I think having --client-compress and --server-compress separately but
> having --compression-level *not* being separate would be confusing and
> I *think* that's what the current patch proposes?
Yep, your understanding is right. T
On Fri, Jan 14, 2022 at 10:53 PM Robert Haas wrote:
>
> On Thu, Jan 13, 2022 at 10:23 PM Michael Paquier wrote:
> > Using --compression-level=NUMBER and --server-compress=METHOD to
> > specify a server-side compression method with a level is fine by me,
> > but I find the reuse of --compress to s
On Fri, Jan 14, 2022 at 04:53:12PM -0500, Robert Haas wrote:
> On Thu, Jan 13, 2022 at 10:23 PM Michael Paquier wrote:
>> Using --compression-level=NUMBER and --server-compress=METHOD to
>> specify a server-side compression method with a level is fine by me,
>> but I find the reuse of --compress t
On Thu, Jan 13, 2022 at 10:23 PM Michael Paquier wrote:
> Using --compression-level=NUMBER and --server-compress=METHOD to
> specify a server-side compression method with a level is fine by me,
> but I find the reuse of --compress to specify a compression method
> confusing as it maps with the pas
On Thu, Jan 13, 2022 at 01:36:00PM -0500, Robert Haas wrote:
> 1. If, as you propose, we add a new flag --compression-method=METHOD
> then how will the user specify server-side compression?
This would require a completely different option switch, which is
basically the same thing as what you are s
On Fri, Jan 7, 2022 at 1:43 AM Michael Paquier wrote:
> Which is why things should be checked once all the options are
> processed. I'd recommend that you read the option patch a bit more,
> that may help.
I don't think that the problem here is my lack of understanding. I
have two basic concerns
On Thu, Jan 06, 2022 at 09:27:19AM -0500, Robert Haas wrote:
> Did you mean that -z would be a synonym for --compression-method=gzip?
> It doesn't really make sense for -Z to be that, unless it's also
> setting the compression level.
Yes, I meant "-z", not "-Z", to be a synonym of
--compression-me
On Thu, Jan 6, 2022 at 12:04 AM Michael Paquier wrote:
> Yeah. There are cases for both. I just got to wonder whether it
> makes sense to allow both server-side and client-side compression to
> be used at the same time. That would be a rather strange case, but
> well, with the correct set of op
On Wed, Jan 05, 2022 at 10:22:06AM -0500, Tom Lane wrote:
> I think the existing precedent is to skip the test if tar isn't there,
> cf pg_basebackup/t/010_pg_basebackup.pl. But certainly the majority of
> buildfarm animals have it.
Even Windows environments should be fine, aka recent edc2332.
--
On Wed, Jan 05, 2022 at 10:33:38AM -0500, Robert Haas wrote:
> I think we're going to want to offer both options. We can't know
> whether the user prefers to consume CPU cycles on the server or on the
> client. Compressing on the server has the advantage of potentially
> saving transfer bandwidth,
On Wed, Jan 5, 2022 at 4:17 AM wrote:
> When the backend-side compression is completed, were there really be a need
> for
> client-side compression? If yes, then it seems logical to have distinct
> options
> for them and this cleanup makes sense. If not, then it seems logical to
> maintain
> th
Robert Haas writes:
> Oh, well, if we have a working tar available, then it's not so bad. I
> was thinking we couldn't really count on that, especially on Windows.
I think the existing precedent is to skip the test if tar isn't there,
cf pg_basebackup/t/010_pg_basebackup.pl. But certainly the ma
On Mon, Dec 20, 2021 at 7:43 PM Michael Paquier wrote:
> Yeah, consistency would be good. For the client-side compression of
> LZ4, we have shaped things around the existing --compress option, and
> there is 6f164e6 that offers an API to parse that at option-level,
> meaning less custom error str
On Wednesday, January 5th, 2022 at 9:00 AM, Michael Paquier
wrote:
> On Mon, Jan 03, 2022 at 03:35:57PM +, gkokola...@pm.me wrote:
> > I propose to initialize streamer to NULL when declared at the top of
> > CreateBackupStreamer().
>
> Yes, that may be noisy. Done this way.
Great.
> > You
(Added Jeevan in CC, for awareness)
On Mon, Jan 03, 2022 at 03:35:57PM +, gkokola...@pm.me wrote:
> I propose to initialize streamer to NULL when declared at the top of
> CreateBackupStreamer().
Yes, that may be noisy. Done this way.
> You can see that after the check_pg_config() test, only
Hi,
‐‐‐ Original Message ‐‐‐
On Saturday, December 18th, 2021 at 12:29 PM, Michael Paquier
wrote:
>Hi all,
>(Added Georgios in CC.)
thank you for the shout out.
>When working on the support of LZ4 for pg_receivewal, walmethods.c has
>gained an extra parameter for the compression metho
On Mon, Dec 20, 2021 at 10:19:44AM -0500, Robert Haas wrote:
> One thing we should keep in mind is that I'm also working on adding
> server-side compression, initially with gzip, but Jeevan Ladhe has
> posted patches to extend that to LZ4. So however we structure the
> options they should take that
On Sat, Dec 18, 2021 at 6:29 AM Michael Paquier wrote:
> This is one step toward the introduction of LZ4 in pg_basebackup, but
> this refactoring is worth doing on its own, hence a separate thread to
> deal with this problem first. The options of pg_basebackup are
> reworked to be consistent with
Hi all,
(Added Georgios in CC.)
When working on the support of LZ4 for pg_receivewal, walmethods.c has
gained an extra parameter for the compression method. It gets used on
the DirectoryMethod instead of the compression level to decide which
type of compression is used. One thing that I left out
42 matches
Mail list logo
