On 3/30/22 08:00, Dagfinn Ilmari Mannsåker wrote:
> Robert Haas writes:
>
>> This patch contains a trivial adjustment to
>> PostgreSQL::Test::Cluster::run_log to make it return a useful value
>> instead of not. I think that should be pulled out and committed
>> independently regardless of what h
Robert Haas writes:
> On Wed, Mar 30, 2022 at 8:00 AM Dagfinn Ilmari Mannsåker
> wrote:
>> Robert Haas writes:
>> > This patch contains a trivial adjustment to
>> > PostgreSQL::Test::Cluster::run_log to make it return a useful value
>> > instead of not. I think that should be pulled out and com
On Wed, Mar 30, 2022 at 8:00 AM Dagfinn Ilmari Mannsåker
wrote:
> Robert Haas writes:
> > This patch contains a trivial adjustment to
> > PostgreSQL::Test::Cluster::run_log to make it return a useful value
> > instead of not. I think that should be pulled out and committed
> > independently regar
On 3/30/22 08:06, Dagfinn Ilmari Mannsåker wrote:
> Robert Haas writes:
>
>> On Mon, Mar 28, 2022 at 12:52 PM Dagfinn Ilmari Mannsåker
>> wrote:
True, but that also means it shows up in the actual failure message,
which seems too verbose. By just using 'print', it ends up in the log
>
Robert Haas writes:
> On Mon, Mar 28, 2022 at 12:52 PM Dagfinn Ilmari Mannsåker
> wrote:
>> > True, but that also means it shows up in the actual failure message,
>> > which seems too verbose. By just using 'print', it ends up in the log
>> > file if it's needed, but not anywhere else. Maybe the
Robert Haas writes:
> This patch contains a trivial adjustment to
> PostgreSQL::Test::Cluster::run_log to make it return a useful value
> instead of not. I think that should be pulled out and committed
> independently regardless of what happens to this patch overall, and
> possibly back-patched.
On Mon, Mar 28, 2022 at 12:57 PM Robert Haas wrote:
> Here's an updated and rebased version of my patch.
Well, that only updated the comment on the client side. Let's try again.
--
Robert Haas
EDB: http://www.enterprisedb.com
v3-0001-Allow-parallel-zstd-compression-when-taking-a-bas.patch
Desc
On Sun, Mar 27, 2022 at 4:50 PM Justin Pryzby wrote:
> Actually, I suggest to remove those comments:
> | "We check for failure here because..."
>
> That should be the rule rather than the exception, so shouldn't require
> justifying why one might checks the return value of library and system calls
On Mon, Mar 28, 2022 at 12:52 PM Dagfinn Ilmari Mannsåker
wrote:
> > True, but that also means it shows up in the actual failure message,
> > which seems too verbose. By just using 'print', it ends up in the log
> > file if it's needed, but not anywhere else. Maybe there's a better way
> > to do t
Robert Haas writes:
> On Thu, Mar 24, 2022 at 9:19 AM Dagfinn Ilmari Mannsåker
> wrote:
>> Per the TAP protocol, every line of non-test-result output should be
>> prefixed by "# ". The note() function does this for you, see
>> https://metacpan.org/pod/Test::More#Diagnostics for details.
>
> True
On Thu, Mar 24, 2022 at 9:19 AM Dagfinn Ilmari Mannsåker
wrote:
> Per the TAP protocol, every line of non-test-result output should be
> prefixed by "# ". The note() function does this for you, see
> https://metacpan.org/pod/Test::More#Diagnostics for details.
True, but that also means it shows u
On Wed, Mar 23, 2022 at 06:57:04PM -0400, Robert Haas wrote:
> On Wed, Mar 23, 2022 at 5:52 PM Justin Pryzby wrote:
> > Also because the library may not be compiled with threading. A few days
> > ago, I
> > tried to rebase the original "parallel workers" patch over the COMPRESS
> > DETAIL
> > p
On Wed, Mar 23, 2022 at 7:31 PM Andres Freund wrote:
> I found this the following section in the manual [1]:
>
> ZSTD_c_nbWorkers=400,/* Select how many threads will be spawned to
> compress in parallel.
> * When nbWorkers >= 1, triggers asynchronous
> mode
Hi Robert,
I haven't reviewed the meat of the patch in detail, but I noticed
something in the tests:
Robert Haas writes:
> diff --git a/src/bin/pg_verifybackup/t/009_extract.pl
> b/src/bin/pg_verifybackup/t/009_extract.pl
> index 9f9cc7540b..e17e7cad51 100644
> --- a/src/bin/pg_verifybackup/t/0
On Wed, Mar 23, 2022 at 7:07 PM Justin Pryzby wrote:
> Did you try this on windows at all ? It's probably no surprise that zstd
> implements threading differently there.
I did not. I haven't had a properly functioning Windows development
environment in about a decade.
--
Robert Haas
EDB: http:
On 2022-03-23 18:07:01 -0500, Justin Pryzby wrote:
> Did you try this on windows at all ?
Really should get zstd installed in the windows cf environment...
> It's probably no surprise that zstd implements threading differently there.
Worth noting that we have a few of our own threads running on
Hi,
On 2022-03-23 18:31:12 -0400, Robert Haas wrote:
> On Wed, Mar 23, 2022 at 5:14 PM Andres Freund wrote:
> > The most likely source of problem would errors thrown while zstd threads are
> > alive. Should make sure that that can't happen.
> >
> > What is the lifetime of the threads zstd spawns?
On Wed, Mar 23, 2022 at 04:34:04PM -0400, Robert Haas wrote:
> be, spawning threads inside the PostgreSQL backend. Short of cats and
> dogs living together, it's hard to think of anything more terrifying,
> because the PostgreSQL backend is very much not thread-safe. However,
> a lot of the things
On Wed, Mar 23, 2022 at 5:52 PM Justin Pryzby wrote:
> Also because the library may not be compiled with threading. A few days ago,
> I
> tried to rebase the original "parallel workers" patch over the COMPRESS DETAIL
> patch but then couldn't test it, even after trying various versions of the
>
On Wed, Mar 23, 2022 at 5:14 PM Andres Freund wrote:
> The most likely source of problem would errors thrown while zstd threads are
> alive. Should make sure that that can't happen.
>
> What is the lifetime of the threads zstd spawns? Are they tied to a single
> compression call? A single ZSTD_cre
+* We check for failure here because (1) older versions of the library
+* do not support ZSTD_c_nbWorkers and (2) the library might want to
+* reject an unreasonable values (though in practice it does not seem
to do
+* so).
+*/
+ ret = ZSTD_CCtx_setPar
Hi,
On 2022-03-23 16:34:04 -0400, Robert Haas wrote:
> Therefore, I think it might be safe to just ... turn this on. One reason I
> think that is that this whole approach was recommended to me by Andres ...
I didn't do a super careful analysis of the issues... But I do think it's
pretty much the
[ Changing subject line in the hopes of attracting more eyeballs. ]
On Mon, Mar 14, 2022 at 12:11 PM Dipesh Pandit wrote:
> I tried to implement support for parallel ZSTD compression.
Here's a new patch for this. It's more of a rewrite than an update,
honestly; commit ffd53659c46a54a6978bcb8c442
23 matches
Mail list logo
