On Fri, Jul 28, 2023 at 12:06:33PM +0200, Alvaro Herrera wrote:
> Hmm, I think having all the workers doing their in the same table is
> better -- if nothing else, because it gives us the opportunity to show
> how to use some other coding technique (but also because we are forced
> to write the SQL
On Fri, Jul 28, 2023 at 01:34:15PM -0700, Andres Freund wrote:
> On 2023-07-28 13:45:29 +0900, Michael Paquier wrote:
>> Having each bgworker on its own schema would be enough to prevent
>> conflicts, but I'd like to add a second thing: a check on
>> pg_stat_activity.wait_event after starting the w
Hi,
On 2023-07-28 13:45:29 +0900, Michael Paquier wrote:
> Having each bgworker on its own schema would be enough to prevent
> conflicts, but I'd like to add a second thing: a check on
> pg_stat_activity.wait_event after starting the workers. I have added
> something like that in the patch I have
On 2023-Jul-28, Michael Paquier wrote:
> So you have faced a race condition where the commit of the transaction
> doing the schema creation for the static workers is delayed long
> enough that the dynamic workers don't see it, and bumped on a catalog
> conflict when they try to create the same sch
On Fri, Jul 28, 2023 at 02:11:48PM +0530, Bharath Rupireddy wrote:
> I don't think something like [1] is complex. It makes worker_spi
> foolproof. Rather, the other approach proposed, that is to provide
> non-conflicting worker IDs to worker_spi_launch in the TAP test file,
> looks complicated to m
On Fri, Jul 28, 2023 at 1:26 PM Michael Paquier wrote:
>
> On Fri, Jul 28, 2023 at 10:47:39AM +0530, Bharath Rupireddy wrote:
> > +# check their existence. Use IDs that do not overlap with the schemas
> > created
> > +# by the previous workers.
> >
> > While using different IDs in tests is a sim
On Fri, Jul 28, 2023 at 10:47:39AM +0530, Bharath Rupireddy wrote:
> +# check their existence. Use IDs that do not overlap with the schemas
> created
> +# by the previous workers.
>
> While using different IDs in tests is a simple fix, -1 for it. I'd
> prefer if worker_spi uses different schema
On Fri, Jul 28, 2023 at 10:15 AM Michael Paquier wrote:
>
> Having each bgworker on its own schema would be enough to prevent
> conflicts, but I'd like to add a second thing: a check on
> pg_stat_activity.wait_event after starting the workers. I have added
> something like that in the patch I hav
On Thu, Jul 27, 2023 at 07:23:32PM -0700, Andres Freund wrote:
> As written, dynamic and static workers race each other. It doesn't make a lot
> of sense to me to use the same ids for either?
>
> The attached patch reproduces the problem on master.
>
> Note that without the sleep(3) in the test t
Hi,
The new test fails with my AIO branch occasionally. But I'm fairly certain
that's just due to timing differences.
Excerpt from the log:
2023-07-27 21:43:00.385 UTC [42339] LOG: worker_spi worker 3 initialized with
schema3.counted
2023-07-27 21:43:00.399 UTC [42344] 001_worker_spi.pl LOG:
On Wed, Jul 26, 2023 at 09:02:54AM +0900, Michael Paquier wrote:
> I've been sleeping on that a bit more, and I'd still go with the
> refactoring where we initialize one cluster and have all the tests
> done by TAP, for the sake of being much cheaper without changing the
> coverage, while being mor
On Mon, Jul 24, 2023 at 05:38:45PM +0900, Michael Paquier wrote:
> Which is basically the same thing with TAP except that these are
> grouped now? The value of a few raw SQL queries with a
> NO_INSTALLCHECK does not strike me as enough on top of having to
> maintain two different sets of tests. I
On Mon, Jul 24, 2023 at 01:50:45PM +0530, Bharath Rupireddy wrote:
> I disagree with removing SQL tests from the worker_spi module. As said
> upthread, it makes the worker_spi a fully demonstrable
> extension/module - one can just take it, start adding required
> functionality and test-cases (both
On Mon, Jul 24, 2023 at 1:10 PM Michael Paquier wrote:
>
> On Mon, Jul 24, 2023 at 08:31:01AM +0530, Bharath Rupireddy wrote:
> > I also added a note atop worker_spi.c that the module also
> > demonstrates how to write core (SQL) tests and extended (TAP) tests.
>
> In terms of runtime the benefits
On Mon, Jul 24, 2023 at 08:31:01AM +0530, Bharath Rupireddy wrote:
> I also added a note atop worker_spi.c that the module also
> demonstrates how to write core (SQL) tests and extended (TAP) tests.
The value of the SQL tests comes down to the DO blocks that emulate
what the TAP tests could equall
On 2023-07-24 12:01, Bharath Rupireddy wrote:
I'm attaching the v3 patch.
I verified it works and it looks good to me.
Thanks to your work, I will be able to implement tests for
custom wait events.
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
On Mon, Jul 24, 2023 at 6:34 AM Masahiro Ikeda wrote:
>
> OK. If so, we need to remove the following comment in Makefile.
>
> > # enable our module in shared_preload_libraries for dynamic bgworkers
Done.
> I also confirmed that the tap tests work with meson and make.
Thanks for verifying.
I al
On 2023-07-22 01:05, Bharath Rupireddy wrote:
On Fri, Jul 21, 2023 at 4:05 PM Masahiro Ikeda
wrote:
(2)
Do we need "worker_spi.total_workers = 0" and
"shared_preload_libraries = worker_spi" in dynamic.conf.
Currently, the static bg workers will not be launched because
"shared_preload_librarie
On Fri, Jul 21, 2023 at 4:05 PM Masahiro Ikeda wrote:
>
> > In SQL tests, I ensured worker_spi doesn't start static bg workers by
> > setting worker_spi.total_workers = 0. Again, all of this is not
> > necessary, but it will be a very good example for someone writing
> > extensions and play around
Hi,
On 2023-07-20 18:39, Bharath Rupireddy wrote:
On Thu, Jul 20, 2023 at 2:59 PM Michael Paquier
wrote:
On Thu, Jul 20, 2023 at 05:54:55PM +0900, Masahiro Ikeda wrote:
> Yes, you're right. When I tried using worker_spi to test wait event,
> I found the behavior. And thanks a lot for your pat
On Fri, Jul 21, 2023 at 11:54 AM Michael Paquier wrote:
>
> On Fri, Jul 21, 2023 at 11:24:08AM +0530, Bharath Rupireddy wrote:
> > Okay. Here's a quick patch for adding TAP tests to the worker_spi
> > module. We can change it to taste.
>
> What do you think if we removed completely the sql/ test,
On Fri, Jul 21, 2023 at 11:24:08AM +0530, Bharath Rupireddy wrote:
> Okay. Here's a quick patch for adding TAP tests to the worker_spi
> module. We can change it to taste.
What do you think if we removed completely the sql/ test, moving it to
TAP so as we have only one cluster set up when running
On Fri, Jul 21, 2023 at 8:38 AM Michael Paquier wrote:
>
> On Thu, Jul 20, 2023 at 03:44:12PM +0530, Bharath Rupireddy wrote:
> > I don't think that change is correct. The worker_spi essentially shows
> > how to start bg workers with RegisterBackgroundWorker and dynamic bg
> > workers with Registe
On Thu, Jul 20, 2023 at 03:44:12PM +0530, Bharath Rupireddy wrote:
> I don't think that change is correct. The worker_spi essentially shows
> how to start bg workers with RegisterBackgroundWorker and dynamic bg
> workers with RegisterDynamicBackgroundWorker. If
> shared_preload_libraries = worker_s
On Thu, Jul 20, 2023 at 2:38 PM Masahiro Ikeda wrote:
>
> Thanks for discussing about the patch. I updated the patch from your
> comments
> * v2-0001-Support-worker_spi-to-execute-the-function-dynamical.patch
>
> I found another thing to be changed better. Though the tests was assumed
> "shared_pr
On Thu, Jul 20, 2023 at 2:59 PM Michael Paquier wrote:
>
> On Thu, Jul 20, 2023 at 05:54:55PM +0900, Masahiro Ikeda wrote:
> > Yes, you're right. When I tried using worker_spi to test wait event,
> > I found the behavior. And thanks a lot for your patch. I wasn't aware
> > of the way. I'll merge
On Thu, Jul 20, 2023 at 05:54:55PM +0900, Masahiro Ikeda wrote:
> Yes, you're right. When I tried using worker_spi to test wait event,
> I found the behavior. And thanks a lot for your patch. I wasn't aware
> of the way. I'll merge your patch to the tests for wait events.
Be careful when using th
of the test.
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATIONFrom 02ef0d8daddd43305842987a6aeac6732b2415a9 Mon Sep 17 00:00:00 2001
From: Masahiro Ikeda
Date: Thu, 20 Jul 2023 10:34:50 +0900
Subject: [PATCH] Support worker_spi to execute the function dynamically.
Currently, the database name
On 2023-07-20 12:55, Michael Paquier wrote:
On Thu, Jul 20, 2023 at 11:15:51AM +0900, Masahiro Ikeda wrote:
While I'm working on the thread[1], I found that the function of
worker_spi module fails if 'shared_preload_libraries' doesn't have
worker_spi.
I guess that you were patching worker_spi
On Thu, Jul 20, 2023 at 10:09 AM Michael Paquier wrote:
>
> On Thu, Jul 20, 2023 at 09:43:37AM +0530, Bharath Rupireddy wrote:
> > +1. However, a comment above helps one to understand why some GUCs are
> > defined before if (!process_shared_preload_libraries_in_progress). As
> > this is an example
On Thu, Jul 20, 2023 at 09:43:37AM +0530, Bharath Rupireddy wrote:
> +1. However, a comment above helps one to understand why some GUCs are
> defined before if (!process_shared_preload_libraries_in_progress). As
> this is an example extension, it will help understand the reasoning
> better. I know
On Thu, Jul 20, 2023 at 9:25 AM Michael Paquier wrote:
>
> > In my understanding, the restriction is not required. So, I think it's
> > better to change the behavior.
> > (v1-0001-Support-worker_spi-to-execute-the-function-dynamical.patch)
> >
> > What do you think?
>
> +1. I'm OK to lift this re
On Thu, Jul 20, 2023 at 11:15:51AM +0900, Masahiro Ikeda wrote:
> While I'm working on the thread[1], I found that the function of
> worker_spi module fails if 'shared_preload_libraries' doesn't have
> worker_spi.
I guess that you were patching worker_spi to register dynamically a
wait event and e
r extensions
https://www.postgresql.org/message-id/flat/b9f5411acda0cf15c8fbb767702ff43e%40oss.nttdata.com
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATIONFrom c6e60c66c4066b4a01981ffae5a168901e7283eb Mon Sep 17 00:00:00 2001
From: Masahiro Ikeda
Date: Thu, 20 Jul 2023 10:34:50 +0900
Subject: [PATCH]
34 matches
Mail list logo
