Re: Logical replication prefetch

2025-07-15 Thread Amit Kapila
On Mon, Jul 14, 2025 at 3:13 PM Amit Kapila wrote: > > On Sun, Jul 13, 2025 at 6:06 PM Konstantin Knizhnik > wrote: > > > > On 13/07/2025 1:28 pm, Amit Kapila wrote: > > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > > wrote: > > >> There is well known Postgres problem that logical

RE: Logical replication prefetch

2025-07-14 Thread Zhijie Hou (Fujitsu)
On Monday, July 14, 2025 2:36 PM Konstantin Knizhnik wrote: > On 14/07/2025 4:20 am, Zhijie Hou (Fujitsu) wrote: > > Additionally, I was also exploring ways to improve performance and have > > tried an > > alternative version of prefetch for experimentation. The alternative design > > is > > tha

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
On Sun, Jul 13, 2025 at 5:59 PM Konstantin Knizhnik wrote: > > > Certainly originally intended use case was different: parallel apply is > performed only for large transactions. Number of of such transactions is > not so big and > so there should be enough parallel apply workers in pool to proceed

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
On Sun, Jul 13, 2025 at 6:06 PM Konstantin Knizhnik wrote: > > On 13/07/2025 1:28 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > wrote: > >> There is well known Postgres problem that logical replication subscriber > >> can not caught-up with publisher just bec

Re: Logical replication prefetch

2025-07-13 Thread Konstantin Knizhnik
On 14/07/2025 4:20 am, Zhijie Hou (Fujitsu) wrote: Thank you for the proposal ! I find it to be a very interesting feature。 I tested the patch you shared in your original email and encountered potential deadlocks when testing pgbench TPC-B like workload. Could you please provide an updated patc

RE: Logical replication prefetch

2025-07-13 Thread Zhijie Hou (Fujitsu)
On Tuesday, July 8, 2025 2:36 PM Konstantin Knizhnik wrote: > > There is well known Postgres problem that logical replication subscriber can > not caught-up with publisher just because LR changes are applied by single > worker and at publisher changes are made by multiple concurrent backends. > T

Re: Logical replication prefetch

2025-07-13 Thread Konstantin Knizhnik
On 13/07/2025 1:28 pm, Amit Kapila wrote: On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: There is well known Postgres problem that logical replication subscriber can not caught-up with publisher just because LR changes are applied by single worker and at publisher changes are made

Re: Logical replication prefetch

2025-07-13 Thread Konstantin Knizhnik
On 13/07/2025 9:28 am, Amit Kapila wrote: I didn't understand your scenario. pa_launch_parallel_worker() should spawn a new worker only if all the workers in the pool are busy, and then it will free the worker if the pool already has enough workers. So, do you mean to say that the workers in th

Re: Logical replication prefetch

2025-07-13 Thread Amit Kapila
On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: > > There is well known Postgres problem that logical replication subscriber > can not caught-up with publisher just because LR changes are applied by > single worker and at publisher changes are made by > multiple concurrent backends. >

Re: Logical replication prefetch

2025-07-12 Thread Amit Kapila
On Fri, Jul 11, 2025 at 7:49 PM Konstantin Knizhnik wrote: > > On 08/07/2025 2:51 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > wrote: > >> There is well known Postgres problem that logical replication subscriber > >> can not caught-up with publisher just bec

Re: Logical replication prefetch

2025-07-11 Thread Konstantin Knizhnik
On 08/07/2025 2:51 pm, Amit Kapila wrote: On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: There is well known Postgres problem that logical replication subscriber can not caught-up with publisher just because LR changes are applied by single worker and at publisher changes are made

Re: Logical replication prefetch

2025-07-11 Thread Konstantin Knizhnik
On 11/07/2025 11:52 am, Amit Kapila wrote: On Wed, Jul 9, 2025 at 12:08 AM Konstantin Knizhnik wrote: On 08/07/2025 2:51 pm, Amit Kapila wrote: On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: It is possible to enforce parallel apply of short transactions using `debug_logical_rep

Re: Logical replication prefetch

2025-07-11 Thread Amit Kapila
On Wed, Jul 9, 2025 at 12:08 AM Konstantin Knizhnik wrote: > > On 08/07/2025 2:51 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > wrote: > > It is possible to enforce parallel apply of short > transactions using `debug_logical_replication_streaming` but then > pe

Re: Logical replication prefetch

2025-07-08 Thread Konstantin Knizhnik
On 08/07/2025 2:51 pm, Amit Kapila wrote: On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: It is possible to enforce parallel apply of short transactions using `debug_logical_replication_streaming` but then performance is ~2x times slower than in case of sequential apply by single wor

Re: Logical replication prefetch

2025-07-08 Thread Amit Kapila
On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik wrote: > > There is well known Postgres problem that logical replication subscriber > can not caught-up with publisher just because LR changes are applied by > single worker and at publisher changes are made by > multiple concurrent backends. The

Logical replication prefetch

2025-07-07 Thread Konstantin Knizhnik
There is well known Postgres problem that logical replication subscriber can not caught-up with publisher just because LR changes are applied by single worker and at publisher changes are made by multiple concurrent backends. The problem is not logical replication specific: physical replication