Re: Make default subscription streaming option as Parallel

2024-10-27 Thread Amit Kapila
On Tue, Oct 22, 2024 at 9:26 PM vignesh C wrote: > > The attached v5 version has the change to create subscriptions in > streaming off mode. I also did not find any other TAP test which > required further changes. > Pushed. -- With Regards, Amit Kapila.

Re: Make default subscription streaming option as Parallel

2024-10-23 Thread Amit Kapila
On Wed, Oct 23, 2024 at 1:21 AM Masahiko Sawada wrote: > > On Tue, Oct 22, 2024 at 2:17 AM Amit Kapila wrote: > > > > On Mon, Oct 21, 2024 at 8:40 PM vignesh C wrote: > > > > > > Attached v3 version patch has a caution added for the same. > > > > > > > Thanks, the patch looks good to me and I am

Re: Make default subscription streaming option as Parallel

2024-10-22 Thread Amit Kapila
On Wed, Oct 23, 2024 at 5:58 AM Peter Smith wrote: > > Hi Vignesh, here are some review comments for the patch v5-0001 (docs only). > > == > Commit message > > 1. > The commit message only refers to this as the "streaming option", but > an option of what? Somewhere we should mention this is an

Re: Make default subscription streaming option as Parallel

2024-10-22 Thread Peter Smith
Hi Vignesh, here are some review comments for the patch v5-0001 (docs only). == Commit message 1. The commit message only refers to this as the "streaming option", but an option of what? Somewhere we should mention this is an option of CREATE SUBSCRIPTION. ~ 2. Previously the default value

Re: Make default subscription streaming option as Parallel

2024-10-22 Thread Masahiko Sawada
On Tue, Oct 22, 2024 at 2:17 AM Amit Kapila wrote: > > On Mon, Oct 21, 2024 at 8:40 PM vignesh C wrote: > > > > Attached v3 version patch has a caution added for the same. > > > > Thanks, the patch looks good to me and I am planning to commit this > early next week unless there are objections or

Re: Make default subscription streaming option as Parallel

2024-10-22 Thread vignesh C
On Tue, 22 Oct 2024 at 16:24, Hayato Kuroda (Fujitsu) wrote: > > Dear Amit, Vignesh, > > > 1. Please ensure that none of the existing tests that use > > subscriptions with large changes will be impacted due to this change. > > I found at least 022_twophase_cascade.pl should be fixed. > The file ha

RE: Make default subscription streaming option as Parallel

2024-10-22 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Vignesh, > 1. Please ensure that none of the existing tests that use > subscriptions with large changes will be impacted due to this change. I found at least 022_twophase_cascade.pl should be fixed. The file has a part which tests non-streaming case: ``` # --- # 2P

Re: Make default subscription streaming option as Parallel

2024-10-22 Thread Amit Kapila
On Mon, Oct 21, 2024 at 8:40 PM vignesh C wrote: > > Attached v3 version patch has a caution added for the same. > Thanks, the patch looks good to me and I am planning to commit this early next week unless there are objections or any major problems. I have slightly updated the docs and commit mes

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread vignesh C
On Mon, 21 Oct 2024 at 14:36, Amit Kapila wrote: > > On Mon, Oct 7, 2024 at 11:05 AM vignesh C wrote: > > > > The tests demonstrate a significant performance improvement when using > > the parallel streaming option, insert shows 40-48 %improvement, delete > > shows 34-39 %improvement, update show

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread Peter Smith
On Mon, Oct 21, 2024 at 5:09 PM Amit Kapila wrote: > > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > > > On Mon, 7 Oct 2024 at 12:26, Hayato Kuroda (Fujitsu) > > wrote: > > > > > One comment for your patch; > > > Shouldn't we add a streaming=off case for pg_dump test? You added lines > >

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread Amit Kapila
On Mon, Oct 7, 2024 at 11:05 AM vignesh C wrote: > > The tests demonstrate a significant performance improvement when using > the parallel streaming option, insert shows 40-48 %improvement, delete > shows 34-39 %improvement, update shows 26-30 %improvement. In the case > of rollback the improvemen

Re: Make default subscription streaming option as Parallel

2024-10-20 Thread Amit Kapila
On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > On Mon, 7 Oct 2024 at 12:26, Hayato Kuroda (Fujitsu) > wrote: > > > One comment for your patch; > > Shouldn't we add a streaming=off case for pg_dump test? You added lines but > > no one > > passes it. > > > > Update existing pg_dump tests to c

Re: Make default subscription streaming option as Parallel

2024-10-19 Thread Dilip Kumar
On Fri, 18 Oct 2024 at 5:24 PM, Amit Kapila wrote: > On Fri, Oct 18, 2024 at 9:48 AM Dilip Kumar wrote: > > > > On Tue, Oct 8, 2024 at 3:38 PM Amit Kapila > wrote: > >> > >> On Tue, Oct 8, 2024 at 2:25 PM shveta malik > wrote: > >> > > >> > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > >>

Re: Make default subscription streaming option as Parallel

2024-10-18 Thread Amit Kapila
On Fri, Oct 18, 2024 at 9:48 AM Dilip Kumar wrote: > > On Tue, Oct 8, 2024 at 3:38 PM Amit Kapila wrote: >> >> On Tue, Oct 8, 2024 at 2:25 PM shveta malik wrote: >> > >> > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: >> > > >> > >> > With parallel streaming as default, do you think there is

Re: Make default subscription streaming option as Parallel

2024-10-17 Thread Dilip Kumar
On Tue, Oct 8, 2024 at 3:38 PM Amit Kapila wrote: > On Tue, Oct 8, 2024 at 2:25 PM shveta malik > wrote: > > > > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > > > > > > With parallel streaming as default, do you think there is a need to > > increase the default for 'max_logical_replicatio

Re: Make default subscription streaming option as Parallel

2024-10-08 Thread shveta malik
On Tue, Oct 8, 2024 at 3:38 PM Amit Kapila wrote: > > On Tue, Oct 8, 2024 at 2:25 PM shveta malik wrote: > > > > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > > > > > > With parallel streaming as default, do you think there is a need to > > increase the default for 'max_logical_replication

Re: Make default subscription streaming option as Parallel

2024-10-08 Thread vignesh C
On Mon, 7 Oct 2024 at 11:05, vignesh C wrote: > > Hi, > > By default, currently streaming of in-progress transactions for > subscriptions is disabled. All transactions are fully decoded on the > publisher before being sent to the subscriber. This approach can lead > to increased latency and reduce

Re: Make default subscription streaming option as Parallel

2024-10-08 Thread Amit Kapila
On Tue, Oct 8, 2024 at 2:25 PM shveta malik wrote: > > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > > > With parallel streaming as default, do you think there is a need to > increase the default for 'max_logical_replication_workers' as IIUC > parallel workers are taken from the same pool.

Re: Make default subscription streaming option as Parallel

2024-10-08 Thread shveta malik
On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > With parallel streaming as default, do you think there is a need to increase the default for 'max_logical_replication_workers' as IIUC parallel workers are taken from the same pool. thanks Shveta

Re: Make default subscription streaming option as Parallel

2024-10-07 Thread vignesh C
On Mon, 7 Oct 2024 at 12:26, Hayato Kuroda (Fujitsu) wrote: > > Dear Vignesh, > > > One key point to consider is that the lock on transaction objects will > > be held for a longer duration when using streaming in parallel. This > > occurs because the parallel apply worker initiates the transaction

RE: Make default subscription streaming option as Parallel

2024-10-06 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, > One key point to consider is that the lock on transaction objects will > be held for a longer duration when using streaming in parallel. This > occurs because the parallel apply worker initiates the transaction as > soon as streaming begins, maintaining the lock until the transacti