On Thu, Jun 12, 2025 at 4:22 PM shveta malik wrote:
>
> On Thu, Jun 12, 2025 at 11:34 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> >
> > Here is the V35 patch set which includes the following changes:
> >
>
> Thank You for the patches. Few comments:
>
> 1)
> Since now we create slot for rci enabled sub
On Thu, Jun 12, 2025 at 11:34 AM Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the V35 patch set which includes the following changes:
>
Thank You for the patches. Few comments:
1)
Since now we create slot for rci enabled subscription, it will require
wal_level >= replica even on subscribers. We sha
On Tue, Jun 10, 2025 at 11:55 AM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V35 patch set which includes the following changes:
>
Few minor comments:
===
1.
+Â * Check if the subscriber's configuration is adequate to enable the
+Â * retain_conflict_info option.
I see some funny
On Tue, Jun 10, 2025 at 11:55 AM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V35 patch set which includes the following changes:
>
Thank You for the patches, few comments:
1)
compute_min_nonremovable_xid:
+ /*
+ * Stop advancing xmin if an invalid non-removable transaction ID is
+ * found, othe
On Wed, Jun 4, 2025 at 4:12 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V33 patch set which includes the following changes:
>
Few comments:
1.
+ if (sub->enabled)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set option %s for enabled subscription",
+
On Wed, Jun 4, 2025 at 4:12 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V33 patch set which includes the following changes:
>
Thank You for the patches, please find few comments for patch003:
1)
+ /*
+ * Skip the track_commit_timestamp check by passing it as
+ * true, since it has already bee
On Wed, Jun 4, 2025 at 6:43 PM Zhijie Hou (Fujitsu) wrote:
>
> On Mon, Jun 2, 2025 at 2:39 PM Amit Kapila wrote:
> >
> > On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > Attaching the V32 patch set which addressed comments in [1]~[5].
> > >
> >
> > Review comments:
>
On Mon, Jun 2, 2025 at 12:09 PM Amit Kapila wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
> >
>
> Review comments:
> ===
> *
> +advance_conflict_slot_xmin(FullTransactionId new_xmin)
>
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V32 patch set which addressed comments in [1]~[5].
>
Review comments:
===
*
+advance_conflict_slot_xmin(FullTransactionId new_xmin)
+{
+ FullTransactionId full_xmin;
+ FullTransactionId next_full_xid;
+
+ A
On Tue, May 27, 2025 at 3:59 PM shveta malik wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
>
> Thanks for the patch, I am still reviewing the patches, please find
> few trivial comments for patch0
On Tue, May 27, 2025 at 11:45 AM Amit Kapila wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
> >
> > >
> > > I am thinking can't we make it more deterministic such that when we
> > > get the status first time if
On Tue, May 27, 2025 at 3:59 PM shveta malik wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
>
> Thanks for the patch, I am still reviewing the patches, please find
> few trivial comments for patch0
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V32 patch set which addressed comments in [1]~[5].
Thanks for the patch, I am still reviewing the patches, please find
few trivial comments for patch001:
1)
+ FullTransactionId last_phase_at; /* publisher transaction
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
>
> >
> > I am thinking can't we make it more deterministic such that when we
> > get the status first time if we find some transactions that are in
> > commit phase then we should j
On Fri, May 23, 2025 at 11:51 PM Xuneng Zhou wrote:
> Thanks for the effort on the patches. I did a quick look on them before
> diving into the logic and discussion. Below are a few minor typos found in
> version 31.
Thanks for the comments! I have fixed these typos in latest version.
Best Regar
On Fri, May 23, 2025 at 2:45 PM shveta malik wrote:
>
> Thanks you for v31 patch-set. Please find few comments on patch001:
Thanks for the comments.
>
> 1)
>
> wait_for_local_flush:
>
> + if (data->last_recv_time &&
> + TimestampDifferenceExceeds(data->flushpos_update_time,
> +data->last
On Sat, May 24, 2025 at 6:28 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 11:00 AM Amit Kapila
> wrote:
> >
> > On Sat, May 24, 2025 at 10:29 AM Dilip Kumar
> wrote:
> > >
> > > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar
> wrote:
> > > >
> > > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zh
On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 4:46 PM Amit Kapila
> wrote:
> >
> > This sounds reasonable to me. Let us see what others think.
> >
>
> I was looking into the for getting the transaction status from
> publisher, what I would assume this patch shou
On Sat, May 24, 2025 at 4:46 PM Amit Kapila wrote:
>
> This sounds reasonable to me. Let us see what others think.
>
I was looking into the for getting the transaction status from
publisher, what I would assume this patch should be doing is request
the publisher status first time, and if some tra
On Sat, May 24, 2025 at 3:58 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 11:00 AM Amit Kapila wrote:
> >
> > On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
> > >
> > > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar
> > > wrote:
> > > >
> > > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zh
On Sat, May 24, 2025 at 11:00 AM Amit Kapila wrote:
>
> On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
> >
> > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
> > >
> > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> > > >
> > > Looking at v31-0001 again, most of it looks fine e
On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
> >
> > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> > >
> > Looking at v31-0001 again, most of it looks fine except this logic of
> > getting the commit_ts after marking the trans
On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
>
> On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> >
> Looking at v31-0001 again, most of it looks fine except this logic of
> getting the commit_ts after marking the transaction in commit. I see
> in RecordTransactionCommit(), we are set
On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
>
Looking at v31-0001 again, most of it looks fine except this logic of
getting the commit_ts after marking the transaction in commit. I see
in RecordTransactionCommit(), we are setting this flag
(DELAY_CHKPT_IN_COMMIT) to put the transaction in
Hi Zhijie,
Thanks for the effort on the patches. I did a quick look on them before
diving into the logic and discussion. Below are a few minor typos found in
version 31.
⸻
1. Spelling of “non-removable”
[PATCH v31 1/7]
In docs and code “removeable” vs. “removable” are used alternatively
On Thu, May 22, 2025 at 8:28 AM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V31 patch set which addressed comments in [1]~[8].
>
Few comments:
1.
+ The oldest transaction ID along that is currently in the commit
+ phase on the server, along with its epoch.
The first 'a
On Fri, May 23, 2025 at 12:15 PM shveta malik wrote:
>
> Thanks you for v31 patch-set. Please find few comments on patch001:
>
> 1)
>
> wait_for_local_flush:
>
> + if (data->last_recv_time &&
> + TimestampDifferenceExceeds(data->flushpos_update_time,
> +data->last_recv_time, WalWriterDelay))
>
Thanks you for v31 patch-set. Please find few comments on patch001:
1)
wait_for_local_flush:
+ if (data->last_recv_time &&
+ TimestampDifferenceExceeds(data->flushpos_update_time,
+data->last_recv_time, WalWriterDelay))
+ {
+ XLogRecPtr writepos;
+ XLogRecPtr flushpos;
+ bool have_pending_tx
On Fri, May 16, 2025 at 7:31 PM Amit Kapila wrote:
>
> A few more comments:
> =
> 3.
> maybe_advance_nonremovable_xid(RetainConflictInfoData *data,
>bool status_received)
> {
> /* Exit early if retaining conflict information is not required */
> if (!MySubscription->retainconfl
On Tue, May 20, 2025 at 6:30 PM shveta malik wrote:
>
> Few more comments mostly for patch001:
Thanks for the comments!
>
> 4)
> For this feature, since we are only interested in remote UPDATEs happening
> concurrently, so shall we ask primary to provide oldest "UPDATE"
> transaction-id in comm
On Tue, May 20, 2025 at 11:08 AM shveta malik wrote:
>
> Please find few more comments:
Thanks for the comments!
>
> 2)
>
> --
> send_feedback(last_received, requestReply, requestReply);
>
> + maybe_advance_nonremovable_xid(&data, false);
> +
> /*
> * Force reporting to ensure l
On Fri, May 16, 2025 at 5:01 PM Amit Kapila wrote:
>
Please find some more comments on the 0001 patch:
1.
We need to know about such transactions
+ * for conflict detection and resolution in logical replication. See
+ * GetOldestTransactionIdInCommit and its use.
Do we need to mention resolution
On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
>
> Please find few more comments:
>
> 1)
> ProcessStandbyPSRequestMessage:
> + /*
> + * This shouldn't happen because we don't support getting primary status
> + * message from standby.
> + */
> + if (RecoveryInProgress())
> + elog(ERROR, "the p
On Tue, May 20, 2025 at 10:24 AM Amit Kapila wrote:
>
> On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
> >
> > Please find few more comments:
> >
> > 1)
> > ProcessStandbyPSRequestMessage:
> > + /*
> > + * This shouldn't happen because we don't support getting primary status
> > + * message
On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
>
> Please find few more comments:
>
> 1)
> ProcessStandbyPSRequestMessage:
> + /*
> + * This shouldn't happen because we don't support getting primary status
> + * message from standby.
> + */
> + if (RecoveryInProgress())
> + elog(ERROR, "the p
Please find few more comments:
1)
ProcessStandbyPSRequestMessage:
+ /*
+ * This shouldn't happen because we don't support getting primary status
+ * message from standby.
+ */
+ if (RecoveryInProgress())
+ elog(ERROR, "the primary status is unavailable during recovery");
a) This error is not cle
On Thu, May 15, 2025 at 6:02 PM Amit Kapila wrote:
>
> Few minor comments on 0001:
> 1.
> + if (TimestampDifferenceExceeds(data->reply_time,
> +data->candidate_xid_time, 0))
> + ereport(ERROR,
> + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("oldest_nonremovable_xid transactio
On Fri, May 16, 2025 at 2:40 PM Amit Kapila wrote:
>
> On Fri, May 16, 2025 at 12:01 PM shveta malik wrote:
> >
> > On Fri, May 16, 2025 at 11:15 AM Amit Kapila
> > wrote:
> > >
> > >
> > > BTW, another related point is that when we decide to stop retaining
> > > dead tuples (via should_stop_co
On Fri, May 16, 2025 at 12:17 PM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 10:08 AM shveta malik wrote:
> >
> > On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
> > wrote:
> >
> > > > Few comments for patch004:
> > > > Config.sgml:
> > > > 1)
> > > > +
> > > > +Maximum du
On Fri, May 16, 2025 at 12:01 PM shveta malik wrote:
>
> On Fri, May 16, 2025 at 11:15 AM Amit Kapila wrote:
> >
> >
> > BTW, another related point is that when we decide to stop retaining
> > dead tuples (via should_stop_conflict_info_retention), should we also
> > consider the case that the app
On Fri, Apr 25, 2025 at 10:08 AM shveta malik wrote:
>
> On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
> wrote:
>
> > > Few comments for patch004:
> > > Config.sgml:
> > > 1)
> > > +
> > > +Maximum duration (in milliseconds) for which conflict
> > > +information can
On Fri, May 16, 2025 at 11:15 AM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
> >
> > 2)
> > in wait_for_local_flush(), we have
> > should_stop_conflict_info_retention() before 'AllTablesyncsReady'
> > check. Should we give a discount for table-sync time and avoid do
On Thu, May 15, 2025 at 6:02 PM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
> >
> > >
> > > Here is V30 patch set includes the following changes:
> > >
> >
> > Thank You for the patch, please find few concerns:
> >
> > 1)
> > By looking at code of ApplyLauncherMain(
On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
>
> 2)
> in wait_for_local_flush(), we have
> should_stop_conflict_info_retention() before 'AllTablesyncsReady'
> check. Should we give a discount for table-sync time and avoid doing
> stop-conflict-retention when table-sync is going on? This is
On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
>
> >
> > Here is V30 patch set includes the following changes:
> >
>
> Thank You for the patch, please find few concerns:
>
> 1)
> By looking at code of ApplyLauncherMain(), it appears that we stopped
> advancing shared-slot's xmin if any of the
On Fri, Apr 25, 2025 at 4:05 PM shveta malik wrote:
>
> >
> > Here is V30 patch set includes the following changes:
> >
>
> Thank You for the patch, please find few concerns:
>
Please find few more concerns:
3)
In get_candidate_xid(), we first set candidate_xid_time and later
candidate_xid. And
>
> Here is V30 patch set includes the following changes:
>
Thank You for the patch, please find few concerns:
1)
By looking at code of ApplyLauncherMain(), it appears that we stopped
advancing shared-slot's xmin if any of the subscriptions with
retain_conflict_info is disabled. If a subscription
On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
wrote:
> > Few comments for patch004:
> > Config.sgml:
> > 1)
> > +
> > +Maximum duration (in milliseconds) for which conflict
> > +information can be retained for conflict detection by the apply
> > worker.
> > +
On Wed, Apr 16, 2025 at 10:30 AM shveta malik wrote:
>
> On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Here's a rebased version of the patch series.
> >
>
Few comments for patch004:
Config.sgml:
1)
+
+Maximum duration (in milliseconds) for which conflict
+
On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here's a rebased version of the patch series.
>
Thanks for the patches.
While testing the GUC "max_conflict_retention_duration", I noticed a
behavior that seems to bypass its intended purpose.
On Pub, if a txn is stuck in the COMMI
On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here's a rebased version of the patch series.
>
Thanks Hou-San for the patches. I am going through this long thread
and patches. One doubt I have is whenever there is a chance of
conflict-slot update (either xmin or possibility of it
Dear Hou,
Thanks for updating the patch! I can finally come back to the thread.
Regarding the max_retain_conflict_duration, I prefer GUC approach because it has
already had a mechanism for converting unit: subscription option does not have
it.
Below part contains my comments:
01. check_new_clu
On Thu, 20 Feb 2025 at 12:50, Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the v28 patch set, which converts the subscription option
> max_conflict_retention_duration into a GUC. Other logic remains unchanged.
After discussing with Hou internally, I have moved this to the next
CommitFest since it wi
On Mon, Feb 10, 2025 at 2:45 PM Amit Kapila wrote:
>
> On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote:
> >
> > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
> >
> > >
> > > > I'm not really sure that these behaviors are the expected behavior of
> > > > users who set max_conflict_retenti
On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote:
>
> On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
>
> >
> > > I'm not really sure that these behaviors are the expected behavior of
> > > users who set max_conflict_retention_duration to some subscriptions.
> > > Or I might have set the wro
On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
>
> On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
> >
> > I'd like to confirm what users would expect of this
> > max_conflict_retention_duration option and it works as expected. IIUC
> > users would want to use this option when they want
On Thu, Feb 6, 2025 at 9:47 PM Amit Kapila wrote:
>
> On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
> >
> > I'd like to confirm what users would expect of this
> > max_conflict_retention_duration option and it works as expected. IIUC
> > users would want to use this option when they want
On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
>
> I'd like to confirm what users would expect of this
> max_conflict_retention_duration option and it works as expected. IIUC
> users would want to use this option when they want to balance between
> the reliable update_deleted conflict detec
On Tue, Feb 4, 2025 at 10:30 PM Amit Kapila wrote:
>
> On Wed, Feb 5, 2025 at 6:00 AM Masahiko Sawada wrote:
> >
> > On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
> > >
> > > >
> > > > I was not sure of the point of
> > > > making the max_conflict_retention_duration a per-subscription
> > >
On Thu, Jan 23, 2025 at 5:17 PM Zhijie Hou (Fujitsu)
wrote:
>
I was reviewing v26 patch set and have some comments so far I reviewed
0001 so most of the comments/question are from this patch.
comments on v26-0001
1.
+ next_full_xid = ReadNextFullTransactionId();
+ epoch = EpochFromFullTransactio
On Wed, Feb 5, 2025 at 6:00 AM Masahiko Sawada wrote:
>
> On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
> >
> > >
> > > I was not sure of the point of
> > > making the max_conflict_retention_duration a per-subscription
> > > parameter.
> > >
> >
> > The idea is to keep it at the same level a
On Sat, Feb 1, 2025 at 10:37 AM Amit Kapila wrote:
>
> On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
> >
> > On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila
> > wrote:
> > >
> > > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > > wrote:
> > > >
> > > > I have one question about the 0
On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
>
> On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
> >
> > On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila
> > wrote:
> > >
> > > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > > wrote:
> > > >
> > > > I have one question about the 0
On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
>
> On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila wrote:
> >
> > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > wrote:
> > >
> > > I have one question about the 0004 patch; it implemented
> > > max_conflict_retntion_duration as a subscri
On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila wrote:
>
> On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada wrote:
> >
> > I have one question about the 0004 patch; it implemented
> > max_conflict_retntion_duration as a subscription parameter. But the
> > launcher invalidates the pg_conflict_detectio
On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada wrote:
>
> I have one question about the 0004 patch; it implemented
> max_conflict_retntion_duration as a subscription parameter. But the
> launcher invalidates the pg_conflict_detection slot only if all
> subscriptions with retain_conflict_info stop
On Thu, Jan 23, 2025 at 3:47 AM Zhijie Hou (Fujitsu)
wrote:
>
> On Wednesday, January 22, 2025 7:54 PM Zhijie Hou (Fujitsu)
> wrote:
> > On Saturday, January 18, 2025 11:45 AM Zhijie Hou (Fujitsu)
> > wrote:
> > > I think invalidating the slot is OK and we could also let the apply
> > > worker
On Sat, Jan 18, 2025 at 9:15 AM Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the V24 patch set. I modified 0004 patch to implement the slot
> Invalidation part. Since the automatic recovery could be an optimization and
> the discussion is in progress, I didn't implement that part.
Few comments for p
On Wed, Jan 8, 2025 at 4:03 PM Masahiko Sawada wrote:
>
> On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila wrote:
> > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> > wrote:
> > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> > > wrote:
> > > > Here is further performance test analysis with v16
Dear hackers,
I've created a new script which simulates that user reduce the workload on the
publisher side. Attached zip file contains a script, execution log and pgbench
outputs. Experiments were done with v24 patch set.
Abstract
==
In this test the conflict slot could be invalidated as e
On Fri, Jan 17, 2025 at 1:37 PM Masahiko Sawada wrote:
>
> On Thu, Jan 16, 2025 at 2:02 AM Amit Kapila wrote:
> >
> > On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > In the latest version, we implemented a simpler approach that allows the
> > > apply
> > > worker to
On Thu, Jan 16, 2025 at 2:02 AM Amit Kapila wrote:
>
> On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > In the latest version, we implemented a simpler approach that allows the
> > apply
> > worker to directly advance the oldest_nonremovable_xid if the waiting time
> > excee
On Wed, Jan 15, 2025 at 9:38 AM Amit Kapila wrote:
>
> On Wed, Jan 15, 2025 at 5:57 AM Masahiko Sawada wrote:
> >
> > Probably retaining dead tuples based on the time duration or its age
> > might be other solutions, it would increase a risk of not being able
> > to detect update_deleted conflict
On Thu, Jan 16, 2025 at 4:02 PM Amit Kapila wrote:
> On Thu, Jan 16, 2025 at 3:45 PM Dilip Kumar wrote:
> >
> > On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila
> wrote:
> >>
> >> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
> >> >
> >> > Conflict detection of truncated updates is detected as upd
On Thu, Jan 16, 2025 at 3:45 PM Dilip Kumar wrote:
>
> On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila wrote:
>>
>> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
>> >
>> > Conflict detection of truncated updates is detected as update_missing
>> > and deleted update is detected as update_deleted. I
On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila wrote:
> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
> >
> > Conflict detection of truncated updates is detected as update_missing
> > and deleted update is detected as update_deleted. I was not sure if
> > truncated updates should also be detected
On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
wrote:
>
> In the latest version, we implemented a simpler approach that allows the apply
> worker to directly advance the oldest_nonremovable_xid if the waiting time
> exceeds the newly introduced option's limit. I've named this option
> "max_c
On Wed, Jan 15, 2025 at 5:57 AM Masahiko Sawada wrote:
>
> On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote:
> >
> > As of now, I can't think of a way to throttle the publisher when the
> > apply_worker lags. Basically, we need some way to throttle (reduce the
> > speed of backends) when the app
On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote:
>
> On Tue, Jan 14, 2025 at 7:14 AM Masahiko Sawada wrote:
> >
> > On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila
> > wrote:
> > >
> > > I don't think we can avoid accumulating garbage especially when the
> > > workload on the publisher is more.
On Tue, Jan 14, 2025 at 7:14 AM Masahiko Sawada wrote:
>
> On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila wrote:
> >
> > I don't think we can avoid accumulating garbage especially when the
> > workload on the publisher is more. Consider the current case being
> > discussed, on the publisher, we hav
On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila wrote:
>
> On Fri, Jan 10, 2025 at 6:13 AM Masahiko Sawada wrote:
> >
> > 3. If the apply worker cannot catch up, it could enter to a bad loop;
> > the publisher sends huge amount of data -> the apply worker cannot
> > catch up -> it needs to wait for
Here are the performance test results and analysis with the recent patches.
Test Setup:
- Created Pub and Sub nodes with logical replication and below configurations.
autovacuum_naptime = '30s'
shared_buffers = '30GB'
max_wal_size = 20GB
min_wal_size = 10GB
track_commit_timestamp =
On Fri, Jan 10, 2025 at 6:13 AM Masahiko Sawada wrote:
>
> 3. If the apply worker cannot catch up, it could enter to a bad loop;
> the publisher sends huge amount of data -> the apply worker cannot
> catch up -> it needs to wait for a longer time to advance its
> oldest_nonremovable_xid -> more ga
On Friday, January 10, 2025 8:43 AM Masahiko Sawada
wrote:
Hi,
>
> On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > >
> > > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
>
> > > wrote
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote:
>
> On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote:
> > [3] Test with pgbench run on both publisher and subscriber.
> >
> > Test setup:
> > - Tests performed on pgHead + v16 patches
> > - Created a pub-sub replication system.
> > - Paramet
On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu)
wrote:
>
> On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
> wrote:
>
> Hi,
>
> >
> > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> > wrote:
> > >
> > >
On Wednesday, January 8, 2025 3:49 PM Nisha Moond
wrote:
>
> On Tue, Jan 7, 2025 at 6:04 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> >
> > Attached the V19 patch which addressed comments in [1][2][3][4][5][6][7].
> >
>
> Here are a couple of initial review comments on v19 patch set:
>
> 1) The sub
On Wednesday, January 8, 2025 7:03 PM vignesh C wrote:
Hi,
> Consider a LR setup with retain_conflict_info=true for a table t1:
> Publisher:
> insert into t1 values(1);
> -- Have a open transaction before delete operation in subscriber begin;
>
> Subscriber:
> -- delete the record that was repl
On Wed, Jan 8, 2025 at 2:24 PM Amit Kapila wrote:
>
> On Wed, Jan 8, 2025 at 2:15 PM Masahiko Sawada wrote:
> >
> > On Tue, Jan 7, 2025 at 2:49 AM Amit Kapila wrote:
> > > >
> > > > We thought of another approach, which is to create/drop this slot first
> > > > as
> > > > soon as one enables/di
On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
Hi,
>
> On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > > wrote:
> > > > On Wed, Jan 8
On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
wrote:
Hi,
>
> On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > > wrote:
> > > > On We
On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
wrote:
>
> On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
>
> Hi,
>
> > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > wrote:
> > > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> > wrote:
> > > >
> > > > On Thu, Dec 19, 2024
On Tue, 7 Jan 2025 at 18:04, Zhijie Hou (Fujitsu)
wrote:
>
> On Friday, January 3, 2025 1:53 PM Amit Kapila
> wrote:
> >
> > On Wed, Dec 25, 2024 at 8:13 AM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > Attach the new version patch set which addressed all other comments.
> > >
> >
> > Some more m
On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
wrote:
Hi,
> On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> wrote:
> > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> wrote:
> > >
> > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> wrote:
> > > >
> > > >
> > > > [3] Test with pgbench r
On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila wrote:
>
> On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote:
> >
> > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> > wrote:
> > >
> > > Here is further performance test analysis with v16 patch-set.
> > >
> > >
> > > In the test scenarios already s
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote:
>
> On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote:
> >
> > Here is further performance test analysis with v16 patch-set.
> >
> >
> > In the test scenarios already shared on -hackers [1], where pgbench was run
> > only on the publisher no
On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote:
>
> Here is further performance test analysis with v16 patch-set.
>
>
> In the test scenarios already shared on -hackers [1], where pgbench was run
> only on the publisher node in a pub-sub setup, no performance degradation was
> observed on ei
On Wed, Jan 8, 2025 at 2:15 PM Masahiko Sawada wrote:
>
> On Tue, Jan 7, 2025 at 2:49 AM Amit Kapila wrote:
> > >
> > > We thought of another approach, which is to create/drop this slot first as
> > > soon as one enables/disables detect_update_deleted (E.g. create/drop slot
> > > during DDL). But
On Tue, Jan 7, 2025 at 2:49 AM Amit Kapila wrote:
>
> On Mon, Jan 6, 2025 at 4:52 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Friday, January 3, 2025 2:36 PM Masahiko Sawada
> > wrote:
> >
> > >
> > > + /*
> > > +* The changes made by this and later transactions are still
> > > no
On Tue, Jan 7, 2025 at 6:04 PM Zhijie Hou (Fujitsu)
wrote:
>
>
> Attached the V19 patch which addressed comments in [1][2][3][4][5][6][7].
>
Here are a couple of initial review comments on v19 patch set:
1) The subscription option 'retain_conflict_info' remains set to
"true" for a subscription e
1 - 100 of 195 matches
Mail list logo
