On Thu, Jun 19, 2025 at 4:34 PM shveta malik wrote:
>
> >
> > Here is the V38 patch set which includes the following changes:
> >
>
> Thank You for the patches. Few comments:
>
> 1)
> +
> +Note that commit timestamps and origin data retained by enabling the
> + linkend="sql-createsubscr
>
> Here is the V38 patch set which includes the following changes:
>
Thank You for the patches. Few comments:
1)
+
+Note that commit timestamps and origin data retained by enabling the
+retain_conflict_info
+option will not be preserved during the upgrade. As a
+result, the up
On Tue, Jun 17, 2025 at 4:26 PM Zhijie Hou (Fujitsu)
wrote:
>
> On Mon, Jun 16, 2025 at 7:37 PM Amit Kapila wrote:
> >
> >
> > 3. Isn't the new check for logical slots in
> > check_new_cluster_subscription_configuration() somewhat redundant with
> > the previous check done in check_new_cluster_log
On Thu, Jun 12, 2025 at 11:34 AM Zhijie Hou (Fujitsu)
wrote:
>
Few comments on v36 patches:
==
1. In advance_conflict_slot_xmin(), we first save the slot to disk,
then update its effective xmin, and then do the required xmin
computation. Now, if we don't save the slot ever
On Mon, Jun 16, 2025 at 9:29 AM Zhijie Hou (Fujitsu)
wrote:
>
> 0001:
> * Removed the slot acquisition as suggested above.
>
> 0002:
> * Addressed the comments above.
>
Thanks for the patches.
In advance_conflict_slot_xmin(), if new_xmin is same as slot's current
xmin, then shall we simply retu
On Thu, Jun 12, 2025 at 4:22 PM shveta malik wrote:
>
> On Thu, Jun 12, 2025 at 11:34 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> >
> > Here is the V35 patch set which includes the following changes:
> >
>
> Thank You for the patches. Few comments:
>
> 1)
> Since now we create slot for rci enabled sub
On Thu, Jun 12, 2025 at 11:34 AM Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the V35 patch set which includes the following changes:
>
Thank You for the patches. Few comments:
1)
Since now we create slot for rci enabled subscription, it will require
wal_level >= replica even on subscribers. We sha
On Tue, Jun 10, 2025 at 11:55 AM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V35 patch set which includes the following changes:
>
Few minor comments:
===
1.
+Â * Check if the subscriber's configuration is adequate to enable the
+Â * retain_conflict_info option.
I see some funny
On Tue, Jun 10, 2025 at 11:55 AM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V35 patch set which includes the following changes:
>
Thank You for the patches, few comments:
1)
compute_min_nonremovable_xid:
+ /*
+ * Stop advancing xmin if an invalid non-removable transaction ID is
+ * found, othe
On Wed, Jun 4, 2025 at 4:12 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V33 patch set which includes the following changes:
>
Few comments:
1.
+ if (sub->enabled)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot set option %s for enabled subscription",
+
On Wed, Jun 4, 2025 at 4:12 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here is the V33 patch set which includes the following changes:
>
Thank You for the patches, please find few comments for patch003:
1)
+ /*
+ * Skip the track_commit_timestamp check by passing it as
+ * true, since it has already bee
On Wed, Jun 4, 2025 at 6:43 PM Zhijie Hou (Fujitsu) wrote:
>
> On Mon, Jun 2, 2025 at 2:39 PM Amit Kapila wrote:
> >
> > On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > Attaching the V32 patch set which addressed comments in [1]~[5].
> > >
> >
> > Review comments:
>
On Mon, Jun 2, 2025 at 12:09 PM Amit Kapila wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
> >
>
> Review comments:
> ===
> *
> +advance_conflict_slot_xmin(FullTransactionId new_xmin)
>
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V32 patch set which addressed comments in [1]~[5].
>
Review comments:
===
*
+advance_conflict_slot_xmin(FullTransactionId new_xmin)
+{
+ FullTransactionId full_xmin;
+ FullTransactionId next_full_xid;
+
+ A
On Tue, May 27, 2025 at 3:59 PM shveta malik wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
>
> Thanks for the patch, I am still reviewing the patches, please find
> few trivial comments for patch0
On Tue, May 27, 2025 at 11:45 AM Amit Kapila wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
> >
> > >
> > > I am thinking can't we make it more deterministic such that when we
> > > get the status first time if
On Tue, May 27, 2025 at 3:59 PM shveta malik wrote:
>
> On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Attaching the V32 patch set which addressed comments in [1]~[5].
>
> Thanks for the patch, I am still reviewing the patches, please find
> few trivial comments for patch0
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V32 patch set which addressed comments in [1]~[5].
Thanks for the patch, I am still reviewing the patches, please find
few trivial comments for patch001:
1)
+ FullTransactionId last_phase_at; /* publisher transaction
On Mon, May 26, 2025 at 12:46 PM Zhijie Hou (Fujitsu)
wrote:
>
> On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
>
> >
> > I am thinking can't we make it more deterministic such that when we
> > get the status first time if we find some transactions that are in
> > commit phase then we should j
On Fri, May 23, 2025 at 11:51 PM Xuneng Zhou wrote:
> Thanks for the effort on the patches. I did a quick look on them before
> diving into the logic and discussion. Below are a few minor typos found in
> version 31.
Thanks for the comments! I have fixed these typos in latest version.
Best Regar
On Fri, May 23, 2025 at 2:45 PM shveta malik wrote:
>
> Thanks you for v31 patch-set. Please find few comments on patch001:
Thanks for the comments.
>
> 1)
>
> wait_for_local_flush:
>
> + if (data->last_recv_time &&
> + TimestampDifferenceExceeds(data->flushpos_update_time,
> +data->last
On Sat, May 24, 2025 at 6:28 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 11:00 AM Amit Kapila
> wrote:
> >
> > On Sat, May 24, 2025 at 10:29 AM Dilip Kumar
> wrote:
> > >
> > > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar
> wrote:
> > > >
> > > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zh
On Sun, May 25, 2025 at 4:36 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 4:46 PM Amit Kapila
> wrote:
> >
> > This sounds reasonable to me. Let us see what others think.
> >
>
> I was looking into the for getting the transaction status from
> publisher, what I would assume this patch shou
On Sat, May 24, 2025 at 4:46 PM Amit Kapila wrote:
>
> This sounds reasonable to me. Let us see what others think.
>
I was looking into the for getting the transaction status from
publisher, what I would assume this patch should be doing is request
the publisher status first time, and if some tra
On Sat, May 24, 2025 at 3:58 PM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 11:00 AM Amit Kapila wrote:
> >
> > On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
> > >
> > > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar
> > > wrote:
> > > >
> > > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zh
On Sat, May 24, 2025 at 11:00 AM Amit Kapila wrote:
>
> On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
> >
> > On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
> > >
> > > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> > > >
> > > Looking at v31-0001 again, most of it looks fine e
On Sat, May 24, 2025 at 10:29 AM Dilip Kumar wrote:
>
> On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
> >
> > On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> > >
> > Looking at v31-0001 again, most of it looks fine except this logic of
> > getting the commit_ts after marking the trans
On Sat, May 24, 2025 at 10:04 AM Dilip Kumar wrote:
>
> On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
> >
> Looking at v31-0001 again, most of it looks fine except this logic of
> getting the commit_ts after marking the transaction in commit. I see
> in RecordTransactionCommit(), we are set
On Fri, May 23, 2025 at 9:21 PM Xuneng Zhou wrote:
>
Looking at v31-0001 again, most of it looks fine except this logic of
getting the commit_ts after marking the transaction in commit. I see
in RecordTransactionCommit(), we are setting this flag
(DELAY_CHKPT_IN_COMMIT) to put the transaction in
Hi Zhijie,
Thanks for the effort on the patches. I did a quick look on them before
diving into the logic and discussion. Below are a few minor typos found in
version 31.
⸻
1. Spelling of “non-removable”
[PATCH v31 1/7]
In docs and code “removeable” vs. “removable” are used alternatively
On Thu, May 22, 2025 at 8:28 AM Zhijie Hou (Fujitsu)
wrote:
>
> Attaching the V31 patch set which addressed comments in [1]~[8].
>
Few comments:
1.
+ The oldest transaction ID along that is currently in the commit
+ phase on the server, along with its epoch.
The first 'a
On Fri, May 23, 2025 at 12:15 PM shveta malik wrote:
>
> Thanks you for v31 patch-set. Please find few comments on patch001:
>
> 1)
>
> wait_for_local_flush:
>
> + if (data->last_recv_time &&
> + TimestampDifferenceExceeds(data->flushpos_update_time,
> +data->last_recv_time, WalWriterDelay))
>
Thanks you for v31 patch-set. Please find few comments on patch001:
1)
wait_for_local_flush:
+ if (data->last_recv_time &&
+ TimestampDifferenceExceeds(data->flushpos_update_time,
+data->last_recv_time, WalWriterDelay))
+ {
+ XLogRecPtr writepos;
+ XLogRecPtr flushpos;
+ bool have_pending_tx
On Fri, May 16, 2025 at 7:31 PM Amit Kapila wrote:
>
> A few more comments:
> =
> 3.
> maybe_advance_nonremovable_xid(RetainConflictInfoData *data,
>bool status_received)
> {
> /* Exit early if retaining conflict information is not required */
> if (!MySubscription->retainconfl
On Tue, May 20, 2025 at 6:30 PM shveta malik wrote:
>
> Few more comments mostly for patch001:
Thanks for the comments!
>
> 4)
> For this feature, since we are only interested in remote UPDATEs happening
> concurrently, so shall we ask primary to provide oldest "UPDATE"
> transaction-id in comm
On Tue, May 20, 2025 at 11:08 AM shveta malik wrote:
>
> Please find few more comments:
Thanks for the comments!
>
> 2)
>
> --
> send_feedback(last_received, requestReply, requestReply);
>
> + maybe_advance_nonremovable_xid(&data, false);
> +
> /*
> * Force reporting to ensure l
On Fri, May 16, 2025 at 5:01 PM Amit Kapila wrote:
>
Please find some more comments on the 0001 patch:
1.
We need to know about such transactions
+ * for conflict detection and resolution in logical replication. See
+ * GetOldestTransactionIdInCommit and its use.
Do we need to mention resolution
On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
>
> Please find few more comments:
>
> 1)
> ProcessStandbyPSRequestMessage:
> + /*
> + * This shouldn't happen because we don't support getting primary status
> + * message from standby.
> + */
> + if (RecoveryInProgress())
> + elog(ERROR, "the p
On Tue, May 20, 2025 at 10:24 AM Amit Kapila wrote:
>
> On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
> >
> > Please find few more comments:
> >
> > 1)
> > ProcessStandbyPSRequestMessage:
> > + /*
> > + * This shouldn't happen because we don't support getting primary status
> > + * message
On Tue, May 20, 2025 at 8:38 AM shveta malik wrote:
>
> Please find few more comments:
>
> 1)
> ProcessStandbyPSRequestMessage:
> + /*
> + * This shouldn't happen because we don't support getting primary status
> + * message from standby.
> + */
> + if (RecoveryInProgress())
> + elog(ERROR, "the p
Please find few more comments:
1)
ProcessStandbyPSRequestMessage:
+ /*
+ * This shouldn't happen because we don't support getting primary status
+ * message from standby.
+ */
+ if (RecoveryInProgress())
+ elog(ERROR, "the primary status is unavailable during recovery");
a) This error is not cle
On Thu, May 15, 2025 at 6:02 PM Amit Kapila wrote:
>
> Few minor comments on 0001:
> 1.
> + if (TimestampDifferenceExceeds(data->reply_time,
> +data->candidate_xid_time, 0))
> + ereport(ERROR,
> + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("oldest_nonremovable_xid transactio
On Fri, May 16, 2025 at 2:40 PM Amit Kapila wrote:
>
> On Fri, May 16, 2025 at 12:01 PM shveta malik wrote:
> >
> > On Fri, May 16, 2025 at 11:15 AM Amit Kapila
> > wrote:
> > >
> > >
> > > BTW, another related point is that when we decide to stop retaining
> > > dead tuples (via should_stop_co
On Fri, May 16, 2025 at 12:17 PM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 10:08 AM shveta malik wrote:
> >
> > On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
> > wrote:
> >
> > > > Few comments for patch004:
> > > > Config.sgml:
> > > > 1)
> > > > +
> > > > +Maximum du
On Fri, May 16, 2025 at 12:01 PM shveta malik wrote:
>
> On Fri, May 16, 2025 at 11:15 AM Amit Kapila wrote:
> >
> >
> > BTW, another related point is that when we decide to stop retaining
> > dead tuples (via should_stop_conflict_info_retention), should we also
> > consider the case that the app
On Fri, Apr 25, 2025 at 10:08 AM shveta malik wrote:
>
> On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
> wrote:
>
> > > Few comments for patch004:
> > > Config.sgml:
> > > 1)
> > > +
> > > +Maximum duration (in milliseconds) for which conflict
> > > +information can
On Fri, May 16, 2025 at 11:15 AM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
> >
> > 2)
> > in wait_for_local_flush(), we have
> > should_stop_conflict_info_retention() before 'AllTablesyncsReady'
> > check. Should we give a discount for table-sync time and avoid do
On Thu, May 15, 2025 at 6:02 PM Amit Kapila wrote:
>
> On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
> >
> > >
> > > Here is V30 patch set includes the following changes:
> > >
> >
> > Thank You for the patch, please find few concerns:
> >
> > 1)
> > By looking at code of ApplyLauncherMain(
On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
>
> 2)
> in wait_for_local_flush(), we have
> should_stop_conflict_info_retention() before 'AllTablesyncsReady'
> check. Should we give a discount for table-sync time and avoid doing
> stop-conflict-retention when table-sync is going on? This is
On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote:
>
> >
> > Here is V30 patch set includes the following changes:
> >
>
> Thank You for the patch, please find few concerns:
>
> 1)
> By looking at code of ApplyLauncherMain(), it appears that we stopped
> advancing shared-slot's xmin if any of the
On Fri, Apr 25, 2025 at 4:05 PM shveta malik wrote:
>
> >
> > Here is V30 patch set includes the following changes:
> >
>
> Thank You for the patch, please find few concerns:
>
Please find few more concerns:
3)
In get_candidate_xid(), we first set candidate_xid_time and later
candidate_xid. And
>
> Here is V30 patch set includes the following changes:
>
Thank You for the patch, please find few concerns:
1)
By looking at code of ApplyLauncherMain(), it appears that we stopped
advancing shared-slot's xmin if any of the subscriptions with
retain_conflict_info is disabled. If a subscription
On Thu, Apr 24, 2025 at 6:11 PM Zhijie Hou (Fujitsu)
wrote:
> > Few comments for patch004:
> > Config.sgml:
> > 1)
> > +
> > +Maximum duration (in milliseconds) for which conflict
> > +information can be retained for conflict detection by the apply
> > worker.
> > +
On Wed, Apr 16, 2025 at 10:30 AM shveta malik wrote:
>
> On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > Here's a rebased version of the patch series.
> >
>
Few comments for patch004:
Config.sgml:
1)
+
+Maximum duration (in milliseconds) for which conflict
+
On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here's a rebased version of the patch series.
>
Thanks for the patches.
While testing the GUC "max_conflict_retention_duration", I noticed a
behavior that seems to bypass its intended purpose.
On Pub, if a txn is stuck in the COMMI
On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
wrote:
>
> Here's a rebased version of the patch series.
>
Thanks Hou-San for the patches. I am going through this long thread
and patches. One doubt I have is whenever there is a chance of
conflict-slot update (either xmin or possibility of it
Dear Hou,
Thanks for updating the patch! I can finally come back to the thread.
Regarding the max_retain_conflict_duration, I prefer GUC approach because it has
already had a mechanism for converting unit: subscription option does not have
it.
Below part contains my comments:
01. check_new_clu
On Thu, 20 Feb 2025 at 12:50, Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the v28 patch set, which converts the subscription option
> max_conflict_retention_duration into a GUC. Other logic remains unchanged.
After discussing with Hou internally, I have moved this to the next
CommitFest since it wi
On Mon, Feb 10, 2025 at 2:45 PM Amit Kapila wrote:
>
> On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote:
> >
> > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
> >
> > >
> > > > I'm not really sure that these behaviors are the expected behavior of
> > > > users who set max_conflict_retenti
On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote:
>
> On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
>
> >
> > > I'm not really sure that these behaviors are the expected behavior of
> > > users who set max_conflict_retention_duration to some subscriptions.
> > > Or I might have set the wro
On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote:
>
> On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
> >
> > I'd like to confirm what users would expect of this
> > max_conflict_retention_duration option and it works as expected. IIUC
> > users would want to use this option when they want
On Thu, Feb 6, 2025 at 9:47 PM Amit Kapila wrote:
>
> On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
> >
> > I'd like to confirm what users would expect of this
> > max_conflict_retention_duration option and it works as expected. IIUC
> > users would want to use this option when they want
On Fri, Feb 7, 2025 at 2:18 AM Masahiko Sawada wrote:
>
> I'd like to confirm what users would expect of this
> max_conflict_retention_duration option and it works as expected. IIUC
> users would want to use this option when they want to balance between
> the reliable update_deleted conflict detec
On Tue, Feb 4, 2025 at 10:30 PM Amit Kapila wrote:
>
> On Wed, Feb 5, 2025 at 6:00 AM Masahiko Sawada wrote:
> >
> > On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
> > >
> > > >
> > > > I was not sure of the point of
> > > > making the max_conflict_retention_duration a per-subscription
> > >
On Thu, Jan 23, 2025 at 5:17 PM Zhijie Hou (Fujitsu)
wrote:
>
I was reviewing v26 patch set and have some comments so far I reviewed
0001 so most of the comments/question are from this patch.
comments on v26-0001
1.
+ next_full_xid = ReadNextFullTransactionId();
+ epoch = EpochFromFullTransactio
On Wed, Feb 5, 2025 at 6:00 AM Masahiko Sawada wrote:
>
> On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
> >
> > >
> > > I was not sure of the point of
> > > making the max_conflict_retention_duration a per-subscription
> > > parameter.
> > >
> >
> > The idea is to keep it at the same level a
On Sat, Feb 1, 2025 at 10:37 AM Amit Kapila wrote:
>
> On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
> >
> > On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila
> > wrote:
> > >
> > > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > > wrote:
> > > >
> > > > I have one question about the 0
On Fri, Jan 31, 2025 at 9:07 PM Amit Kapila wrote:
>
> On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
> >
> > On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila
> > wrote:
> > >
> > > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > > wrote:
> > > >
> > > > I have one question about the 0
On Sat, Feb 1, 2025 at 2:54 AM Masahiko Sawada wrote:
>
> On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila wrote:
> >
> > On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada
> > wrote:
> > >
> > > I have one question about the 0004 patch; it implemented
> > > max_conflict_retntion_duration as a subscri
On Thu, Jan 30, 2025 at 10:39 PM Amit Kapila wrote:
>
> On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada wrote:
> >
> > I have one question about the 0004 patch; it implemented
> > max_conflict_retntion_duration as a subscription parameter. But the
> > launcher invalidates the pg_conflict_detectio
On Fri, Jan 31, 2025 at 4:10 AM Masahiko Sawada wrote:
>
> I have one question about the 0004 patch; it implemented
> max_conflict_retntion_duration as a subscription parameter. But the
> launcher invalidates the pg_conflict_detection slot only if all
> subscriptions with retain_conflict_info stop
On Thu, Jan 23, 2025 at 3:47 AM Zhijie Hou (Fujitsu)
wrote:
>
> On Wednesday, January 22, 2025 7:54 PM Zhijie Hou (Fujitsu)
> wrote:
> > On Saturday, January 18, 2025 11:45 AM Zhijie Hou (Fujitsu)
> > wrote:
> > > I think invalidating the slot is OK and we could also let the apply
> > > worker
On Sat, Jan 18, 2025 at 9:15 AM Zhijie Hou (Fujitsu)
wrote:
>
>
> Here is the V24 patch set. I modified 0004 patch to implement the slot
> Invalidation part. Since the automatic recovery could be an optimization and
> the discussion is in progress, I didn't implement that part.
Few comments for p
On Wed, Jan 8, 2025 at 4:03 PM Masahiko Sawada wrote:
>
> On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila wrote:
> > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> > wrote:
> > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> > > wrote:
> > > > Here is further performance test analysis with v16
Dear hackers,
I've created a new script which simulates that user reduce the workload on the
publisher side. Attached zip file contains a script, execution log and pgbench
outputs. Experiments were done with v24 patch set.
Abstract
==
In this test the conflict slot could be invalidated as e
On Fri, Jan 17, 2025 at 1:37 PM Masahiko Sawada wrote:
>
> On Thu, Jan 16, 2025 at 2:02 AM Amit Kapila wrote:
> >
> > On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > In the latest version, we implemented a simpler approach that allows the
> > > apply
> > > worker to
On Thu, Jan 16, 2025 at 2:02 AM Amit Kapila wrote:
>
> On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > In the latest version, we implemented a simpler approach that allows the
> > apply
> > worker to directly advance the oldest_nonremovable_xid if the waiting time
> > excee
On Wed, Jan 15, 2025 at 9:38 AM Amit Kapila wrote:
>
> On Wed, Jan 15, 2025 at 5:57 AM Masahiko Sawada wrote:
> >
> > Probably retaining dead tuples based on the time duration or its age
> > might be other solutions, it would increase a risk of not being able
> > to detect update_deleted conflict
On Thu, Jan 16, 2025 at 4:02 PM Amit Kapila wrote:
> On Thu, Jan 16, 2025 at 3:45 PM Dilip Kumar wrote:
> >
> > On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila
> wrote:
> >>
> >> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
> >> >
> >> > Conflict detection of truncated updates is detected as upd
On Thu, Jan 16, 2025 at 3:45 PM Dilip Kumar wrote:
>
> On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila wrote:
>>
>> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
>> >
>> > Conflict detection of truncated updates is detected as update_missing
>> > and deleted update is detected as update_deleted. I
On Fri, Jan 3, 2025 at 4:31 PM Amit Kapila wrote:
> On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote:
> >
> > Conflict detection of truncated updates is detected as update_missing
> > and deleted update is detected as update_deleted. I was not sure if
> > truncated updates should also be detected
On Wed, Jan 15, 2025 at 2:20 PM Zhijie Hou (Fujitsu)
wrote:
>
> In the latest version, we implemented a simpler approach that allows the apply
> worker to directly advance the oldest_nonremovable_xid if the waiting time
> exceeds the newly introduced option's limit. I've named this option
> "max_c
On Wed, Jan 15, 2025 at 5:57 AM Masahiko Sawada wrote:
>
> On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote:
> >
> > As of now, I can't think of a way to throttle the publisher when the
> > apply_worker lags. Basically, we need some way to throttle (reduce the
> > speed of backends) when the app
On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote:
>
> On Tue, Jan 14, 2025 at 7:14 AM Masahiko Sawada wrote:
> >
> > On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila
> > wrote:
> > >
> > > I don't think we can avoid accumulating garbage especially when the
> > > workload on the publisher is more.
On Tue, Jan 14, 2025 at 7:14 AM Masahiko Sawada wrote:
>
> On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila wrote:
> >
> > I don't think we can avoid accumulating garbage especially when the
> > workload on the publisher is more. Consider the current case being
> > discussed, on the publisher, we hav
On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila wrote:
>
> On Fri, Jan 10, 2025 at 6:13 AM Masahiko Sawada wrote:
> >
> > 3. If the apply worker cannot catch up, it could enter to a bad loop;
> > the publisher sends huge amount of data -> the apply worker cannot
> > catch up -> it needs to wait for
Here are the performance test results and analysis with the recent patches.
Test Setup:
- Created Pub and Sub nodes with logical replication and below configurations.
autovacuum_naptime = '30s'
shared_buffers = '30GB'
max_wal_size = 20GB
min_wal_size = 10GB
track_commit_timestamp =
On Fri, Jan 10, 2025 at 6:13 AM Masahiko Sawada wrote:
>
> 3. If the apply worker cannot catch up, it could enter to a bad loop;
> the publisher sends huge amount of data -> the apply worker cannot
> catch up -> it needs to wait for a longer time to advance its
> oldest_nonremovable_xid -> more ga
On Friday, January 10, 2025 8:43 AM Masahiko Sawada
wrote:
Hi,
>
> On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > >
> > > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
>
> > > wrote
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote:
>
> On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote:
> > [3] Test with pgbench run on both publisher and subscriber.
> >
> > Test setup:
> > - Tests performed on pgHead + v16 patches
> > - Created a pub-sub replication system.
> > - Paramet
On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu)
wrote:
>
> On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
> wrote:
>
> Hi,
>
> >
> > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> > wrote:
> > >
> > >
On Wednesday, January 8, 2025 3:49 PM Nisha Moond
wrote:
>
> On Tue, Jan 7, 2025 at 6:04 PM Zhijie Hou (Fujitsu)
> wrote:
> >
> >
> > Attached the V19 patch which addressed comments in [1][2][3][4][5][6][7].
> >
>
> Here are a couple of initial review comments on v19 patch set:
>
> 1) The sub
On Wednesday, January 8, 2025 7:03 PM vignesh C wrote:
Hi,
> Consider a LR setup with retain_conflict_info=true for a table t1:
> Publisher:
> insert into t1 values(1);
> -- Have a open transaction before delete operation in subscriber begin;
>
> Subscriber:
> -- delete the record that was repl
On Wed, Jan 8, 2025 at 2:24 PM Amit Kapila wrote:
>
> On Wed, Jan 8, 2025 at 2:15 PM Masahiko Sawada wrote:
> >
> > On Tue, Jan 7, 2025 at 2:49 AM Amit Kapila wrote:
> > > >
> > > > We thought of another approach, which is to create/drop this slot first
> > > > as
> > > > soon as one enables/di
On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
Hi,
>
> On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > > wrote:
> > > > On Wed, Jan 8
On Thursday, January 9, 2025 9:48 AM Masahiko Sawada
wrote:
Hi,
>
> On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
> wrote:
> >
> > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > > wrote:
> > > > On We
On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu)
wrote:
>
> On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
> wrote:
>
> Hi,
>
> > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> > wrote:
> > > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> > wrote:
> > > >
> > > > On Thu, Dec 19, 2024
On Tue, 7 Jan 2025 at 18:04, Zhijie Hou (Fujitsu)
wrote:
>
> On Friday, January 3, 2025 1:53 PM Amit Kapila
> wrote:
> >
> > On Wed, Dec 25, 2024 at 8:13 AM Zhijie Hou (Fujitsu)
> > wrote:
> > >
> > > Attach the new version patch set which addressed all other comments.
> > >
> >
> > Some more m
On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada
wrote:
Hi,
> On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila
> wrote:
> > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada
> wrote:
> > >
> > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> wrote:
> > > >
> > > >
> > > > [3] Test with pgbench r
On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila wrote:
>
> On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote:
> >
> > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond
> > wrote:
> > >
> > > Here is further performance test analysis with v16 patch-set.
> > >
> > >
> > > In the test scenarios already s
1 - 100 of 200 matches
Mail list logo
