Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-29 Thread Amit Kapila
On Tue, Jul 29, 2025 at 4:26 PM vignesh C wrote: > > On Tue, 29 Jul 2025 at 14:46, Ajin Cherian wrote: > > > > On Tue, Jul 29, 2025 at 1:13 PM Amit Kapila wrote: > > > > > > Yes, that makes sense to me. For HEAD and PG18, we can still add a new > > >

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-28 Thread Amit Kapila
On Tue, Jul 29, 2025 at 2:43 AM Doruk Yilmaz wrote: > > On Mon, Mar 3, 2025 at 6:39 AM Amit Kapila wrote: > > > > To use replication_origin by multiple processes, one must maintain the > > commit order as we do internally by allowing the leader process to > > w

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-28 Thread Amit Kapila
assertion check when the assertion is enabled in this build. > Thought? > Yes, that makes sense to me. For HEAD and PG18, we can still add a new argument to the API. For other bank branches, it is better to use a new Ex function as suggested by Kuroda-San. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-28 Thread Amit Kapila
n't do any such thing in RelationFindReplTupleSeq(), so, if we do something differently here, it should be explained in the comments. -- With Regards, Amit Kapila.

Re: Enhance pg_createsubscriber to create required standby.

2025-07-25 Thread Amit Kapila
On Wed, Jun 18, 2025 at 10:43 AM David G. Johnston wrote: > > On Tue, Jun 17, 2025 at 9:22 PM Amit Kapila wrote: >> >> >> As shown in Vignesh's email [1] (point 4), there could be multiple >> additional parameters required for the first option suggested b

Re: Conflict detection for update_deleted in logical replication

2025-07-25 Thread Amit Kapila
he may not be interested in eventual consistency as some of the data may not be replicated, so a conflict detection followed by any resolution may not be helpful. The other point is that if we report update_delete in such cases, it won't be reliable, sometimes it can be update_missing as vacuum would have removed the row, OTOH, if we report update_missing, it will always be the same conflict, and we can document it. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-24 Thread Amit Kapila
his can happen then still in most cases the index scan will happen. -- With Regards, Amit Kapila.

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-23 Thread Amit Kapila
w about > "lock_shared_object"? > I think if we lock in a caller, we don't need to use any lock during table_open. We can use the parameter name as already_locked as we do at some other places in the code. -- With Regards, Amit Kapila.

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-22 Thread Amit Kapila
cessExclusiveLock on SubscriptionRelRelationId in DropSubscription. Kindly test again after fixing the first comment above. -- With Regards, Amit Kapila.

Re: Question on any plans to use the User Server/User Mapping to provide Logical Replication Subscriptions the user/password in an encrypted manner

2025-07-22 Thread Amit Kapila
e13de9333f2c09.camel%40j-davis.com -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-22 Thread Amit Kapila
the ideas for future optimization and or special notes for some not so obvious design decisions. -- With Regards, Amit Kapila.

Re: Question on any plans to use the User Server/User Mapping to provide Logical Replication Subscriptions the user/password in an encrypted manner

2025-07-22 Thread Amit Kapila
totally hidden from prying eyes. > Can you check the work being discussed in thread [1] and see if that addresses your requirement? [1] - https://www.postgresql.org/message-id/149ff9264db27cdf724b65709fbbaee4bf316835.camel%40j-davis.com -- With Regards, Amit Kapila.

Re: Issues with hash and GiST LP_DEAD setting for kill_prior_tuple

2025-07-22 Thread Amit Kapila
e of the interlocking provided by vacuum scan. See comments atop _hash_kill_items() [There are never any scans active in a bucket at the time VACUUM begins ...]. I think we only need to worry about vacuum processing the page after we have collected the killed_items which shouldn't happen as explained in comments atop _hash_kill_items(). -- With Regards, Amit Kapila.

Re: Update Examples in Logical Replication Docs

2025-07-21 Thread Amit Kapila
lumns' was added in "\dRp+" by commit [3], so I think we should > update the docs to reflect the same. I have attached a patch with the > changes. > The changes apply to both the branches REL_18_STABLE and HEAD. > Thanks for the report and patch. I'll take care of it. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-21 Thread Amit Kapila
On Mon, Jul 21, 2025 at 11:27 PM Masahiko Sawada wrote: > > On Sun, Jul 20, 2025 at 9:00 PM Amit Kapila wrote: > > > > > If so, I agree > > with you, we don't need XIDs of other databases as logical WALSender > > will anyway won't process transaction

Re: Logical Replication of sequences

2025-07-21 Thread Amit Kapila
hould we do anything more here? > It would be better if we can give ERROR for options that are not specific to sequences. -- With Regards, Amit Kapila.

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
There could be multiple ways to preserve the slot, one is users always create a special slot on the primary for this purpose or we can provide a slot_option which users can specify/alter so that they get ERROR/WARNING on the last such slot being dropped. I feel we should choose the simplest option and rely on users to use the feature appropriately. We can always enhance the feature in future versions based on feedback from the field. -- With Regards, Amit Kapila.

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > > > > > One concern I have is regarding the default setting of > > > 'force_slot_drop' . I assume the default value of this new D

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:08 AM shveta malik wrote: > > On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > > wrote: > &g

Re: Document slot's restart_lsn can go backward

2025-07-20 Thread Amit Kapila
uroda proposed a patch [5], which improves > comments by describing this behavior. As proposed by Amit Kapila [6], > I'm starting a new thread about this comments patch. > > Any feedback on this patch? > I have tried to slightly improve the comments. You can include the suggesti

Re: Conflict detection for update_deleted in logical replication

2025-07-20 Thread Amit Kapila
On Sat, Jul 19, 2025 at 10:32 AM Amit Kapila wrote: > > On Sat, Jul 19, 2025 at 3:01 AM Masahiko Sawada wrote: > > > > On Fri, Jul 18, 2025 at 5:03 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > Here are some review comments and questions:

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-19 Thread Amit Kapila
romotion or other reason like timeout. However, at this stage it is not very clear whether it will be useful to provide additional timeout parameter. But we can consider retruning true/false depending on whether we are successful in syncing the slots or not. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-18 Thread Amit Kapila
or_conflict > Among these, the first option is better but I think it is better to name it just retain_dead_tuples. The explanation of the option will explain its use. It is similar to other options like binary or streaming. We are not naming them like request_data_binary_format to make the meaning apparent. There is a value in keeping names succinct. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-07-18 Thread Amit Kapila
dea to add comments in this area. But as this is for pre-existing cases, won't it be better to start a new thread explaining the cases and a patch? We may get feedback from others as well. -- With Regards, Amit Kapila.

Re: Logical Replication of sequences

2025-07-18 Thread Amit Kapila
)); [1]: https://www.postgresql.org/message-id/CAA4eK1%2B6L%2BAoGS3LHdnYnCE%3DnRHergSQyhyO7Y%3D-sOp7isGVMw%40mail.gmail.com [2]: https://www.postgresql.org/message-id/CABdArM52CSDuYsfTAEp4ZSWe%2BGFBvxgnPFgkG%2Bid9T88DUE%2B1Q%40mail.gmail.com -- With Regards, Amit Kapila.

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-18 Thread Amit Kapila
On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > > wrote: >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
On Fri, Jul 18, 2025 at 12:26 AM Masahiko Sawada wrote: > > On Thu, Jul 17, 2025 at 2:36 AM Amit Kapila wrote: > > > > What if we make DROP SUBSCRIPTION fail if it can lead to removal of > > the last slot on publisher and allow DROP to succeed when the > > subscri

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Tue, Jun 10, 2025 at 02:

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
iption option) is set? Now, users can still be allowed to Drop the subscription, if it disassociates the subscription from the slot by using method explained in docs [1] (See Notes section). Similarly when a user is trying to drop the last logical slot via pg_drop_replication_slot, we will allow it only with the force option. This should ensure that the user is aware of the consequences of dropping the last slot. -- With Regards, Amit Kapila.

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-16 Thread Amit Kapila
usiveLock on ReplicationOriginRelationId via replorigin_drop_by_name() and then RowExclusiveLock on SubscriptionRelRelationId via UpdateSubscriptionRelState(). Won't such a scenario taking conflicting locks in reverse order can lead to deadlock at least in PG15? -- With Regards, Amit Kapila.

Re: Logical replication prefetch

2025-07-15 Thread Amit Kapila
On Mon, Jul 14, 2025 at 3:13 PM Amit Kapila wrote: > > On Sun, Jul 13, 2025 at 6:06 PM Konstantin Knizhnik > wrote: > > > > On 13/07/2025 1:28 pm, Amit Kapila wrote: > > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > > wrote: > > >>

Re: failover logical replication slots

2025-07-15 Thread Amit Kapila
ll slots on new standby after failover to avoid excessive resource usage. -- With Regards, Amit Kapila.

Re: Report bytes and transactions actually sent downtream

2025-07-14 Thread Amit Kapila
On Mon, Jul 14, 2025 at 10:55 AM Ashutosh Bapat wrote: > > On Sun, Jul 13, 2025 at 4:34 PM Amit Kapila wrote: > > > > > > I think we don't want to make it mandatory for plugins to implement > > these stats, so instead of throwing ERROR, the view should show

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
and try to come up with a good way to perform pre-fetch or parallelization of short transactions. -- With Regards, Amit Kapila.

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
On Sun, Jul 13, 2025 at 6:06 PM Konstantin Knizhnik wrote: > > On 13/07/2025 1:28 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > wrote: > >> There is well known Postgres problem that logical replication subscriber > >> c

Re: failover logical replication slots

2025-07-13 Thread Amit Kapila
n implementing that API as well. Note: I suggest starting a new thread with the concrete proposal for the new API or GUC, stating how it will be helpful. It might help in getting suggestions from others as well. -- With Regards, Amit Kapila.

Re: Report bytes and transactions actually sent downtream

2025-07-13 Thread Amit Kapila
On Tue, Jul 1, 2025 at 7:35 PM Ashutosh Bapat wrote: > > On Tue, Jul 1, 2025 at 4:23 PM Amit Kapila wrote: > > > > On Mon, Jun 30, 2025 at 3:24 PM Ashutosh Bapat > > wrote: > > > > > > Hi All, > > > In a recent logical replication issue, t

Re: Logical replication prefetch

2025-07-13 Thread Amit Kapila
e pub-sub pairs by using row filters. -- With Regards, Amit Kapila.

Re: Logical replication prefetch

2025-07-12 Thread Amit Kapila
On Fri, Jul 11, 2025 at 7:49 PM Konstantin Knizhnik wrote: > > On 08/07/2025 2:51 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > wrote: > >> There is well known Postgres problem that logical replication subscriber > >> c

Re: Conflict detection for update_deleted in logical replication

2025-07-11 Thread Amit Kapila
On Thu, Jul 10, 2025 at 6:46 PM Masahiko Sawada wrote: > > On Wed, Jul 9, 2025 at 9:09 PM Amit Kapila wrote: > > > > > > I think that even with retain_conflict_info = off, there is probably a > > > point at which the subscriber can no longer keep up with the >

Re: Logical replication prefetch

2025-07-11 Thread Amit Kapila
On Wed, Jul 9, 2025 at 12:08 AM Konstantin Knizhnik wrote: > > On 08/07/2025 2:51 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > wrote: > > It is possible to enforce parallel apply of short > transactions using `debug_logical_replica

Re: A recent message added to pg_upgade

2025-07-10 Thread Amit Kapila
good to me as well. Shall we commit this today or wait for beta2 as per email by Tom [1]? We are on the borderline to see that most of the BF members have run with this, but as the change is straightforward, I think we can proceed. [1] - https://www.postgresql.org/message-id/1625598.1752180243%40sss.pgh.pa.us -- With Regards, Amit Kapila.

Re: A recent message added to pg_upgade

2025-07-10 Thread Amit Kapila
a few special things in other parts of the code during BinaryUpgrade, but don't mention those, so don't think we need to mention this one as well. -- With Regards, Amit Kapila.

Re: A recent message added to pg_upgade

2025-07-09 Thread Amit Kapila
r exposed check_hook functions check_max_slot_wal_keep_size() and check_idle_replication_slot_timeout() in backbranch? Shall we remove there as well or leave them to avoid the risk of breaking any application? -- With Regards, Amit Kapila.

Re: Using failover slots for PG-non_PG logical replication

2025-07-09 Thread Amit Kapila
t: ""+ or when creating replication slots directly" you mentioned could be considered to be added. But I see that is already explained in the link mentioned in the doc, see [1]. So, I suggest we leave this part of docs as it is. [1]: https://www.postgresql.org/docs/current/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS-SYNCHRONIZATION -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-09 Thread Amit Kapila
c2be62e9%40garret.ru [2] - https://www.postgresql.org/message-id/CABdArM4OEwmh_31dQ8_F__VmHwk2ag_M%3DYDD4H%2ByYQBG%2BbHGzg%40mail.gmail.com -- With Regards, Amit Kapila.

Re: A recent message added to pg_upgade

2025-07-08 Thread Amit Kapila
On Tue, Jul 8, 2025 at 4:49 PM Dilip Kumar wrote: > > On Tue, Jul 8, 2025 at 2:29 PM Amit Kapila wrote: > > > > On Tue, Jul 8, 2025 at 11:32 AM Dilip Kumar wrote: > > > > > > On Mon, Jul 7, 2025 at 11:22 PM Tom Lane wrote: > > > > > >

Re: Logical replication prefetch

2025-07-08 Thread Amit Kapila
ke more work > is needed here. > I understand that it is just a POC, so you haven't figured out all the details, but it would be good to know the reason of these deadlocks. > > I wonder if such LR prefetching approach is considered to be useful? > Or it is better to investigate other ways to improve LR apply speed > (parallel apply)? > I think it could be a good intermediate step till we are able to find a solution for tracking the dependencies. Do you think this work will be useful once we have parallel apply, and if so how? -- With Regards, Amit Kapila.

Re: A recent message added to pg_upgade

2025-07-08 Thread Amit Kapila
max_slot_wal_keep_size, and also skip InvalidateObsoleteReplicationSlots. The one advantage of such a change is that after this, we can remove Assert in InvalidatePossiblyObsoleteSlot, remove check_hook functions for GUCs max_slot_wal_keep_size and idle_replication_slot_timeout, and remove special settings for these GUCs in pg_upgrade. -- With Regards, Amit Kapila.

Re: Using failover slots for PG-non_PG logical replication

2025-07-03 Thread Amit Kapila
On Thu, Jul 3, 2025 at 7:07 PM Ashutosh Bapat wrote: > > On Thu, Jul 3, 2025 at 9:32 AM Amit Kapila wrote: > > > > On Wed, Jul 2, 2025 at 5:50 PM Ashutosh Bapat > > wrote: > > > > > > On Wed, Jul 2, 2025 at 12:36 PM shveta malik > > > wro

Re: Conflict detection for update_deleted in logical replication

2025-07-03 Thread Amit Kapila
On Thu, Jul 3, 2025 at 10:57 AM Dilip Kumar wrote: > > On Thu, Jul 3, 2025 at 10:43 AM Amit Kapila wrote: > > > > On Thu, Jul 3, 2025 at 10:26 AM Dilip Kumar wrote: > > > > > > > > > You changes related to write barrier LGTM, however I have qu

Re: Conflict detection for update_deleted in logical replication

2025-07-02 Thread Amit Kapila
e this is a physical slot and we need to follow PhysicalConfirmReceivedLocation()/PhysicalReplicationSlotNewXmin(). The patch has kept a comment in advance_conflict_slot_xmin() as to why it is okay not to flush the slot immediately. -- With Regards, Amit Kapila.

Re: Using failover slots for PG-non_PG logical replication

2025-07-02 Thread Amit Kapila
n-native logical replication solutions, something on the lines of what Shveta is proposing. OTOH, if you or Shveta have some clear guidelines for how a downstream can find the required slots which can work in all or most cases, then it is okay to mention that as well. -- With Regards, Amit Kapila.

Re: Report bytes and transactions actually sent downtream

2025-07-01 Thread Amit Kapila
mple, one may be interested in how much plugin has filtered the data because it was not published or because something like row_filter caused it skip sending such data? -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-07-01 Thread Amit Kapila
On Tue, Jul 1, 2025 at 10:53 AM Dilip Kumar wrote: > > On Tue, Jul 1, 2025 at 10:31 AM Dilip Kumar wrote: > > > > On Mon, Jun 30, 2025 at 6:59 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Mon, Jun 30, 2025 at 7:22 PM Amit Kapila wrote: > > &g

Re: Correct comment atop PublicationPartOpt

2025-06-30 Thread Amit Kapila
). > Corrected the comment. > LGTM. -- With Regards, Amit Kapila.

Re: Report replica identity in pg_publication_tables

2025-06-30 Thread Amit Kapila
f > publication, per say, so it's arguable whether it should be included > in pg_publication_tables or not. > Right, discussing the use case a bit more might help us to find if this is the right place to add 'replica identity' information. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-30 Thread Amit Kapila
the remaining part of the sentence. How about writing it as: "Acquire the conflict detection slot at startup to ensure it can be dropped if no longer needed after a restart."? -- With Regards, Amit Kapila.

Re: Check for existing replication slot in pg_createsubscriber

2025-06-29 Thread Amit Kapila
On Mon, Jun 30, 2025 at 8:37 AM Zane Duffield wrote: > > On Mon, Jun 30, 2025 at 1:01 PM Amit Kapila wrote: >> >> >> I see the difference you are pointing to. Ideally, the checks should >> be the same unless there is a specific reason for them to be >> differe

Re: Check for existing replication slot in pg_createsubscriber

2025-06-29 Thread Amit Kapila
the names for the objects created by pg_subscriber are either generated names (with an intention that it doesn't conflict) or user-provided. In both cases, chances should be less that they conflict with existing objects. -- With Regards, Amit Kapila.

Re: Minor patch; missing comment update in worker.c

2025-06-25 Thread Amit Kapila
On Mon, Jun 23, 2025 at 8:56 AM Amit Kapila wrote: > > On Mon, Jun 23, 2025 at 8:22 AM Hayato Kuroda (Fujitsu) > wrote: > > > > But this is not correct anymore, 1462aad2 allows to alter two_phase option. > > I was an original author, but I did oversight. > &

Re: Conflict detection for update_deleted in logical replication

2025-06-25 Thread Amit Kapila
quot;altering retain_conflict_info is allowed for disabled subscription"); In all places, the comments use failover as an option name, whereas it is testing retain_conflict_info. 3. It is better to merge the 0004 into 0001 as it tests the core part of the functionality added by 0001. -- With Regards, Amit Kapila.

Re: Logrep launcher race conditions leading to slow tests

2025-06-25 Thread Amit Kapila
g_tables_for_apply (the other caller of > logicalrep_worker_launch), it seems okay to ignore the > result of logicalrep_worker_launch, but I think it should > fill hentry->last_start_time before not after the call. With this, won't we end up retrying to launch the worker sooner if the launch took time, but still failed to launch the worker? [1] - https://www.postgresql.org/message-id/ff0663d9-8011-420f-a169-efbf57327cb5%40iki.fi -- With Regards, Amit Kapila.

Re: Logrep launcher race conditions leading to slow tests

2025-06-24 Thread Amit Kapila
On Tue, Jun 24, 2025 at 9:53 PM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Jun 24, 2025 at 5:26 AM Tom Lane wrote: > >> 1. WaitForReplicationWorkerAttach sometimes has to clear a process > >> latch event so that it can keep waiting for the worker to launc

Re: Improve doc on parallel stream changes for Stream Abort message

2025-06-24 Thread Amit Kapila
On Tue, Jun 24, 2025 at 11:58 AM Anthonin Bonnefoy wrote: > > On Tue, Jun 24, 2025 at 7:26 AM Amit Kapila wrote: > > How about a slightly modified version like: (a) The LSN of the abort > > operation, present only when the change stream can be applied in > > parallel.

Re: Improve doc on parallel stream changes for Stream Abort message

2025-06-24 Thread Amit Kapila
field is available since protocol version 4. (b) Abort timestamp of the transaction, present only when the change stream can be applied in parallel. The value is in number of microseconds since PostgreSQL epoch (2000-01-01). This field is available since protocol version 4. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-23 Thread Amit Kapila
shape now, can we merge them (0001, 0002, 0003)? -- With Regards, Amit Kapila. diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml index 4217a2e7dee..bfb9c2fb31c 100644 --- a/doc/src/sgml/logical-replication.sgml +++ b/doc/src/sgml/logical-replication.sgml @@

Re: Memory allocation error DDL invalidation (seen with 15.13 & 16.9)

2025-06-22 Thread Amit Kapila
Or will we need to reimplement our approach to RLS? > I think you can wait for a few weeks to see if someone can come up with a patch to improve this area. -- With Regards, Amit Kapila.

Re: Minor patch; missing comment update in worker.c

2025-06-22 Thread Amit Kapila
ike > that. How do you feel? > Thanks for the report and patch. I'll look into it. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-06-19 Thread Amit Kapila
le. As per analysis till now, it seems removal of new assert is correct and we just need to figure out the reason in all failure cases as to why the physical slot's restart_lsn goes backward, and then add a comment somewhere to ensure that we don't repeat a similar mistake in the future. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-06-19 Thread Amit Kapila
get the position later than the previous restart_lsn. > Thus, I propose to remove the > assertion introduced by ca307d5cec90. > If what I said above is correct, then the following part of the commit message will be incorrect: "As stated in the ReplicationSlotReserveWal() comment, this is not always true. Additionally, this issue has been spotted by some buildfarm members." -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-19 Thread Amit Kapila
On Tue, Jun 17, 2025 at 4:26 PM Zhijie Hou (Fujitsu) wrote: > > On Mon, Jun 16, 2025 at 7:37 PM Amit Kapila wrote: > > > > > > 3. Isn't the new check for logical slots in > > check_new_cluster_subscription_configuration() somewhat redundant

Re: Replication slot is not able to sync up

2025-06-19 Thread Amit Kapila
On Wed, Jun 18, 2025 at 10:56 AM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 8:52 AM shveta malik wrote: > > > > On Tue, Jun 17, 2025 at 12:01 PM Amit Kapila > > wrote: > > > > > > This whole paragraph sounds like a duplicate of its previous se

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-06-18 Thread Amit Kapila
a pattern, as you are suggesting. So, we can think of adding such special switches only for those objects, if required, similar to what pg_dump has for schema or tables. -- With Regards, Amit Kapila.

Re: Replication slot is not able to sync up

2025-06-18 Thread Amit Kapila
On Wed, Jun 18, 2025 at 8:52 AM shveta malik wrote: > > On Tue, Jun 17, 2025 at 12:01 PM Amit Kapila wrote: > > > > This whole paragraph sounds like a duplicate of its previous section, > > and the line alignment in the first paragraph has some issues. > >

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-06-17 Thread Amit Kapila
iscussion about leaving room > for future expansion, but I've only found one or two suggestions about > what might be needed. > The list can be longer than one or two. We may need to provide similar options for other objects, such as replication slots (synced failover replication slots on the physical standby), user-defined functions, triggers, views, materialized views, operators, policies, etc. And then, we would also need 'all' kind of additional option to allow cleaning all such objects. The newly formed subscriber may need a few of the objects that got replicated on the prior physical standby to operate, but not all. -- With Regards, Amit Kapila.

Re: Enhance pg_createsubscriber to create required standby.

2025-06-17 Thread Amit Kapila
On Thu, Jun 5, 2025 at 1:50 AM Peter Eisentraut wrote: > > On 04.06.25 11:56, Amit Kapila wrote: > >> It's not clear to me how this change now would substantially improve the > >> user experience. The number of characters you type is approximately the > >>

Re: Replication slot is not able to sync up

2025-06-16 Thread Amit Kapila
nt on the corresponding standby server. This is + needed to prevent any data loss and to allow logical replication to continue + ... This whole paragraph sounds like a duplicate of its previous section, and the line alignment in the first paragraph has some issues. -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-16 Thread Amit Kapila
heck_new_cluster_logical_replication_slots()? Can't we combine both? Apart from this, I have made a number of changes in the comments and a few other cosmetic changes in the attached. [1]: https://www.postgresql.org/message-id/28c8bf-68470780-3-51b29480%4089454035 -- With Regards, Amit Ka

Re: pglogical3 : support

2025-06-16 Thread Amit Kapila
s. > > Do we have any plan to make pglogical3 open source in near future? 😊 or is > there better way to achieve the same using open source packages ? > The PostgreSQL community doesn't support pglogical. You can contact pglogical community for your questions on the same. -- With Regards, Amit Kapila.

Re: Replication slot is not able to sync up

2025-06-13 Thread Amit Kapila
ch. It would also be better to briefly explain in user terms why the API is more prone to such a failure. -- With Regards, Amit Kapila.

Re: pg_recvlogical cannot create slots with failover=true

2025-06-13 Thread Amit Kapila
consistency in naming of these options across all APIs/tools. But, their usage in a tool like pg_recvlogical could be perceived differently as well, so it is also okay to consider naming them differently. -- With Regards, Amit Kapila.

Re: Logical Replication slot disappeared after promote Standby

2025-06-13 Thread Amit Kapila
7;t be able to reuse this slot to allow the subscribers to continue replication. (1) - https://www.postgresql.org/docs/devel/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS-SYNCHRONIZATION (2) - https://www.postgresql.org/docs/devel/logical-replication-failover.html -- With Regards, Amit Kapila.

Re: Question on error code selection in conflict detection

2025-06-13 Thread Amit Kapila
On Thu, Jun 12, 2025 at 6:56 PM Robert Haas wrote: > > On Thu, Jun 12, 2025 at 3:09 AM Amit Kapila wrote: > > cases like UPDATE_MISSING, DELETE_MISSING, the existing code > > ERRCODE_NO_DATA_FOUND seems to be an exact match. The LOG message > > appears when we don't

Re: Question on error code selection in conflict detection

2025-06-13 Thread Amit Kapila
odes-sqlstate-values-common-error#db2z_sqlstatevalues__classcode55 -- With Regards, Amit Kapila.

Re: failover logical replication slots

2025-06-12 Thread Amit Kapila
is attached to the primary, can we imagine that when the > sync worker process is started we check if a failover slot exists on the > standby, if so we drop it before recreating a new one for syncing? > This has the risk of dropping an unwarranted slot. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-06-12 Thread Amit Kapila
ant issue. From the other > > hand, > > the alternate solution simplifies the logic of WAL removal, backward > > compatible > > (avoids addition new in-memory states), decreases the number of locks in > > ReplicationSlotsComputeRequiredLSN - no need to recalculate oldest slots' > > restart lsn every time when a slot is advanced. > > So, my proposal is to commit the attached patchset to the HEAD, and > commit [1] to the back branches. Any objections? > No objections. I think we can keep discussing if slot's xmin computation has any issues or not, but you can proceed with the LSN stuff. -- With Regards, Amit Kapila.

Re: failover logical replication slots

2025-06-12 Thread Amit Kapila
an propose a patch in that sense if it seems appropriate. > Yeah, we can consider that, though I don't know at this stage if GUC is the only way, but I hope you understand that it will be for PG19. -- With Regards, Amit Kapila.

Re: failover logical replication slots

2025-06-12 Thread Amit Kapila
On Thu, Jun 12, 2025 at 3:07 PM Amit Kapila wrote: > > On Thu, Jun 12, 2025 at 2:32 PM Fabrice Chapuis > wrote: > > > > > After the first failover, the following failovers will work given that the > > sync flag is true on both the primary and standby slots. > &

Re: Question on error code selection in conflict detection

2025-06-12 Thread Amit Kapila
On Wed, Jun 11, 2025 at 7:33 PM Robert Haas wrote: > > On Tue, Jun 10, 2025 at 2:09 AM Amit Kapila wrote: > > Can we instead try to use other suitable existing error codes? > > Why? > > I mean, I'm not 100% against using existing error codes, but I feel > like we

Re: failover logical replication slots

2025-06-11 Thread Amit Kapila
s (maybe keep the slots that are required for failover) when starting the node as a standby. -- With Regards, Amit Kapila.

Re: Replication slot is not able to sync up

2025-06-11 Thread Amit Kapila
the slot on the primary using pg_logical_slot_get_changes or pg_logical_slot_get_binary_changes, allowing synchronization to proceed. Let me know what you think of above? -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-11 Thread Amit Kapila
to me. Do you think we can avoid this? Apart from this, I have made a number of cosmetic changes in the attached. Kindly include these in the next version, if these look okay to you. Also, I think we can combine 0001 and 0002 at this stage, as both are looking in good shape now. -- With Regards, Ami

Re: Replication slot is not able to sync up

2025-06-10 Thread Amit Kapila
that we can't yet build a consistent snapshot on the standby to allow decoding. Would it be better to have errdetail like: "Standby could not build a consistent snapshot to decode WALs at LSN %X/%X.? -- With Regards, Amit Kapila.

Re: Question on error code selection in conflict detection

2025-06-10 Thread Amit Kapila
those seem quite different; we can consider those separately. Thoughts? -- With Regards, Amit Kapila.

Re: Conflict detection for update_deleted in logical replication

2025-06-06 Thread Amit Kapila
sabled ALTER SUBSCRIPTION Here, we should have a WARNING like: "deleted rows to detect conflicts would not be removed till the subscription is enabled"; this should be followed by errdetail like: "Consider setting retain_conflict_info to false." -- With Regards, Amit Kapila.

Re: pg18: Virtual generated columns are not (yet) safe when superuser selects from them

2025-06-05 Thread Amit Kapila
traction, etc. Allowing UDFs with some safety definition can be done in future releases. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-06-05 Thread Amit Kapila
nto account when calculating the oldest This doesn't follow our practice for multi-line comments. -- With Regards, Amit Kapila.

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-06-05 Thread Amit Kapila
be more lazy in this case. > > Taking into account these thoughts, I can't see any problems with the > alternative > patch where oldest wal lsn is calculated only in checkpoint. > The alternative will needlessly prevent removing WAL segments in some cases when logical slots are in use. [1] - https://www.postgresql.org/message-id/CAA4eK1KMaPA5jir_SFu%2Bqr3qu55OOdFWVZpuUkqTSGZ9fyPpHA%40mail.gmail.com -- With Regards, Amit Kapila.

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Amit Kapila
s in backbranch, then kindly let us know. [1] - https://www.postgresql.org/message-id/CAA4eK1Jx7Ed_58%2BywXozmHvRRhAVO2Yfcyoi0e5PRexZn5A6Gw%40mail.gmail.com -- With Regards, Amit Kapila.

  1   2   3   4   5   6   7   8   9   10   >