Re: An improvement of ProcessTwoPhaseBuffer logic

2025-01-09 Thread Давыдов Виталий
On Monday, December 30, 2024 04:08 MSK, Michael Paquier wrote: > Instead, I have gone with a new > FullTransactionIdFromCurrentEpoch() that replaces > AdjustToFullTransactionId(). It cleans up most of the calls to > ReadNextFullTransactionId() compared to the previous patch. It is > true that

Re: An improvement of ProcessTwoPhaseBuffer logic

2024-12-26 Thread Давыдов Виталий
Hi Michael, > Here are two patches to address both issues: Thank you for the preparing the patches and test simplification. My bad, I overcompilcated it. I concerned about twophase file name generation. The function TwoPhaseFilePath() is pretty straitforward and unambiguous in 16 and earlier

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2024-11-21 Thread Давыдов Виталий
On Thursday, November 21, 2024 17:56 MSK, "Vitaly Davydov" wrote: > I'm trying to create a perl test to reproduce it. Please, give me some time > to create the test script. Attached is the test script which reproduces my problem. It should be run on a patched postgresql with the following cha

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-15 Thread Давыдов Виталий
Dear All, On Wednesday, April 10, 2024 17:16 MSK, Давыдов Виталий wrote:  Hi Amit, Ajin, All Thank you for the patch and the responses. I apologize for my delayed answer due to some curcumstances. On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila wrote: Vitaly, does the minimal solution

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-10 Thread Давыдов Виталий
Hi Amit, Ajin, All Thank you for the patch and the responses. I apologize for my delayed answer due to some curcumstances. On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila wrote: Vitaly, does the minimal solution provided by the proposed patch (Allow to alter two_phase option of a subscri

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-03-05 Thread Давыдов Виталий
Hi Heikki, Thank you for the reply. On Tuesday, March 05, 2024 12:05 MSK, Heikki Linnakangas wrote:  In a nutshell, this changes PREPARE TRANSACTION so that if synchronous_commit is 'off', the PREPARE TRANSACTION is not fsync'd to disk. So if you crash after the PREPARE TRANSACTION has returne

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-29 Thread Давыдов Виталий
Dear All, Consider, please, my patch for async commit for twophase transactions. It can be applicable when catchup performance is not enought with publication parameter twophase = on. The key changes are: * Use XLogSetAsyncXactLSN instead of XLogFlush as it is for usual transactions. * In ca

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-27 Thread Давыдов Виталий
It is why I store 2pc state in the local memory in my patch. It would be good if you could link those threads.Sure, I will find and add some links to the discussions from past. Thank you! With best regards, Vitaly  On Tue, Feb 27, 2024 at 4:49 PM Давыдов Виталий wrote: > > Thank you for

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-27 Thread Давыдов Виталий
Hi Amit, Thank you for your interest in the discussion! On Monday, February 26, 2024 16:24 MSK, Amit Kapila wrote:   I think the reason is probably that when the WAL record for prepared is already flushed then what will be the idea of async commit here?I think, the idea of async commit shoul

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Amit, Amit Kapila wrote: I don't see we do anything specific for 2PC transactions to make them behave differently than regular transactions with respect to synchronous_commit setting. What makes you think so? Can you pin point the code you are referring to?Yes, sure. The function RecordTran

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Ajin, Thank you for your feedback. Could you please try to increase the number of clients (-c pgbench option) up to 20 or more? It seems, I forgot to specify it. With best regards, Vitaly Davydov On Fri, Feb 23, 2024 at 12:29 AM Давыдов Виталий wrote: Dear All, I'd like to presen

Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-22 Thread Давыдов Виталий
Dear All, I'd like to present and talk about a problem when 2PC transactions are applied quite slowly on a replica during logical replication. There is a master and a replica with established logical replication from the master to the replica with twophase = true. With some load level on the ma

Re: How to accurately determine when a relation should use local buffers?

2023-11-27 Thread Давыдов Виталий
Hi Aleksander, Well even assuming this patch will make it to the upstream some day, which I seriously doubt, it will take somewhere between 2 and 5 years. Personally I would recommend reconsidering this design. I understand what you are saying. I have no plans to create a patch for this issue. I

Re: How to accurately determine when a relation should use local buffers?

2023-11-23 Thread Давыдов Виталий
Hi Aleksander, I sort of suspect that you are working on a very specific extension and/or feature for PG fork. Any chance you could give us more details about the case?I'm trying to adapt a multimaster solution to some changes in pg16. We replicate temp table DDL due to some reasons. Furthermore,

Re: How to accurately determine when a relation should use local buffers?

2023-11-22 Thread Давыдов Виталий
Hi Aleksander, Thank you for your answers. It seems, local buffers are used for temporary relations unconditionally. In this case, we may check either relpersistence or backend id, or both of them. I didn't do a deep investigation of the code in this particular aspect but that could be a fair

How to accurately determine when a relation should use local buffers?

2023-11-20 Thread Давыдов Виталий
Dear Hackers, I would like to clarify, what the correct way is to determine that a given relation is using local buffers. Local buffers, as far as I know, are used for temporary tables in backends. There are two functions/macros (bufmgr.c): SmgrIsTemp, RelationUsesLocalBuffers. The first funct