On Thu, Jan 30, 2025 at 08:57:23AM +0900, Michael Paquier wrote:
> So the situation is a bit of a mess in all the branches, a bit worse
> in 17~ due to the fact that epochs are poorly integrated, but I'm
> getting there with a set of patches mostly ready to be sent, and I
> think that this would be
On Wed, Jan 29, 2025 at 03:00:54PM +0300, Vitaly Davydov wrote:
> It seems, there are much deeper problems with twophase transactions
> as I thought. I'm interested in fixing twophase transactions,
> because I support a solution which actively uses twophase
> transactions. I'm interested to get mor
It seems, there are much deeper problems with twophase transactions as I
thought. I'm interested in fixing twophase transactions, because I support a
solution which actively uses twophase transactions. I'm interested to get more
deeply into the twophase functionality. Below, I just want to clari
On Wed, Jan 22, 2025 at 03:45:30PM +0300, Vitaly Davydov wrote:
> I hope you may find this patch worth to consider. You may consider
> this patch as one more step to a complete migration to
> FullTransactionId in twophase.c for 17+ versions.
Heavy refactorings are only going to be something for 18
On Thu, Jan 16, 2025 at 06:44:16PM -0800, Noah Misch wrote:
> I got here on a yak shave for postgr.es/m/20250111214454.9a.nmi...@google.com.
> I expect that project will still want FullTransactionIdFromAllowableAt(). If
> so, I'll include it in that thread's patch series.
Note that this one has b
Dear Hackers,
I hope you may find this patch worth to consider. You may consider this patch
as one more step to a complete migration to FullTransactionId in twophase.c for
17+ versions.
The patch fixes TwoPhaseFilePath function. The current version gets
TransactionId as an argument and makes a
On Thu, Jan 16, 2025 at 06:44:16PM -0800, Noah Misch wrote:
> On Fri, Jan 17, 2025 at 11:04:03AM +0900, Michael Paquier wrote:
>> +typedef void (*TwoPhaseCallback) (FullTransactionId fxid, uint16 info,
>>void *recdata, uint32 len);
>>
>> Based on your latest pat
On Fri, Jan 17, 2025 at 11:04:03AM +0900, Michael Paquier wrote:
> On Thu, Jan 16, 2025 at 04:52:21PM -0800, Noah Misch wrote:
> > In other words, the root problem that led to commits e358425 and 7e125b2 was
> > recovery interpreting pg_twophase file content before reaching consistency.
> > We can'
On Thu, Jan 16, 2025 at 04:52:21PM -0800, Noah Misch wrote:
> In other words, the root problem that led to commits e358425 and 7e125b2 was
> recovery interpreting pg_twophase file content before reaching consistency.
> We can't make the ProcessTwoPhaseBuffer() checks safe before reaching
> consiste
On Thu, Jan 16, 2025 at 12:52:54PM -0800, Noah Misch wrote:
> On Thu, Jan 16, 2025 at 04:50:09PM +0900, Michael Paquier wrote:
> > As far as I understand, the most important point of the logic is to
> > detect and discard the future files first in restoreTwoPhaseData() ->
> > ProcessTwoPhaseBuffer(
On Thu, Jan 16, 2025 at 04:50:09PM +0900, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 05:00:51PM -0800, Noah Misch wrote:
> > I think "using the current epoch" is wrong for half of the nextFullXid
> > values
> > having epoch > 0. For example, nextFullId==2^32 is in epoch 1, but all the
> > a
On Wed, Jan 15, 2025 at 05:00:51PM -0800, Noah Misch wrote:
> I agree with accepting +4 bytes in GlobalTransactionData.
Let's just bite the bullet and do that on HEAD and v17, then,
integrating deeper FullTransactionIds into the internals of
twophase.c.
> I think "using the current epoch" is wron
On Mon, Dec 30, 2024 at 10:08:31AM +0900, Michael Paquier wrote:
> On Fri, Dec 27, 2024 at 06:16:24PM +0300, Vitaly Davydov wrote:
> > As an idea, I would like to propose to store FullTransactionId in
> > global transaction state instead of TransactionId. I'm not sure, it
> > will consume significa
On Thu, Jan 09, 2025 at 11:21:38AM +0300, Давыдов Виталий wrote:
> Could you please send the latest version of the patch, if any? I haven't
> found any new patches in the latest email.
I've applied fixes for this stuff as of e3584258154f (down to 13) and
7e125b20eed6 (down to 17) with tests for a
On Monday, December 30, 2024 04:08 MSK, Michael Paquier
wrote:
> Instead, I have gone with a new
> FullTransactionIdFromCurrentEpoch() that replaces
> AdjustToFullTransactionId(). It cleans up most of the calls to
> ReadNextFullTransactionId() compared to the previous patch. It is
> true that
On Fri, Dec 27, 2024 at 06:16:24PM +0300, Vitaly Davydov wrote:
> In general, I like your solution to use FullTransactionId. I haven't
> found any evident problems. I just would like to propose to create a
> couple of new functions like RemoveTwoPhaseFileInCurrentEpoch,
> TwoPhaseFilePathInCurrentE
On Friday, December 27, 2024 10:37 MSK, Michael Paquier
wrote:
> So please see the attached. You will note that RemoveTwoPhaseFile(),
> ProcessTwoPhaseBuffer() and TwoPhaseFilePath() now require a
> FullTransactionId, hence callers need to think about the epoch to use.
> That should limit futur
On Thu, Dec 26, 2024 at 06:11:25PM +0300, Давыдов Виталий wrote:
> I concerned about twophase file name generation. The function
> TwoPhaseFilePath() is pretty straitforward and unambiguous in 16 and
> earlier versions. The logic of this function in 17+ seems to be more
> complex. I do not understa
Hi Michael,
> Here are two patches to address both issues:
Thank you for the preparing the patches and test simplification. My bad, I
overcompilcated it.
I concerned about twophase file name generation. The function
TwoPhaseFilePath() is pretty straitforward and unambiguous in 16 and earlier
On Thu, Dec 26, 2024 at 07:42:34AM +0900, Michael Paquier wrote:
> I suspect that this can be still dangerous as-is while complicating
> the code with more possible paths for the removal of the 2PC files,
> because we may still build a file name from an XID, and that's what's
> causing this issue..
On Wed, Dec 25, 2024 at 07:18:18PM +0300, Vitaly Davydov wrote:
> I tried your patch and it seems the server is started
> successfully. But I've found another problem in my synthetic test -
> it can not remove the file with the following message:
>
> LOG: database system was not properly shut dow
Hi Michael,
Thank you for the explanation and the patch! I'm happy, that I seem to be on
the right way.
On Wednesday, December 25, 2024 08:04 MSK, Michael Paquier
wrote:
> Vitaly, have you seen that in the wild as an effect of future 2PC files?
I haven't heard about this problem in production
On Tue, Dec 24, 2024 at 04:26:32PM +0300, Vitaly Davydov wrote:
> In some, very rare scenarios, the postgres instance will newer
> recover because of such logic. Imagine, that the two_phase directory
> contains some files with two-phase states of transactions of distant
> future. I assume, it can h
Dear Hackers,
I would like to discuss ProcessTwoPhaseBuffer function. It reads two-phase
transaction states from disk or the WAL. It takes xid as well as some other
input parameters and executes the following steps:
Step #1: Check if xid is committed or aborted in clog (TransactionIdDidCommit,
24 matches
Mail list logo
