Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-04-03 Thread Masahiko Sawada
On Mon, Aug 12, 2024 at 9:25 PM cca5507 wrote: > > Hi, > > I refactor the code and fix the git apply warning according to [1]. > > Here are the new version patches. > I've investigated the reported issue and reviewed the patch. IIUC to fix this issue, what we need to do is to track catalog-change

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-28 Thread Ajin Cherian
On Thu, Mar 27, 2025 at 2:47 PM cca5507 wrote: > > Hi, > > According to the comment above DecodeTXNNeedSkip(), transactions committed > before SNAPBUILD_CONSISTENT state won't be decoded. It's important for > initial table data synchronization because the table sync workers use the > consistent

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-27 Thread cca5507
Hi, This change doesn't solve a bug. But I think it makes the code and comments more consistent. I currently have no idea about how to test it. -- Regards, ChangAo Chen

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-26 Thread cca5507
Hi, According to the comment above DecodeTXNNeedSkip(), transactions committed before SNAPBUILD_CONSISTENT state won't be decoded. It's important for initial table data synchronization because the table sync workers use the consistent snapshot to copy data so transactions before SNAPBUILD_CONSI

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-26 Thread Ajin Cherian
On Thu, Mar 27, 2025 at 1:13 PM cca5507 wrote: > > Hi, > > - IIUC your "fast forward" concern is not related to this particular thread > but you > - think it's already an issue on the master branch (outside of the > BUILDING_SNAPSHOT > - handling we are discussing here), is that correct? (that's

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-26 Thread cca5507
Hi, There are 3 threads currently. The latest patches are v4-0001 and v3-0002 in [2]. [1]https://www.postgresql.org/message-id/flat/tencent_6aaf072a7623a11a85c0b5fd290232467...@qq.com [2]https://www.postgresql.org/message-id/flat/tencent_8dec9842690a9b6afd52d4659ef0700e9...@qq.com [3]https://

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2025-03-26 Thread Ajin Cherian
On Wed, Mar 26, 2025 at 9:11 PM cca5507 wrote: > > Hi, > > Thanks for the comments! > > Here are the new version patches, I think it will be more clear. > > -- +{ +/* + * Note that during or after BUILDING_SNAPSHOT, we need handle the xlog + * that might mark a transac

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-13 Thread cca5507
Hi, - IIUC your "fast forward" concern is not related to this particular thread but you - think it's already an issue on the master branch (outside of the BUILDING_SNAPSHOT - handling we are discussing here), is that correct? (that's also what your coding - changes makes me think of). If so, I

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-13 Thread Bertrand Drouvot
Hi, On Tue, Aug 13, 2024 at 03:32:42PM +0800, cca5507 wrote: > Hi, > > - I re-read your comments in [0] and it looks like you've concern about > - the 2 "if" I'm proposing above and the fast forward handling. Is that the > case > - or is your fast forward concern unrelated to my proposals? > >

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-13 Thread cca5507
Hi, - I re-read your comments in [0] and it looks like you've concern about - the 2 "if" I'm proposing above and the fast forward handling. Is that the case - or is your fast forward concern unrelated to my proposals? In your proposals, we will just return when fast forward. But I think we need

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
Hi, On Tue, Aug 13, 2024 at 12:23:04PM +0800, cca5507 wrote: > Hi, > > I refactor the code and fix the git apply warning according to [1]. > > > Here are the new version patches. Thanks! 1 === + /* True if the xlog marks the transaction as containing catalog changes */ + bool

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread cca5507
Hi, I refactor the code and fix the git apply warning according to [1]. Here are the new version patches. -- Regards, ChangAo Chen [1] https://www.postgresql.org/message-id/Zrmh7X8jYCbFYXjH%40ip-10-97-1-34.eu-west-3.compute.internal v3-0002-Add-test-case-snapshot_build-for-test_decoding.pa

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread cca5507
Hi, Thanks for the comments! - I think it's fine to skip during fast forward as we are not generating logical - changes. It's done that way in master, in your proposal and in my "if" proposals. - Note that my proposals related to the if conditions are for heap2_decode and - heap_decode (not xac

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
Hi, On Mon, Aug 12, 2024 at 04:34:25PM +0800, cca5507 wrote: > Hi, > > > 4, 5 === > > > > if (SnapBuildCurrentState(builder) < SNAPBUILD_BUILDING_SNAPSHOT || > >     (SnapBuildCurrentState(builder) == > SNAPBUILD_BUILDING_SNAPSHOT && info != XLOG_HEAP_INPLACE) || > >     ctx->fast_forward) >

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread cca5507
Hi, 4, 5 === > if (SnapBuildCurrentState(builder) < SNAPBUILD_BUILDING_SNAPSHOT || >     (SnapBuildCurrentState(builder) == SNAPBUILD_BUILDING_SNAPSHOT && info != XLOG_HEAP_INPLACE) || >     ctx->fast_forward) >     return; I think during fast forward, we also need handle the xlog that mark

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-11 Thread Bertrand Drouvot
Hi, On Sat, Aug 10, 2024 at 06:07:30PM +0800, cca5507 wrote: > Hi, > > > Thanks for the comments! > > > Here are the new version patches, I think it will be more clear. Thanks! 1 === When applying I get: Applying: Track transactions committed in BUILDING_SNAPSHOT. .git/rebase-apply/patch:7

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-10 Thread cca5507
Hi, Thanks for the comments! Here are the new version patches, I think it will be more clear. -- Regards, ChangAo Chen v3-0001-Track-transactions-committed-in-BUILDING_SNAPSHOT.patch Description: Binary data v3-0002-Add-test-case-snapshot_build-for-test_decoding.patch Description: Binary d

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-09 Thread Bertrand Drouvot
Hi, On Thu, Aug 08, 2024 at 03:53:29PM +0800, cca5507 wrote: > Hi, > > > Thanks for pointing it out! > > > Here are the new version patches with a test case. Thanks! I think the approach that the patch implements makes sense and that we should track the transactions that have been commmitted

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-08 Thread cca5507
Hi, Thanks for pointing it out! Here are the new version patches with a test case. -- Regards, ChangAo Chen -- Original -- From: "Bertrand

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-07 Thread Bertrand Drouvot
Hi, On Mon, Jun 10, 2024 at 10:04:31PM +0800, cca5507 wrote: > Thank you for reply!I am trying to fix it. This patch (pass check-world) will > track txns > committed in BUILDING_SNAPSHOT state and can fix this bug. Thanks for the report and the patch! I did not look at the patch in detail but I

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-06-10 Thread cca5507
Thank you for reply!I am trying to fix it. This patch (pass check-world) will track txns committed in BUILDING_SNAPSHOT state and can fix this bug. -- Regards, ChangAo Chen v1-0001-Track-transactions-committed-in-BUILDING_SNAPSHOT.patch Description: Binary data

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-06-10 Thread Michael Paquier
On Sun, Jun 09, 2024 at 11:21:52PM +0800, cca5507 wrote: > Hello hackers, I found that we currently don't track txns committed in > BUILDING_SNAPSHOT state because of the code in xact_decode(): > /* >* If the snapshot isn't yet fully built, we cannot decode anything, so >* bai

Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-06-10 Thread cca5507
Hello hackers, I found that we currently don't track txns committed in BUILDING_SNAPSHOT state because of the code in xact_decode(): /* * If the snapshot isn't yet fully built, we cannot decode anything, so * bail out. */ if (SnapBuildCurrentState(builder)