One of the problems to allow logical decoding of in-progress (or prepared) transactions is that the transaction which we are decoding can abort concurrently and we might not be able to detect it which leads to getting the wrong version of a row from system catalogs. This can further lead to unpredictable behavior when decoding WAL with wrong catalog information.
The problem occurs when we have a sequence of operations like Alter-Abort-Alter as discussed previously in the context of two-phase transactions [1]. I am reiterating the same problem so as to ease the discussion for in-progress transactions. Suppose we created a table (mytable), then in some transaction (say with txid-200) we are altering it and after that aborting that txn. So pg_class will have something like this: xmin | xmax | relname 100 | 200 | mytable 200 | 0 | mytable After the previous abort, tuple (100,200,mytable) becomes visible to other transactions and if we will alter the table again then xmax of first tuple will be set current xid, resulting in following table: xmin | xmax | relname 100 | 300 | mytable 200 | 0 | mytable 300 | 0 | mytable At that moment we’ve lost information that first tuple was deleted by our txn (txid-200). And from POV of the historic snapshot, the first tuple (100, 300, mytable) will become visible and used to decode the WAL, but actually the second tuple should have been used. Moreover, such a snapshot could see both tuples (first and second) violating oid uniqueness. Now, we are planning to handle such a situation by returning ERRCODE_TRANSACTION_ROLLBACK sqlerrcode from system table scan APIs to the backend decoding a specific uncommitted transaction. The decoding logic on the receipt of such an sqlerrcode aborts the ongoing decoding, discard the changes of current (sub)xact and continue processing the changes of other txns. The basic idea is that while setting up historic snapshots (SetupHistoricSnapshot), we remember the xid corresponding to ReorderBufferTXN (provided it's not committed yet) whose changes we are going to decode. Now, after getting the tuple from a system table scan, we check if the remembered XID is aborted and if so we return a specific error ERRCODE_TRANSACTION_ROLLBACK which helps the caller to proceed gracefully by discarding the changes of such a transaction. This is based on the idea/patch [2][3] discussed among Andres Freund and Nikhil Sontakke in the context of logical decoding of two-phase transactions. We need to deal with this problem to allow logical decoding of in-progress transactions which is being discussed in a separate thread [4]. I have previously tried to discuss this in the main thread [5] but didn't get much response so trying again. Thoughts? [1] - https://www.postgresql.org/message-id/EEBD82AA-61EE-46F4-845E-05B94168E8F2%40postgrespro.ru [2] - https://www.postgresql.org/message-id/20180720145836.gxwhbftuoyx5h4gc%40alap3.anarazel.de [3] - https://www.postgresql.org/message-id/CAMGcDxcBmN6jNeQkgWddfhX8HbSjQpW%3DUo70iBY3P_EPdp%2BLTQ%40mail.gmail.com [4] - https://www.postgresql.org/message-id/688b0b7f-2f6c-d827-c27b-216a8e3ea700%402ndquadrant.com [5] - https://www.postgresql.org/message-id/CAA4eK1KtxLpSp2rP6Rt8izQnPmhiA%3D2QUpLk%2BvoagTjKowc0HA%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
