On 29 January 2018 at 07:15, Nikhil Sontakke <nikh...@2ndquadrant.com> wrote:
>> Having this as responsibility of plugin sounds interesting. It certainly >> narrows the scope for which we need to solve the abort issue. For 2PC >> that may be okay as we need to somehow interact with transaction manager >> as Simon noted. I am not sure if this helps streaming use-case though as >> there is not going to be any external transaction management involved there. I think we should recognize that the two cases are different. 2PC decoding patch is looking at what happens AFTER a PREPARE has occurred on a transaction. Streaming looks to have streaming occur right in the middle of a transaction, so it has other concerns and likely other solutions as a result. >> In any case all this interlocking could potentially be made less >> impact-full by only doing it when we know the transaction did catalog >> changes prior to currently decoded change (which we do during decoding) >> since that's the only time we are interested in if it aborted or not. >> >> This all leads me to another idea. What if logical decoding provided API >> for "locking/unlocking" the currently decoded transaction against abort. >> This function would then be called by both decoding and output plugin >> before any catalog read. The function can be smart enough to be NOOP if >> the transaction is not running (ie we are not doing 2PC decoding or >> streaming) or when the transaction didn't do any catalog modifications >> (we already have that info easily accessible as bool). >> >> That would mean we'd never do any kind of heavy locking for prolonged >> periods of time (ie network calls) but only during catalog access and >> only when needed. It would also solve this for both 2PC and streaming >> and it would be easy to use by plugin authors. Just document that some >> call should be done before catalog access when in output plugin, can >> even be Asserted that the call was done probably. >> >> Thoughts? >> > > Yeah, this might work. We already have SET_LOCKTAG_TRANSACTION() via > which every transaction takes an exclusive lock on its own > transactionid when it starts, for example. We ideally want a single > solution to handle 2PC and ongoing (streaming) transactions. We could > introduce a new SET_LOCKTAG_LOGICALTRANSACTION(). The logical decoding > process could take a SHARED lock on this, check if the XID is still ok > to decode, read the catalog and unlock. Abort/Commit transaction > processing could take this in EXCLUSIVE mode. > > As mentioned above, the plugin API which takes this lock will be smart > enough to be a NOOP if the transaction is not running (i.e we are not > doing 2PC decoding or streaming) or when the transaction didn't do any > catalog modifications. I think it is enough to say that ROLLBACK PREPARED cannot abort a transaction while decoding is taking place, if the plugin decides it wishes to prevent that. So the plugin API must have a pre-rollback-prepared call to allow the plugin to decide how to handle that (Is that what you meant by the pre-abort hook?). That call will just return when and if it is OK to abort. Any waiting or throwing of ERRORs can occur inside the plugin. The plugin can provide a function to allow an abort that type of abort if it wishes, but no new core functionality required for that. If needed, we would wait on a latch the plugin provides for that. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
