Hi, On 2018-07-19 12:42:08 -0700, Andres Freund wrote: > I actually think the balance of all the solutions discussed in this > thread seem to make neutering pruning *a bit* by far the most palatable > solution. We don't need to fully prevent removal of such tuple chains, > it's sufficient that we can detect that a tuple has been removed. A > large-sledgehammer approach would be to just error out when attempting > to read such a tuple. The existing error handling logic can relatively > easily be made to work with that.
So. I'm just back from not working for a few days. I've not followed this discussion in all it's detail over the last months. I've an annoying bout of allergies. So I might be entirely off. I think this whole issue only exists if we actually end up doing catalog lookups, not if there's only cached lookups (otherwise our invalidation handling is entirely borked). And we should normally do cached lookups for a large large percentage of the cases. Therefore we can make the cache-miss cases a bit slower. So what if we, at the begin / end of cache miss handling, re-check if the to-be-decoded transaction is still in-progress (or has committed). And we throw an error if that happened. That error is then caught in reorderbuffer, the in-progress-xact aborted callback is called, and processing continues (there's a couple nontrivial details here, but it should be doable). The biggest issue is what constitutes a "cache miss". It's fairly trivial to do this for syscache / relcache, but that's not sufficient: there's plenty cases where catalogs are accessed without going through either. But as far as I can tell if we declared that all historic accesses have to go through systable_beginscan* - which'd imo not be a crazy restriction - we could put the checks at that layer. That'd require that an index lookup can't crash if the corresponding heap entry doesn't exist (etc), but that's something we need to handle anyway. The issue that multiple separate catalog lookups need to be coherent (say Robert's pg_class exists, but pg_attribute doesn't example) is solved by virtue of the the pg_attribute lookups failing if the transaction aborted. Am I missing something here? Greetings, Andres Freund
