On Sat, Aug 17, 2019 at 9:35 PM Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund <and...@anarazel.de> wrote: > > > > Again, I think it's not ok to just assume you can lock an essentially > > > > unbounded number of buffers. This seems almost guaranteed to result in > > > > deadlocks. And there's limits on how many lwlocks one can hold etc. > > > > > > I think for controlling that we need to put a limit on max prepared > > > undo? I am not sure any other way of limiting the number of buffers > > > because we must lock all the buffer in which we are going to insert > > > the undo record under one WAL logged operation. > > > > I heard that a number of times. But I still don't know why that'd > > actually be true. Why would it not be sufficient to just lock the buffer > > currently being written to, rather than all buffers? It'd require a bit > > of care updating the official current "logical end" of a log, but > > otherwise ought to not be particularly hard? Only one backend can extend > > the log after all, and until the log is externally visibily extended, > > nobody can read or write those buffers, no? > > Well, I don't understand why you're on about this. We've discussed it > a number of times but I'm still confused. I'll repeat my previous > arguments on-list: > > 1. It's absolutely fine to just put a limit on this, because the > higher-level facilities that use this shouldn't be doing a single > WAL-logged operation that touches a zillion buffers. We have been > careful to avoid having WAL-logged operations touch an unbounded > number of buffers in plenty of other places, like the btree code, and > we are going to have to be similarly careful here for multiple > reasons, deadlock avoidance being one. So, saying, "hey, you're going > to lock an unlimited number of buffers" is a straw man. We aren't. > We can't.
Right. So basically, we need to put a limit on how many max undo can be prepared under single WAL logged operation and that will internally put the limit on max undo buffers. Suppose we limit max_prepared_ undo to 2 then we need to lock at max 5 undo buffers. We need to somehow deal with the multi-insert code in the zheap because in that code for inserting in a single page we write one undo record per range if all the tuple which we are inserting on a single page are interleaved. But, maybe we can handle that by just inserting one undo record which can have multiple ranges. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com
