On Thu, Jun 3, 2010 at 11:41 AM, Greg Stark <gsst...@mit.edu> wrote: > I think to make it work you need to store a whole 64-bit reference > transaction id consisting of both a cycle counter and a transaction > id. The invariant for the page is that every xid on the page can be > compared to that reference transaction id using normal transactionid > semantics. Actually I think the easiest way to do that is to set it to > the oldest xid on the page. The first thing to do before comparing any > transaction id on the page with a real transaction id would be to > figure out whether the reference xid is comparable to the live xid, > which if it's the oldest xid on the page implies they'll all be > comparable. > > The way to maintain that invariant would be that any xid insertion on > the page must advance the reference xid if it's not comparable to the > newly inserted xid. It has to be advanced to the oldest xid that's > still comparable with the newly inserted xid. Any xids on the page > that are older than the new refernce xid have to be frozen or removed. > I'm not sure how to do that without keeping clog forever though.
So the more I think about this the more I think it's unavoidable that we would need to retain clog forever. I think the goal here is to be able to load data into the database and then never write the data ever again. Even if you visit the page years later after the transaction ids have wrapped around several times. In that case there's no avoiding that you'll need to know whether that transaction committed or aborted. Now we could make a bet that most transactions commit and therefore we could keep a list of aborted transactions only which we might be able to keep "forever" in very little space if very few transactions abort. Presumably we would only use this form once the transaction was about to be truncated out of clog. I'm not too happy with the assumption that there aren't many aborts though. Someone could come along with a use case where they have lots of aborts and run into strange limitations and performance characteristics. Alternatively we could do something like keeping a list of tables touched by any transaction. Then vacuum could look for any non-committed transactions old enough to be in danger of aging out of clog and ensure those tables are frozen. But any tables which have never been touched by any such old transaction could be left alone. when we read in the page we'll be able to recognize the old transactions as committed if they're beyond the end of the clog horizon. I don't really like that idea either because it leaves performance really quite unpredictable. I could have a large table that goes unvacuumed for a long time -- then when I come along with some tiny query where I hit C-c and cause an abort I suddenly set a trap which causes a huge vacuum freeze to fire off. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers