AW: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Zeugswetter Andreas SB
> > > It will be sufficient if DB will not use all 2^32 XIDs > > > without shutdown. > > > > I liked the xid wraparound idea, won't that be sufficient here too ? > > I don't like the idea to reuse a xid sooner than absolutely necessary. > > We need it to reduce pg_log size requirements. Yes,

Re: [HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Andrew McMillan
Tom Lane wrote: > > "Mikheev, Vadim" <[EMAIL PROTECTED]> writes: > >> That'll be true in any case, unless we refuse to start up at all upon > >> detecting xlog corruption (which doesn't seem like the way to fly). > >> Not sure what we can do about that. > > > What I would refuse in the event of

Re: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Hiroshi Inoue
"Mikheev, Vadim" wrote: > > > >> What I was thinking about in that last paragraph was manual > > >> analysis and recovery. I don't think it's a good idea for automatic > > >> system startup to skip over gaps in the log. > > > > > But if we'll not try to read after gap then after restart > > > syst

Re: [HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Justin Clift
Maybe there should be an error message like : "PostgreSQL has detected severe xlog corruption. Please fix this with pg_recover (or similar) manually before restarting the database"? Guess I'm suggesting a separate xlog recovery tool for "bad cases" of xlog corruption, so decisions can by manual

[HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Thomas Lockhart
> >> That'll be true in any case, unless we refuse to start up at all upon > >> detecting xlog corruption (which doesn't seem like the way to fly). > >> Not sure what we can do about that. > > What I would refuse in the event of log corruption is continuing > > normal database operations. > Hmm.

[HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Mikheev, Vadim
> > What I would refuse in the event of log corruption is continuing > > normal database operations. > > Hmm. We could do that if we had some notion of a read-only operating > mode, perhaps. But we don't have one now and I don't want to add it > for 7.1. Can we agree to look at this more for 7

[HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Tom Lane
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes: >> That'll be true in any case, unless we refuse to start up at all upon >> detecting xlog corruption (which doesn't seem like the way to fly). >> Not sure what we can do about that. > What I would refuse in the event of log corruption is continuing >

RE: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Mikheev, Vadim
> > It will be sufficient if DB will not use all 2^32 XIDs > > without shutdown. > > I liked the xid wraparound idea, won't that be sufficient here too ? > I don't like the idea to reuse a xid sooner than absolutely necessary. We need it to reduce pg_log size requirements. > This would complic

[HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Mikheev, Vadim
> >> What I was thinking about in that last paragraph was manual > >> analysis and recovery. I don't think it's a good idea for automatic > >> system startup to skip over gaps in the log. > > > But if we'll not try to read after gap then after restart > > system will not notice gap and valid re

[HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Tom Lane
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes: >> What I was thinking about in that last paragraph was manual >> analysis and recovery. I don't think it's a good idea for automatic >> system startup to skip over gaps in the log. > But if we'll not try to read after gap then after restart system wi

[HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Mikheev, Vadim
> >> xlog.c's ReadRecord formerly contained code that would zero > >> out the rest of the log segment (and delete the next log segment, > >> if any) upon detecting a missing or corrupted xlog record. > >> I removed that code because I considered it horribly dangerous > >> where it was. If there i

[HACKERS] Re: xlog loose ends, continued

2001-03-13 Thread Tom Lane
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes: >> That doesn't seem like a good idea --- once we fail to read an XLOG >> record, it's probably best to stop there rather than continue on. >> I think we want to try for a consistent recovery to a past point in >> time (ie, wherever the xlog gap is) not

AW: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Zeugswetter Andreas SB
> > > StartUpID counts database startups and so has timestamp characteristics. > > > Actually, idea is to use SUI in future to allow reusing XIDs after startup: >seeing > > > old SUI in data pages we'll know that all transaction on this page was committed > > > "long ago" (ie visible from MVCC P

Re: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Vadim Mikheev
> > StartUpID counts database startups and so has timestamp characteristics. > > Actually, idea is to use SUI in future to allow reusing XIDs after startup: seeing > > old SUI in data pages we'll know that all transaction on this page was committed > > "long ago" (ie visible from MVCC POV). This r

Re: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Vadim Mikheev
> > Anyway I like idea of StartUpID in page headers - this will help > > Can you please describe StartUpID for me ? > Ideal would be a stamp that has the last (smallest open) XID, or something else > that has more or less timestamp characteristics (without the actual need of >wallclock) > in r

AW: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Zeugswetter Andreas SB
> > > Anyway I like idea of StartUpID in page headers - this will help > > > > Can you please describe StartUpID for me ? > > Ideal would be a stamp that has the last (smallest open) XID, or something else > > that has more or less timestamp characteristics (without the actual need of >wallcl

AW: [HACKERS] RE: xlog loose ends, continued

2001-03-13 Thread Zeugswetter Andreas SB
> Anyway I like idea of StartUpID in page headers - this will help Can you please describe StartUpID for me ? Ideal would be a stamp that has the last (smallest open) XID, or something else that has more or less timestamp characteristics (without the actual need of wallclock) in regard to the

[HACKERS] RE: xlog loose ends, continued

2001-03-12 Thread Mikheev, Vadim
> > Should we try to read log up to the *physical end* - ie end of last > > log file - regardless invalid CRC-s/zero pages with attempt to > > re-apply interim valid records? (Or do we already do this?) > > That doesn't seem like a good idea --- once we fail to read an XLOG > record, it's probabl

[HACKERS] Re: xlog loose ends, continued

2001-03-12 Thread Tom Lane
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes: > This code was from the old days when there was no CRC in log records. Ah, right. The CRC makes things safer ... but there's still a risk that old log pages could look like a valid continuation. > Should we try to read log up to the *physical end* -

[HACKERS] RE: xlog loose ends, continued

2001-03-12 Thread Mikheev, Vadim
> I presume the point of this code was that if we recover and > then suffer > a later crash at a point where we've just written an xlog record that > exactly fills an xlog page, a subsequent scan of the log > might continue > on from that point and pick up xlog records from the prior (failed) >