Re: [HACKERS] WAL & SHM principles

2001-03-19 Thread Martin Devera
> > When you mmap, you don't use write() ! mlock actualy locks page in > > memory and as long as the page is locked the OS doesn't attempt to > > store the dirty page. It is intended also for security app to > > ensure that sensitive data are not written to unsecure storage > > (hdd). It is defi

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Matthew Kirkwood
On Tue, 13 Mar 2001, Alfred Perlstein wrote: [..] > Linux does not filesystem-sync file-backed writable mmap pages on a > regular basis. Very intersting. I'm not sure that is necessarily the case in 2.4, though -- my understanding is that the new all-singing, all-dancing page cache makes very l

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Alfred Perlstein
* Matthew Kirkwood <[EMAIL PROTECTED]> [010313 13:12] wrote: > On Tue, 13 Mar 2001, Ken Hirsch wrote: > > > > mlock() guarantees that the locked address space is in memory. This > > > doesn't imply that updates are not written to the backing file. > > > > I've wondered about this myself. It _is

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Matthew Kirkwood
On Tue, 13 Mar 2001, Ken Hirsch wrote: > > mlock() guarantees that the locked address space is in memory. This > > doesn't imply that updates are not written to the backing file. > > I've wondered about this myself. It _is_ true on Linux that mlock > prevents writes to the backing store, I don

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Ken Hirsch
Giles Lean <[EMAIL PROTECTED]> wrote: > > When you mmap, you don't use write() ! mlock actualy locks page in > > memory and as long as the page is locked the OS doesn't attempt to > > store the dirty page. It is intended also for security app to > > ensure that sensitive data are not written to

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Ken Hirsch
Giles Lean <[EMAIL PROTECTED]> wrote: > > When you mmap, you don't use write() ! mlock actualy locks page in > > memory and as long as the page is locked the OS doesn't attempt to > > store the dirty page. It is intended also for security app to > > ensure that sensitive data are not written to

Re: [HACKERS] WAL & SHM principles

2001-03-13 Thread Giles Lean
> When you mmap, you don't use write() ! mlock actualy locks page in > memory and as long as the page is locked the OS doesn't attempt to > store the dirty page. It is intended also for security app to > ensure that sensitive data are not written to unsecure storage > (hdd). It is definition of

Re: [HACKERS] WAL & SHM principles

2001-03-12 Thread Nathan Myers
Sorry for taking so long to reply... On Wed, Mar 07, 2001 at 01:27:34PM -0800, Mikheev, Vadim wrote: > Nathan wrote: > > It is possible to build a logging system so that you mostly don't care > > when the data blocks get written [after being changed, as long as they get written by an fsync];

RE: [HACKERS] WAL & SHM principles

2001-03-12 Thread Mikheev, Vadim
>> > It is possible to build a logging system so that you >> > mostly don't care when the data blocks get written; >> > a particular data block on disk is considered garbage >> > until the next checkpoint, so that you >> > > > How to know if a particular data page was modified if there is no > >

Re: [HACKERS] WAL & SHM principles

2001-03-12 Thread Kevin T. Manley \(Home\)
""Mikheev, Vadim"" <[EMAIL PROTECTED]> wrote in message [EMAIL PROTECTED]">news:[EMAIL PROTECTED]... > > It is possible to build a logging system so that you mostly don't care > > when the data blocks get written; a particular data block on disk is > > considered garbage until the next checkpoint

RE: [HACKERS] WAL & SHM principles

2001-03-09 Thread Mikheev, Vadim
> > But needed if we want to get rid of vacuum and have savepoints. > > Hmm. How do you implement savepoints ? When there is rollback > to savepoint do you use xlog to undo all changes which the particular > transaction has done ? Hmmm it seems nice ... these resords are locked by > such trans

RE: [HACKERS] WAL & SHM principles

2001-03-09 Thread Martin Devera
> > Pros: upper layers can think thet buffers are always safe/logged and > > there is no special handling for indices; very simple/fast redo > > Cons: can't implement undo - but in non-overwriting is not needed (?) > > But needed if we want to get rid of vacuum and have savepoints. Hmm. Ho

Re: [HACKERS] WAL & SHM principles

2001-03-09 Thread Martin Devera
> > BTW, what means "bummer" ? > > Sorry, it means, "Oh, I am disappointed." thanks :) > > But for many OSes you CAN control when to write data - you can mlock > > individual pages. > > mlock() controls locking in physical memory. I don't see it controling > write(). When you mmap, you don't

Re: [HACKERS] WAL & SHM principles

2001-03-08 Thread Bruce Momjian
> > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > > The only problem is that we would no longer have control over which > > > > pages made it to disk. The OS would perhaps write pages as we modified > > > > them. Not sure how important that is. > > > > > > Unfortunately, this alone is a *fa

Re: [HACKERS] WAL & SHM principles

2001-03-08 Thread Tim Allen
On Thu, 8 Mar 2001, Martin Devera wrote: > > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > Unfortunately, this alone is a *fatal* objection. See nearby > > > discussions about WAL behavior: we must be able to control the relative > > > timing of WAL write/flush and data page writes. > > > >

Re: [HACKERS] WAL & SHM principles

2001-03-08 Thread Martin Devera
> > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > The only problem is that we would no longer have control over which > > > pages made it to disk. The OS would perhaps write pages as we modified > > > them. Not sure how important that is. > > > > Unfortunately, this alone is a *fatal* objecti

RE: [HACKERS] WAL & SHM principles

2001-03-07 Thread Mikheev, Vadim
> 1) WAL > We have buffer manager, ok. So why not to use WAL as part of > it and don't log INSERT/UPDATE/DELETE xlog records but directly > changes into buffer pages ? When someone dirties page it has to > inform bmgr about dirty region and bmgr would formulate xlog record. > The record could be

RE: [HACKERS] WAL & SHM principles

2001-03-07 Thread Mikheev, Vadim
> It is possible to build a logging system so that you mostly don't care > when the data blocks get written; a particular data block on disk is > considered garbage until the next checkpoint, so that you How to know if a particular data page was modified if there is no log record for that modif

Re: [HACKERS] WAL & SHM principles

2001-03-07 Thread Nathan Myers
On Wed, Mar 07, 2001 at 11:21:37AM -0500, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > The only problem is that we would no longer have control over which > > pages made it to disk. The OS would perhaps write pages as we modified > > them. Not sure how important that is. > >

Re: [HACKERS] WAL & SHM principles

2001-03-07 Thread Bruce Momjian
> Bruce Momjian <[EMAIL PROTECTED]> writes: > > The only problem is that we would no longer have control over which > > pages made it to disk. The OS would perhaps write pages as we modified > > them. Not sure how important that is. > > Unfortunately, this alone is a *fatal* objection. See nea

Re: [HACKERS] WAL & SHM principles

2001-03-07 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > The only problem is that we would no longer have control over which > pages made it to disk. The OS would perhaps write pages as we modified > them. Not sure how important that is. Unfortunately, this alone is a *fatal* objection. See nearby discussi

Re: [HACKERS] WAL & SHM principles

2001-03-07 Thread Martin Devera
> This was brought up a week ago, and I consider it an interesting idea. > The only problem is that we would no longer have control over which > pages made it to disk. The OS would perhaps write pages as we modified > them. Not sure how important that is. Yes. As I work on linux kernel I know

Re: [HACKERS] WAL & SHM principles

2001-03-07 Thread Bruce Momjian
> 2) SHM vs. MMAP > Why don't use mmap to share pages (instead of shm) ? There would be no > problem with tuning pg's buffer cache size - it is balanced by OS. > When using SHM there are often two copies of page: one in OS' page cache > and one in SHM (vaste of memory). > When using mmap the data