Re: [9fans] 9pfuse and O_APPEND

2008-12-24 Thread erik quanstrom
> That's fair. But let me flip a question then, a bit: what do you all use > DMAPPEND for? What's are the examples of the most appropriate > usage for it in existing Plan9 software? i think log files are the cannonical use of append-only files. mbox style mailboxes also use append-only sematics fo

Re: [9fans] 9pfuse and O_APPEND

2008-12-24 Thread Roman Shaposhnik
On Dec 21, 2008, at 6:45 AM, erik quanstrom wrote: is your 9p server ever going to be running on an nfs-mounted partition? As with any software -- it would be pretty difficult for me to prevent somebody from doing that, but in general -- no. i use "in general" to mean the exact opposite of

Re: [9fans] 9pfuse and O_APPEND

2008-12-22 Thread roger peppe
On Sun, Dec 21, 2008 at 2:45 PM, erik quanstrom wrote: > okay, so you're using DMAPPEND like sbrk(2). how do you avoid > clients caring about the address of this new hunk of memory?^u > clients caring about the offset of this hunk of the file? > that is, the same problem malloc has in a multi-thr

Re: [9fans] 9pfuse and O_APPEND

2008-12-21 Thread erik quanstrom
> > is your 9p server ever going to be running on an nfs-mounted > > partition? > > As with any software -- it would be pretty difficult for me to prevent > somebody from doing that, but in general -- no. i use "in general" to mean the exact opposite of what you are saying here; there is a case

Re: [9fans] 9pfuse and O_APPEND

2008-12-20 Thread Roman V. Shaposhnik
On Fri, 2008-12-19 at 20:18 +, Charles Forsyth wrote: > >And nobody yet cared to give a concrete explanation of why it might be a bad > >idea. > > what's the application you've got in mind? Legacy ones :-( At the moment -- they are homegrown databases. And yes, as Erik pointed out -- they lo

Re: [9fans] 9pfuse and O_APPEND

2008-12-20 Thread Roman V. Shaposhnik
On Fri, 2008-12-19 at 13:00 -0800, ron minnich wrote: > > * emulate per-FID DMAPPEND by letting the server (which I also control) > > accept Qid > > modifications on wstat. My understanding is that existing 9P servers > > would simply > > reply with Rerror and I can then fallback onto l

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 1:32 PM, Charles Forsyth wrote: >> if that -1 would be seen on the wire > > no. it's just a flag to select the code path that provides the offset, > and entirely internal (just as well). > > I figured as much. Oh well. Sorry, Roman. ron

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Thu, Dec 18, 2008 at 3:34 PM, Roman Shaposhnik wrote: > > So far, I see the following choices for myself: > * follow what v9fs does and emulate it with llseek(...SEEK_END). Not > ideal, > since it doesn't always guarantee POSIX sematics, but way better > than nothing. and it won't

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Charles Forsyth
> if that -1 would be seen on the wire no. it's just a flag to select the code path that provides the offset, and entirely internal (just as well).

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> It is difficult to answer your question without knowing what rewrite > actually does and how mailboxes are structured. But in an imaginary > world where a mailbox is a list of constant sized blocks (size a bunch of simultaneous appends and rewrites of existing blocks > would work perfectly well.

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 11:56 AM, erik quanstrom wrote: Two questions: 1. But before I ask this one: I don't deny that per-file append- only is *extremely* useful. My question is a different one: what is the danger of N clients accesing the file X in append-only mode and M clients ac

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> And how is it different from what I was suggesting: A Fid that makes > *all* writes be at EOF? You want to write at EOF? Easy -- just use > that pre-negotiated Fid that was opened with (now non existent) > DMAPPEND flag added to the mode. You want a random-access > write AT THE SAME TIME? Easy --

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Charles Forsyth
>And nobody yet cared to give a concrete explanation of why it might be a bad >idea. what's the application you've got in mind?

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 11:49 AM, Roman Shaposhnik wrote: > Two questions: > 1. But before I ask this one: I don't deny that per-file append-only >is *extremely* useful. My question is a different one: what is >the danger of N clients accesing the file X in append-only mode >and M c

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 8:44 AM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:59 PM, Roman Shaposhnik wrote: On Dec 18, 2008, at 7:26 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: Its fun, yes. But I believe this is more of a testament to the statelessness o

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> Two questions: > 1. But before I ask this one: I don't deny that per-file append-only > is *extremely* useful. My question is a different one: what is > the danger of N clients accesing the file X in append-only mode > and M clients accesing it in random access mode? Could you,

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 12:23 AM, Russ Cox wrote: Append-only and exclusive-use are properties of files and need to be enforced uniformly across all clients to be meaningful. They must be per-file, not per-fd. Two questions: 1. But before I ask this one: I don't deny that per-file append-only

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 11:21 AM, Anthony Sorace wrote: >> client by definition knows more than the server. > > i assume you mean "knows less"? the server knows where EOF is and > which files to enforce append-only on. your #1 seems to only exist > because the client doesn't have that info. in st

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
On Fri Dec 19 14:24:52 EST 2008, ano...@gmail.com wrote: > > client by definition knows more than the server. > > i assume you mean "knows less"? the server knows where EOF is and > which files to enforce append-only on. your #1 seems to only exist > because the client doesn't have that info. i t

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Anthony Sorace
> client by definition knows more than the server. i assume you mean "knows less"? the server knows where EOF is and which files to enforce append-only on. your #1 seems to only exist because the client doesn't have that info.

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Thu, Dec 18, 2008 at 7:59 PM, Roman Shaposhnik wrote: > On Dec 18, 2008, at 7:26 PM, ron minnich wrote: >> >> On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: >>> >>> Its fun, yes. But I believe this is more of a testament to the >>> statelessness >>> of the NFS >>> plus the fact that

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> The places that DMAPPEND is used most commonly are log files and mail > boxes. mailboxes are append only, however deleting a message requires rewriting the mailbox, which isn't possible. so a temporary mbox is written, has its mode tweaked and then replaces the mbox. L.mbox is exclusive open a

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Russ Cox
Append-only and exclusive-use are properties of files and need to be enforced uniformly across all clients to be meaningful. They must be per-file, not per-fd. Russ

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread geoff
The places that DMAPPEND is used most commonly are log files and mail boxes. In both cases, I don't want the decision of whether to truncate or append left to the whims of some program. I want writes to append, by god, and DMAPPEND on actual disk-based file servers such as fossil and fs does that

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 7:26 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: Its fun, yes. But I believe this is more of a testament to the statelessness of the NFS plus the fact that the "end of file" is not a well defined offset (unlike beginning of the file).

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 7:43 PM, erik quanstrom wrote: Agreed. Now, here's a bit that I still don't quite understand: Plan9 does have DMAPPEND on a per-Qid basis. Why was it decided not to have it on a per-Fid basis (which would match POSIX semantics 100%)? The way I understand -- DMAPPEND is just a

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread erik quanstrom
> Agreed. Now, here's a bit that I still don't quite > understand: Plan9 does have DMAPPEND on > a per-Qid basis. Why was it decided not to > have it on a per-Fid basis (which would match > POSIX semantics 100%)? > > The way I understand -- DMAPPEND is just a hint > to the server to *alway* ignore

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread ron minnich
On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: > Its fun, yes. But I believe this is more of a testament to the statelessness > of the NFS > plus the fact that the "end of file" is not a well defined offset (unlike > beginning of > the file). > no, it's even worse with stateful systems

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 4:03 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 3:57 PM, Russ Cox wrote: I would just seek to the end. That's fine unless you have multiple programs writing O_APPEND simultaneously, in which case you are asking for trouble. yep. The code in nfs clients to support O_AP

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 3:57 PM, Russ Cox wrote: I would just seek to the end. Got it. In that case, is there any reason the current version of 9pfuse doesn't just skip O_APPEND (like it does with O_LARGEFILE, etc.)? Since 9pfuse revalidate i_size before writes that's the best one can do anyway(*)

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread ron minnich
On Thu, Dec 18, 2008 at 3:57 PM, Russ Cox wrote: > I would just seek to the end. > That's fine unless you have multiple > programs writing O_APPEND simultaneously, > in which case you are asking for trouble. > yep. The code in nfs clients to support O_APPEND is a wonder to behold. A nicer combina

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Russ Cox
I would just seek to the end. That's fine unless you have multiple programs writing O_APPEND simultaneously, in which case you are asking for trouble. Russ