Re: [9fans] 9pfuse and O_APPEND

2008-12-24 Thread erik quanstrom
> That's fair. But let me flip a question then, a bit: what do you all use > DMAPPEND for? What's are the examples of the most appropriate > usage for it in existing Plan9 software? i think log files are the cannonical use of append-only files. mbox style mailboxes also use append-only sematics fo

Re: [9fans] 9pfuse and O_APPEND

2008-12-24 Thread Roman Shaposhnik
On Dec 21, 2008, at 6:45 AM, erik quanstrom wrote: is your 9p server ever going to be running on an nfs-mounted partition? As with any software -- it would be pretty difficult for me to prevent somebody from doing that, but in general -- no. i use "in general" to mean the exact opposite of

Re: [9fans] 9pfuse and O_APPEND

2008-12-22 Thread roger peppe
On Sun, Dec 21, 2008 at 2:45 PM, erik quanstrom wrote: > okay, so you're using DMAPPEND like sbrk(2). how do you avoid > clients caring about the address of this new hunk of memory?^u > clients caring about the offset of this hunk of the file? > that is, the same problem malloc has in a multi-thr

Re: [9fans] 9pfuse and O_APPEND

2008-12-21 Thread erik quanstrom
> > is your 9p server ever going to be running on an nfs-mounted > > partition? > > As with any software -- it would be pretty difficult for me to prevent > somebody from doing that, but in general -- no. i use "in general" to mean the exact opposite of what you are saying here; there is a case

Re: [9fans] 9pfuse and O_APPEND

2008-12-20 Thread Roman V. Shaposhnik
On Fri, 2008-12-19 at 20:18 +, Charles Forsyth wrote: > >And nobody yet cared to give a concrete explanation of why it might be a bad > >idea. > > what's the application you've got in mind? Legacy ones :-( At the moment -- they are homegrown databases. And yes, as Erik pointed out -- they lo

Re: [9fans] 9pfuse and O_APPEND

2008-12-20 Thread Roman V. Shaposhnik
On Fri, 2008-12-19 at 13:00 -0800, ron minnich wrote: > > * emulate per-FID DMAPPEND by letting the server (which I also control) > > accept Qid > > modifications on wstat. My understanding is that existing 9P servers > > would simply > > reply with Rerror and I can then fallback onto l

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 1:32 PM, Charles Forsyth wrote: >> if that -1 would be seen on the wire > > no. it's just a flag to select the code path that provides the offset, > and entirely internal (just as well). > > I figured as much. Oh well. Sorry, Roman. ron

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Thu, Dec 18, 2008 at 3:34 PM, Roman Shaposhnik wrote: > > So far, I see the following choices for myself: > * follow what v9fs does and emulate it with llseek(...SEEK_END). Not > ideal, > since it doesn't always guarantee POSIX sematics, but way better > than nothing. and it won't

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Charles Forsyth
> if that -1 would be seen on the wire no. it's just a flag to select the code path that provides the offset, and entirely internal (just as well).

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> It is difficult to answer your question without knowing what rewrite > actually does and how mailboxes are structured. But in an imaginary > world where a mailbox is a list of constant sized blocks (size a bunch of simultaneous appends and rewrites of existing blocks > would work perfectly well.

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 11:56 AM, erik quanstrom wrote: Two questions: 1. But before I ask this one: I don't deny that per-file append- only is *extremely* useful. My question is a different one: what is the danger of N clients accesing the file X in append-only mode and M clients ac

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> And how is it different from what I was suggesting: A Fid that makes > *all* writes be at EOF? You want to write at EOF? Easy -- just use > that pre-negotiated Fid that was opened with (now non existent) > DMAPPEND flag added to the mode. You want a random-access > write AT THE SAME TIME? Easy --

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Charles Forsyth
>And nobody yet cared to give a concrete explanation of why it might be a bad >idea. what's the application you've got in mind?

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 11:49 AM, Roman Shaposhnik wrote: > Two questions: > 1. But before I ask this one: I don't deny that per-file append-only >is *extremely* useful. My question is a different one: what is >the danger of N clients accesing the file X in append-only mode >and M c

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 8:44 AM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:59 PM, Roman Shaposhnik wrote: On Dec 18, 2008, at 7:26 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: Its fun, yes. But I believe this is more of a testament to the statelessness o

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> Two questions: > 1. But before I ask this one: I don't deny that per-file append-only > is *extremely* useful. My question is a different one: what is > the danger of N clients accesing the file X in append-only mode > and M clients accesing it in random access mode? Could you,

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Roman Shaposhnik
On Dec 19, 2008, at 12:23 AM, Russ Cox wrote: Append-only and exclusive-use are properties of files and need to be enforced uniformly across all clients to be meaningful. They must be per-file, not per-fd. Two questions: 1. But before I ask this one: I don't deny that per-file append-only

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Fri, Dec 19, 2008 at 11:21 AM, Anthony Sorace wrote: >> client by definition knows more than the server. > > i assume you mean "knows less"? the server knows where EOF is and > which files to enforce append-only on. your #1 seems to only exist > because the client doesn't have that info. in st

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
On Fri Dec 19 14:24:52 EST 2008, ano...@gmail.com wrote: > > client by definition knows more than the server. > > i assume you mean "knows less"? the server knows where EOF is and > which files to enforce append-only on. your #1 seems to only exist > because the client doesn't have that info. i t

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Anthony Sorace
> client by definition knows more than the server. i assume you mean "knows less"? the server knows where EOF is and which files to enforce append-only on. your #1 seems to only exist because the client doesn't have that info.

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread ron minnich
On Thu, Dec 18, 2008 at 7:59 PM, Roman Shaposhnik wrote: > On Dec 18, 2008, at 7:26 PM, ron minnich wrote: >> >> On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: >>> >>> Its fun, yes. But I believe this is more of a testament to the >>> statelessness >>> of the NFS >>> plus the fact that

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread erik quanstrom
> The places that DMAPPEND is used most commonly are log files and mail > boxes. mailboxes are append only, however deleting a message requires rewriting the mailbox, which isn't possible. so a temporary mbox is written, has its mode tweaked and then replaces the mbox. L.mbox is exclusive open a

Re: [9fans] 9pfuse and O_APPEND

2008-12-19 Thread Russ Cox
Append-only and exclusive-use are properties of files and need to be enforced uniformly across all clients to be meaningful. They must be per-file, not per-fd. Russ

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread geoff
The places that DMAPPEND is used most commonly are log files and mail boxes. In both cases, I don't want the decision of whether to truncate or append left to the whims of some program. I want writes to append, by god, and DMAPPEND on actual disk-based file servers such as fossil and fs does that

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 7:26 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: Its fun, yes. But I believe this is more of a testament to the statelessness of the NFS plus the fact that the "end of file" is not a well defined offset (unlike beginning of the file).

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 7:43 PM, erik quanstrom wrote: Agreed. Now, here's a bit that I still don't quite understand: Plan9 does have DMAPPEND on a per-Qid basis. Why was it decided not to have it on a per-Fid basis (which would match POSIX semantics 100%)? The way I understand -- DMAPPEND is just a

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread erik quanstrom
> Agreed. Now, here's a bit that I still don't quite > understand: Plan9 does have DMAPPEND on > a per-Qid basis. Why was it decided not to > have it on a per-Fid basis (which would match > POSIX semantics 100%)? > > The way I understand -- DMAPPEND is just a hint > to the server to *alway* ignore

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread ron minnich
On Thu, Dec 18, 2008 at 7:06 PM, Roman Shaposhnik wrote: > Its fun, yes. But I believe this is more of a testament to the statelessness > of the NFS > plus the fact that the "end of file" is not a well defined offset (unlike > beginning of > the file). > no, it's even worse with stateful systems

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 4:03 PM, ron minnich wrote: On Thu, Dec 18, 2008 at 3:57 PM, Russ Cox wrote: I would just seek to the end. That's fine unless you have multiple programs writing O_APPEND simultaneously, in which case you are asking for trouble. yep. The code in nfs clients to support O_AP

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
On Dec 18, 2008, at 3:57 PM, Russ Cox wrote: I would just seek to the end. Got it. In that case, is there any reason the current version of 9pfuse doesn't just skip O_APPEND (like it does with O_LARGEFILE, etc.)? Since 9pfuse revalidate i_size before writes that's the best one can do anyway(*)

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread ron minnich
On Thu, Dec 18, 2008 at 3:57 PM, Russ Cox wrote: > I would just seek to the end. > That's fine unless you have multiple > programs writing O_APPEND simultaneously, > in which case you are asking for trouble. > yep. The code in nfs clients to support O_APPEND is a wonder to behold. A nicer combina

Re: [9fans] 9pfuse and O_APPEND

2008-12-18 Thread Russ Cox
I would just seek to the end. That's fine unless you have multiple programs writing O_APPEND simultaneously, in which case you are asking for trouble. Russ

[9fans] 9pfuse and O_APPEND

2008-12-18 Thread Roman Shaposhnik
I guess this is mainly a question for Russ: I'm using 9pfuse for a proof-of-concept project here at Sun and it all works quite well. My goal is to avoid the 9P2000.u route and use 9P2000 semantics as much as possible, yet allow most of the POSIX FS functionality to simply work. In order to do th