Re: light weight user level semaphores

2001-04-23 Thread Alexander Viro
On 24 Apr 2001, David Wagner wrote: > Linus Torvalds wrote: > >Ehh.. I will bet you $10 USD that if libc allocates the next file > >descriptor on the first "malloc()" in user space (in order to use the > >semaphores for mm protection), programs _will_ break. > > > >You want to take the bet? >

Re: light weight user level semaphores

2001-04-23 Thread David Wagner
Linus Torvalds wrote: >Ehh.. I will bet you $10 USD that if libc allocates the next file >descriptor on the first "malloc()" in user space (in order to use the >semaphores for mm protection), programs _will_ break. > >You want to take the bet? Good point. Speaking of which: ioctl(fd, UIOCATTA

Re: light weight user level semaphores

2001-04-23 Thread Ingo Oeser
On Thu, Apr 19, 2001 at 09:46:17AM -0700, Linus Torvalds wrote: > > libc is entitled to, and most definitely does exactly that. Take a look at > > things like gethostent, getpwent etc etc. > > Ehh.. I will bet you $10 USD that if libc allocates the next file > descriptor on the first "malloc()" i

Re: light weight user level semaphores

2001-04-23 Thread David Howells
Alon Ziv <[EMAIL PROTECTED]> wrote: > Obviously... since they're handles, not FDs... > [BTW, are you using Windows' idea of storing the objects in process space, > in a page that's inaccessible to the app itself, and passing pointers into > this page as the handles?] No... I grab a page in kernel

Re: light weight user level semaphores

2001-04-23 Thread Jeff Garzik
Linus Torvalds wrote: > Trust me. You're arguing for clearly broken behaviour. malloc() and > friends MUST NOT open file descriptors. It _will_ break programs that > rely on traditional and documented features. Indeed; STDIN_FILENO and friends are constants... -- Jeff Garzik | The differen

Re: light weight user level semaphores

2001-04-23 Thread Alon Ziv
From: "David Howells" <[EMAIL PROTECTED]> > David Woodhouse <[EMAIL PROTECTED]> wrote: > > [EMAIL PROTECTED] said: > > > [BTW, another solution is to truly support opaque "handles" to kernel > > > objects; I believe David Howells is already working on something like > > > this for Wine? > > Yes.

Re: light weight user level semaphores

2001-04-23 Thread David Howells
David Woodhouse <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] said: > > [BTW, another solution is to truly support opaque "handles" to kernel > > objects; I believe David Howells is already working on something like > > this for Wine? Yes. However, it uses a different system call set to use the

Re: light weight user level semaphores

2001-04-22 Thread Alon Ziv
n FD, so they can be 'read' out of it... with that, an alarm() is a 'timed-wait' waitable by poll() :-) ] -az - Original Message - From: "Alexander Viro" <[EMAIL PROTECTED]> To: "Alon Ziv" <[EMAIL PROTECTED]> Cc: <[EMAIL P

Re: light weight user level semaphores

2001-04-22 Thread Alexander Viro
On Sun, 22 Apr 2001, Alon Ziv wrote: > Well, that's the reason for my small-negative-integer semaphore-FD idea... > (It won't support select() easily, but poll() is prob'ly good enough) > Still, there is the problem of read()/write()/etc. semantics; sure, we can > declare that 'negative FDs' ha

Re: light weight user level semaphores

2001-04-22 Thread Alon Ziv
tics which just happen to include poll(), but it sure looks like a kludge... -az - Original Message - From: "Alan Cox" <[EMAIL PROTECTED]> To: "Alon Ziv" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, April 22, 2001 14:44 Subject: Re: lig

Re: light weight user level semaphores

2001-04-22 Thread David Woodhouse
[EMAIL PROTECTED] said: > [BTW, another solution is to truly support opaque "handles" to kernel > objects; I believe David Howells is already working on something like > this for Wine? The poll interface can be trivially extended to support > waiting on those...] ISTR it wasn't quite trivial to

Re: light weight user level semaphores

2001-04-21 Thread Olaf Titz
> This is guaranteed behaviour of UNIX. You get file handles in order, or > you don't get them at all. You get the _next free_ file handle in order. What if your program assumes they are all contiguous, and it is called by some other program which forgot about FD_CLOEXEC and has some higher fds s

Re: light weight user level semaphores

2001-04-20 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Olaf Titz <[EMAIL PROTECTED]> wrote: >> Ehh.. I will bet you $10 USD that if libc allocates the next file >> descriptor on the first "malloc()" in user space (in order to use the >> semaphores for mm protection), programs _will_ break. > >Of course, but this is a r

Re: light weight user level semaphores

2001-04-20 Thread Olaf Titz
> Optimization use in select: If all "interesting" file id's are known > to be below "n", then only the first "n" bits in a FD_ISSET need to > be examined. As soon as the bits are scattered, it takes MUCH longer > to check for activity That's an optimization, not a correctness issue. > f

Re: light weight user level semaphores

2001-04-20 Thread Jesse Pollard
Olaf Titz <[EMAIL PROTECTED]>: > > Ehh.. I will bet you $10 USD that if libc allocates the next file > > descriptor on the first "malloc()" in user space (in order to use the > > semaphores for mm protection), programs _will_ break. > > Of course, but this is a result from sloppy coding. In gener

Re: light weight user level semaphores

2001-04-20 Thread Olaf Titz
> Ehh.. I will bet you $10 USD that if libc allocates the next file > descriptor on the first "malloc()" in user space (in order to use the > semaphores for mm protection), programs _will_ break. Of course, but this is a result from sloppy coding. In general, open() can just return anything and a

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Alexander Viro <[EMAIL PROTECTED]> writes: > > If the new interface can be useful for anything it must allow to > > implement process-shared POSIX mutexes. > > Pardon me the bluntness, but... Why? Because otherwise there is no reason to even waste a second with this. At least for me and everybo

Re: light weight user level semaphores

2001-04-19 Thread Alexander Viro
On 19 Apr 2001, Ulrich Drepper wrote: > Linus Torvalds <[EMAIL PROTECTED]> writes: > > > I'm not interested in re-creating the idiocies of Sys IPC. > > I'm not talking about sysv semaphores (couldn't care less). And you > haven't read any of the mails with examples I sent. > > If the new in

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Linus Torvalds <[EMAIL PROTECTED]> writes: > I'm not interested in re-creating the idiocies of Sys IPC. I'm not talking about sysv semaphores (couldn't care less). And you haven't read any of the mails with examples I sent. If the new interface can be useful for anything it must allow to imple

Re: light weight user level semaphores

2001-04-19 Thread Rogier Wolff
Alan Cox wrote: > > > libc is entitled to, and most definitely does exactly that. Take a look at > > > things like gethostent, getpwent etc etc. > > > > Ehh.. I will bet you $10 USD that if libc allocates the next file > > descriptor on the first "malloc()" in user space (in order to use the > >

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On 19 Apr 2001, Ulrich Drepper wrote: > Linus Torvalds <[EMAIL PROTECTED]> writes: > > > > I fail to see how this works across processes. > > > > It's up to FS_create() to create whatever shared mapping is needed. > > No, the point is that FS_create is *not* the one creating the shared > mapping

Re: light weight user level semaphores

2001-04-19 Thread Alan Cox
> Are you sure, you can implement SMP-safe, atomic operations (which you need > for all up()/down() in user space) WITHOUT using privileged > instructions on ALL archs Linux supports? You don't need to. For some architectures the semaphore code would always call into the kernel. For those that al

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Linus Torvalds <[EMAIL PROTECTED]> writes: > > I fail to see how this works across processes. > > It's up to FS_create() to create whatever shared mapping is needed. No, the point is that FS_create is *not* the one creating the shared mapping. The user is explicitly doing this her/himself. --

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Ingo Oeser wrote: > On Thu, Apr 19, 2001 at 09:11:56AM -0700, Linus Torvalds wrote: > > No, this is NOT what the UNIX dogmas are all about. > > > > When UNIX says "everything is a file", it really means that "everything is > > a stream of bytes". Things like magic operation

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Ingo Oeser wrote: > > Are you sure, you can implement SMP-safe, atomic operations (which you need > for all up()/down() in user space) WITHOUT using privileged > instructions on ALL archs Linux supports? Why do you care? Sure, there are broken architectures out there. They

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On 19 Apr 2001, Ulrich Drepper wrote: > Linus Torvalds <[EMAIL PROTECTED]> writes: > > > Looks good to me. Anybody want to try this out and test some benchmarks? > > I fail to see how this works across processes. It's up to FS_create() to create whatever shared mapping is needed. For threads,

Re: light weight user level semaphores

2001-04-19 Thread Ingo Oeser
On Thu, Apr 19, 2001 at 09:11:56AM -0700, Linus Torvalds wrote: > No, this is NOT what the UNIX dogmas are all about. > > When UNIX says "everything is a file", it really means that "everything is > a stream of bytes". Things like magic operations on file desciptors are > _anathema_ to UNIX. ioct

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Ingo Oeser <[EMAIL PROTECTED]> writes: > Are you sure, you can implement SMP-safe, atomic operations (which you need > for all up()/down() in user space) WITHOUT using privileged > instructions on ALL archs Linux supports? Which processors have no such instructions but are SMP-capable? > How do

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Alan Cox <[EMAIL PROTECTED]> writes: > > I don't want nor need file permissions. A program would look like this: > > Your example opens/mmaps so has file permissions. Which is what I was asking There are no permissions on the mutex object. It is the shared memory which counts. If you would i

Re: light weight user level semaphores

2001-04-19 Thread Ingo Oeser
On Thu, Apr 19, 2001 at 12:26:03PM -0700, Ulrich Drepper wrote: > In any case all kinds of user-level operations are possible as well > and all the schemes suggested for dealing with the common case without > syscalls can be applied here as well. Are you sure, you can implement SMP-safe, atomic o

Re: light weight user level semaphores

2001-04-19 Thread Alan Cox
> I don't want nor need file permissions. A program would look like this: Your example opens/mmaps so has file permissions. Which is what I was asking > The shared mem segment can be retrieved in whatever way. The mutex in > this case is anonymous. Everybody who has access to the shared mem >

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Alan Cox <[EMAIL PROTECTED]> writes: > mknod foo p. Or use sockets (although AF_UNIX sockets are higher latency) > Thats why I suggested using flock - its name based. Whether you mkstemp() > stuff and pass it around isnt something I care about > > Files give you permissions for free too I don't

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Alan Cox <[EMAIL PROTECTED]> writes: > > can libraries use fast semaphores behind the back of the user? They might > > well want to use the semaphores exactly for things like memory allocator > > locking etc. But libc certainly cant use fd's behind peoples backs. > > libc is entitled to, and mos

Re: light weight user level semaphores

2001-04-19 Thread Alan Cox
> I fail to see how this works across processes. How can you generate a > file descriptor for this pipe in a second process which simply shares > some memory with the first one? The first process is passive: no file > descriptor passing must be necessary. mknod foo p. Or use sockets (although A

Re: light weight user level semaphores

2001-04-19 Thread Ulrich Drepper
Linus Torvalds <[EMAIL PROTECTED]> writes: > Looks good to me. Anybody want to try this out and test some benchmarks? I fail to see how this works across processes. How can you generate a file descriptor for this pipe in a second process which simply shares some memory with the first one? The

Re: light weight user level semaphores

2001-04-19 Thread Olaf Titz
> problems: just _how_ high woul dyou move it? Would it potentially disturb > an application that opens thousands of files, and knows that they get > consecutive file descriptors? Which is _legal_ and well-defined in UNIX. Only if you close them before. The process may have been started with arbi

Re: light weight user level semaphores

2001-04-19 Thread Alexander Viro
On Thu, 19 Apr 2001, Linus Torvalds wrote: > > > On Thu, 19 Apr 2001, Alexander Viro wrote: > > > > Ehh... Non-lazy variant is just read() and write() as down_failed() and > > up_wakeup() Lazy... How about > > Looks good to me. Anybody want to try this out and test some benchmarks? Ugh. It

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Alexander Viro wrote: > > Ehh... Non-lazy variant is just read() and write() as down_failed() and > up_wakeup() Lazy... How about Looks good to me. Anybody want to try this out and test some benchmarks? There may be problems with large numbers of semaphores, but hopefully

Re: light weight user level semaphores

2001-04-19 Thread Alexander Viro
On Thu, 19 Apr 2001, Linus Torvalds wrote: > > > On Thu, 19 Apr 2001, Alexander Viro wrote: > > > > I certainly agree that introducing ioctl() in _any_ API is a shootable > > offense. However, I wonder whether we really need any kernel changes > > at all. > > I'd certainly be interested in s

Re: light weight user level semaphores

2001-04-19 Thread Alan Cox
> > libc is entitled to, and most definitely does exactly that. Take a look at > > things like gethostent, getpwent etc etc. > > Ehh.. I will bet you $10 USD that if libc allocates the next file > descriptor on the first "malloc()" in user space (in order to use the > semaphores for mm protection

Re: light weight user level semaphores

2001-04-19 Thread Abramo Bagnara
Linus Torvalds wrote: > > On Thu, 19 Apr 2001, Abramo Bagnara wrote: > > > > > [ Using file descriptors ] > > > > This would also permit: > > - to have poll() > > - to use mmap() to obtain the userspace area > > > > It would become something very near to sacred Unix dogmas ;-) > > No, this is NO

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Alan Cox wrote: > > can libraries use fast semaphores behind the back of the user? They might > > well want to use the semaphores exactly for things like memory allocator > > locking etc. But libc certainly cant use fd's behind peoples backs. > > libc is entitled to, and mos

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Abramo Bagnara wrote: > > > [ Using file descriptors ] > > This would also permit: > - to have poll() > - to use mmap() to obtain the userspace area > > It would become something very near to sacred Unix dogmas ;-) No, this is NOT what the UNIX dogmas are all about. When U

Re: light weight user level semaphores

2001-04-19 Thread Linus Torvalds
On Thu, 19 Apr 2001, Alon Ziv wrote: > > * the userspace struct was just a signed count and a file handle. The main reason I wanted to avoid a filehandle is just because it's another name space that people already use, and that people know what the semantics are for (ie "open()" is _defined_ to

Re: light weight user level semaphores

2001-04-19 Thread George Talbot
On Tue, 17 Apr 2001 12:48:48 -0700 (PDT) Linus Torvalds wrote: [deletia] > /* > * a fast semaphore is a 128-byte opaque thing, > * aligned on a 128-byte boundary. This is partly > * to minimize false sharing in the L1 (we assume > * that 128-byte cache

Re: light weight user level semaphores

2001-04-19 Thread Alan Cox
> My lightweight-semaphores were actually even simpler in userspace: > * the userspace struct was just a signed count and a file handle. > * Uncontended case is exactly like Linus' version (i.e., down() is decl + > js, up() is incl()). > * The contention syscall was (in my implementation) an ioctl

Re: light weight user level semaphores

2001-04-19 Thread Abramo Bagnara
Alexander Viro wrote: > > I suspect that simple pipe with would be sufficient to handle contention > case - nothing fancy needed (read when you need to block, write upon up() > when you have contenders) > > Would something along the lines of (inline as needed, etc.) > > down: > lock dec

Re: light weight user level semaphores

2001-04-19 Thread Ingo Oeser
On Thu, Apr 19, 2001 at 10:20:48AM +0200, Alon Ziv wrote: > My lightweight-semaphores were actually even simpler in userspace: > * the userspace struct was just a signed count and a file handle. > * Uncontended case is exactly like Linus' version (i.e., down() is decl + > js, up() is incl()). > *

Re: light weight user level semaphores

2001-04-19 Thread Alexander Viro
On Thu, 19 Apr 2001, Abramo Bagnara wrote: > Alon Ziv wrote: > > > > Hmm... > > I already started (long ago, and abandoned since due to lack of time :-( ) > > down another path; I'd like to resurrect it... > > > > My lightweight-semaphores were actually even simpler in userspace: > > * the us

Re: light weight user level semaphores

2001-04-19 Thread Abramo Bagnara
Alon Ziv wrote: > > Hmm... > I already started (long ago, and abandoned since due to lack of time :-( ) > down another path; I'd like to resurrect it... > > My lightweight-semaphores were actually even simpler in userspace: > * the userspace struct was just a signed count and a file handle. > *

Re: light weight user level semaphores

2001-04-19 Thread Alon Ziv
L PROTECTED]> To: "Linus Torvalds" <[EMAIL PROTECTED]> Cc: "Mike Kravetz" <[EMAIL PROTECTED]>; "Kernel Mailing List" <[EMAIL PROTECTED]> Sent: Wednesday, April 18, 2001 21:35 Subject: Re: light weight user level semaphores > Linus Torvalds &l

Re: light weight user level semaphores

2001-04-18 Thread Ulrich Drepper
Linus Torvalds <[EMAIL PROTECTED]> writes: Sounds good so far. Some comments. > - FS_create is responsible for allocating a shared memory region >at "FS_create()" time. This is not so great. The POSIX shared semaphores require that an pthread_mutex_t object placed in a shared memory regi

Re: light weight user level semaphores

2001-04-18 Thread Bernd Eckenfels
In article <[EMAIL PROTECTED]> you wrote: > So FS_create() starts out by allocating the backing store for the > semaphore. This can basically be done in user space, although the > kernel does need to get involved for the second part of it, which > is to (a) allocate a kernel "backing store

Re: light weight user level semaphores

2001-04-18 Thread Linus Torvalds
[ Cc'd to linux-kernel, to get feedback etc. I've already talked this over with some people a long time ago, but more people might get interested ] On Tue, 17 Apr 2001, Mike Kravetz wrote: > > In the near future, I should have some time to begin > working on a prototype implementation. One th