Re: mbuf external buffer reference counters

2002-07-15 Thread Scott Hess
On Fri, 12 Jul 2002, Bosko Milekic wrote: > On Fri, Jul 12, 2002 at 04:26:53AM -0700, Jon Mini wrote: > > I'm probably speaking out of turn here (I have no idea what structure > > you all are talking about), but a monodirectional ring can be safely > > modified with a compare-and-exchange atomic

RE: mbuf external buffer reference counters

2002-07-12 Thread Jim McGrath
> Julian Elischer writes: > > > > > > Te stuff under consideration originally came from OSF/1 which became > > true-64 > > > > that was heavily SMP > > can anyone find out what they did? > > From looking at a Tru64 5.1 header file, it looks like they do per-ext > locking and declare an MBUF

Re: mbuf external buffer reference counters

2002-07-12 Thread Andrew Gallatin
Bosko Milekic writes: <...> > If we decide to allocate jumbo bufs from their own seperate map as > well then we have no wastage for the counters for clusters if we keep > them in a few pages, like in -STABLE, and it should all work out fine. That sounds good. > For the jumbo bufs I

Re: mbuf external buffer reference counters

2002-07-12 Thread Bosko Milekic
On Fri, Jul 12, 2002 at 06:55:37PM -0400, Andrew Gallatin wrote: [...] FWIW, BSD/OS also does similar to -STABLE. [...] > I agree with John about where to put the refcnts: I think we should > have a big hunk of memory for the refcnts like in -stable. My > understanding is that the larger virtu

Re: mbuf external buffer reference counters

2002-07-12 Thread Andrew Gallatin
Julian Elischer writes: > > > On Fri, 12 Jul 2002, Giorgos Keramidas wrote: > > > On 2002-07-12 07:45 +, Bosko Milekic wrote: > > > > > > So I guess that what we're dealing with isn't really a > > > "monodirectional" ring. Right? > > > > No it isn't. It looks more like the "di

Re: mbuf external buffer reference counters

2002-07-12 Thread John Polstra
In article <[EMAIL PROTECTED]>, Bosko Milekic <[EMAIL PROTECTED]> wrote: > > I've thought about the cache issue with regards to the ref. counts > before, actually, and initially, I also thought the exact same thing > as you bring up here. However, there are a few things you need to > re

Re: mbuf external buffer reference counters

2002-07-12 Thread Bosko Milekic
On Fri, Jul 12, 2002 at 11:03:45AM -0700, John Polstra wrote: > I've been out of town and I realize I'm coming into this thread late > and that it has evolved a bit. But I still think it's worthwhile to > point out a very big problem with the idea of putting the reference > count at the end of e

Re: mbuf external buffer reference counters

2002-07-12 Thread Julian Elischer
On Fri, 12 Jul 2002, Giorgos Keramidas wrote: > On 2002-07-12 07:45 +, Bosko Milekic wrote: > > > > So I guess that what we're dealing with isn't really a > > "monodirectional" ring. Right? > > No it isn't. It looks more like the "dining philosophers" problem. > But that problem's soluti

Re: mbuf external buffer reference counters

2002-07-12 Thread John Polstra
In article <[EMAIL PROTECTED]>, Bosko Milekic <[EMAIL PROTECTED]> wrote: > > Right now, in -CURRENT, there is this hack that I introduced that > basically just allocates a ref. counter for external buffers attached > to mbufs with malloc(9). What this means is that if you do something >

Re: mbuf external buffer reference counters

2002-07-12 Thread Giorgos Keramidas
On 2002-07-12 07:45 +, Bosko Milekic wrote: > The jist of the problem is that when you want to say, remove yourself > from the list, you have to: > > 1) your "next"'s back pointer to your "back" pointer > 2) your "Prev"'s next pointer to your "next" pointer > > So that's two operations but for

Re: mbuf external buffer reference counters

2002-07-12 Thread Bosko Milekic
On Fri, Jul 12, 2002 at 04:26:53AM -0700, Jon Mini wrote: > On Thu, Jul 11, 2002 at 11:41:04PM -0700, Alfred Perlstein wrote: > > > That's a cool idea.. haven't looked at NetBSD but am imagining the > > > mbufs would be linked in a 'ring'. This works because you never > > > care how many referenc

Re: mbuf external buffer reference counters

2002-07-12 Thread Giorgos Keramidas
On 2002-07-11 17:12 +, Bosko Milekic wrote: > On Thu, Jul 11, 2002 at 01:56:08PM -0700, Luigi Rizzo wrote: > > example: userland does an 8KB write, in the old case this requires > > 4 clusters, with the new one you end up using 4 clusters and stuff > > the remaining 16 bytes in a regular mbuf,

Re: mbuf external buffer reference counters

2002-07-12 Thread Jon Mini
On Fri, Jul 12, 2002 at 07:45:07AM -0400, Bosko Milekic wrote: > > [ ... Description of modifying a bidrectional ring ... ] > > So I guess that what we're dealing with isn't really a > "monodirectional" ring. Right? Yep. =) -- Jonathan Mini <[EMAIL PROTECTED]> http://www.freebsd.org/ To Un

Re: mbuf external buffer reference counters

2002-07-12 Thread Bosko Milekic
On Fri, Jul 12, 2002 at 12:10:41AM -0700, Alfred Perlstein wrote: > * Julian Elischer <[EMAIL PROTECTED]> [020712 00:00] wrote: > > > > > > On Thu, 11 Jul 2002, Alfred Perlstein wrote: > > > > > > That's true, but could someone explain how one can safely and > > > effeciently manipulate such a

Re: mbuf external buffer reference counters

2002-07-12 Thread Jon Mini
On Thu, Jul 11, 2002 at 11:41:04PM -0700, Alfred Perlstein wrote: > > That's a cool idea.. haven't looked at NetBSD but am imagining the > > mbufs would be linked in a 'ring'. This works because you never > > care how many references are, just whether there's one or more than > > one, and this is

Re: mbuf external buffer reference counters

2002-07-11 Thread Alfred Perlstein
* Julian Elischer <[EMAIL PROTECTED]> [020712 00:00] wrote: > > > On Thu, 11 Jul 2002, Alfred Perlstein wrote: > > > > That's true, but could someone explain how one can safely and > > effeciently manipulate such a structure in an SMP environment? > > what does NetBSD do for that? They don't!

Re: mbuf external buffer reference counters

2002-07-11 Thread Julian Elischer
On Thu, 11 Jul 2002, Alfred Perlstein wrote: > > That's true, but could someone explain how one can safely and > effeciently manipulate such a structure in an SMP environment? what does NetBSD do for that? > I'm not saying it's impossible, I'm just saying it didn't seem > intuative to me back

Re: mbuf external buffer reference counters

2002-07-11 Thread Alfred Perlstein
* Archie Cobbs <[EMAIL PROTECTED]> [020711 22:30] wrote: > Bosko Milekic writes: > > > mbufs that referred to the same object were linked together. > > > I forget the details exactly. maybe someone else can remember.. > > > it did it without ref counts somehow.. > > > > Yes, this is in NetBSD s

Re: mbuf external buffer reference counters

2002-07-11 Thread Archie Cobbs
Bosko Milekic writes: > > mbufs that referred to the same object were linked together. > > I forget the details exactly. maybe someone else can remember.. > > it did it without ref counts somehow.. > > Yes, this is in NetBSD still and it is very elegant. I remember > looking at this a long ti

Re: mbuf external buffer reference counters

2002-07-11 Thread Alfred Perlstein
* Bosko Milekic <[EMAIL PROTECTED]> [020711 19:28] wrote: > > On Thu, Jul 11, 2002 at 04:10:32PM -0700, Julian Elischer wrote: > > Don't forget that "external" does not neccesarily mean "cluster". > > I still consider the method used in (hmm was it NetBSD or OSF/1?) > > to be very good.. > > > >

Re: mbuf external buffer reference counters

2002-07-11 Thread Barney Wolff
Perhaps it might have something to do with disk sector size and memory page size and BUFSIZ all being powers of 2? :) On Thu, Jul 11, 2002 at 10:34:46PM -0400, Kelly Yancey wrote: > ... that for better or worse userland apps think that > using power-of-2 write buffers will improve performance.

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
I'm sorry. I should have waited before hitting the "send" button. I've had a long and [shitty] day and I shouldn't have blew it off here. Sorry. -- Bosko Milekic [EMAIL PROTECTED] [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of

Re: mbuf external buffer reference counters

2002-07-11 Thread Kelly Yancey
On Thu, 11 Jul 2002, Bosko Milekic wrote: > First of all, I'm not "blowing off" anyone's comments. I don't > appreciate the fact that you're eagerly instructing me to "not blow off > comments" (which I didn't do to begin with) without providing any more > constructive feedback. > > All I p

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
On Thu, Jul 11, 2002 at 04:10:32PM -0700, Julian Elischer wrote: > Don't forget that "external" does not neccesarily mean "cluster". > I still consider the method used in (hmm was it NetBSD or OSF/1?) > to be very good.. > > mbufs that referred to the same object were linked together. > I forget

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
On Thu, Jul 11, 2002 at 07:31:17PM -0400, Kelly Yancey wrote: > > This is a good observation if we're going to be doing benchmarking, > > but I'm not sure whether the repercussions are that important (unless, > > as I said, there's a lot of applications that send exactly 8192 > > byte chu

Re: mbuf external buffer reference counters

2002-07-11 Thread Kelly Yancey
On Thu, 11 Jul 2002, Bosko Milekic wrote: > > On Thu, Jul 11, 2002 at 01:56:08PM -0700, Luigi Rizzo wrote: > > example: userland does an 8KB write, in the old case this requires > > 4 clusters, with the new one you end up using 4 clusters and stuff > > the remaining 16 bytes in a regular mbuf, t

Re: mbuf external buffer reference counters

2002-07-11 Thread Julian Elischer
On Thu, 11 Jul 2002, Bosko Milekic wrote: > > Well, I can use a different map, I guess (I use a different map for > mbufs in order to not let huge cluster allocations eat up all of the > address space reserved for mbufs). However, it seems that jumbo bufs > and clusters are logically

Re: mbuf external buffer reference counters

2002-07-11 Thread Julian Elischer
On Thu, 11 Jul 2002, Bosko Milekic wrote: > > On Thu, Jul 11, 2002 at 11:00:56PM +0200, Juan Francisco Rodriguez Hervella wrote: > > First of all, let me say that Im newbie with these topics, > > I only know a bit about mbufs, but I dont understand how > > can an application trim away the ref

Re: mbuf external buffer reference counters

2002-07-11 Thread Juan Francisco Rodriguez Hervella
Ahhh, ok, I misunderstood the first mail :) Thanks. On Thu, 11 Jul 2002, Bosko Milekic wrote: > > On Thu, Jul 11, 2002 at 11:00:56PM +0200, Juan Francisco Rodriguez Hervella wrote: > > First of all, let me say that Im newbie with these topics, > > I only know a bit about mbufs, but I dont un

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
On Thu, Jul 11, 2002 at 11:00:56PM +0200, Juan Francisco Rodriguez Hervella wrote: > First of all, let me say that Im newbie with these topics, > I only know a bit about mbufs, but I dont understand how > can an application trim away the refcount if the size is > MCLBYTES = 2040 - sizeof(refcoun

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
On Thu, Jul 11, 2002 at 01:56:08PM -0700, Luigi Rizzo wrote: > example: userland does an 8KB write, in the old case this requires > 4 clusters, with the new one you end up using 4 clusters and stuff > the remaining 16 bytes in a regular mbuf, then depending on the > relative producer-consumer spe

Re: mbuf external buffer reference counters

2002-07-11 Thread Juan Francisco Rodriguez Hervella
On Thu, 11 Jul 2002, Luigi Rizzo wrote: > Hi, > > certainly removing the malloc will improve performance a lot. > > As I already mentioned to Bosko, in principle the available area > in ext.buffers is irrelevant, and i do not believe this will break > anything (and if it does, it will be easy

Re: mbuf external buffer reference counters

2002-07-11 Thread Luigi Rizzo
On Thu, Jul 11, 2002 at 04:42:25PM -0400, Bosko Milekic wrote: ... > > and trimming away the refcount area might easily result in suboptimal > > allocation of storage within the kernel. > > Can you elaborate on the sub-optimal performance comment with, > perhaps, an example? I'm sorry but I'

Re: mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
On Thu, Jul 11, 2002 at 01:38:02PM -0700, Luigi Rizzo wrote: > Hi, > > certainly removing the malloc will improve performance a lot. > > As I already mentioned to Bosko, in principle the available area > in ext.buffers is irrelevant, and i do not believe this will break > anything (and if it do

Re: mbuf external buffer reference counters

2002-07-11 Thread Luigi Rizzo
Hi, certainly removing the malloc will improve performance a lot. As I already mentioned to Bosko, in principle the available area in ext.buffers is irrelevant, and i do not believe this will break anything (and if it does, it will be easy to fix in the kernel), but some applications might decid

mbuf external buffer reference counters

2002-07-11 Thread Bosko Milekic
Hi, Right now, in -CURRENT, there is this hack that I introduced that basically just allocates a ref. counter for external buffers attached to mbufs with malloc(9). What this means is that if you do something like allocate an mbuf and then a cluster, there's a malloc() call that is ma