On 26 October 2012 14:44, Evan Huus <eapa...@gmail.com> wrote:
> On Fri, Oct 26, 2012 at 9:29 AM, Sébastien Tandel > <sebastien.tan...@gmail.com> wrote: > > > > > > On Wed, Oct 24, 2012 at 11:13 AM, Evan Huus <eapa...@gmail.com> wrote: > >> > >> On Wed, Oct 24, 2012 at 8:10 AM, Sébastien Tandel > >> <sebastien.tan...@gmail.com> wrote: > >> > > >> > > >> > On Wed, Oct 24, 2012 at 1:10 AM, Guy Harris <g...@alum.mit.edu> wrote: > >> >> > >> >> > >> >> On Oct 18, 2012, at 6:01 PM, Evan Huus <eapa...@gmail.com> wrote: > >> >> > >> >> > I have linked a tarball [2] containing the following files: > >> >> > - wmem_allocator.h - the definition of the allocator interface > >> >> > - wmem_allocator_glib.* - a simple implementation of the allocator > >> >> > interface backed by g_malloc and a singly-linked list. > >> >> > >> >> Presumably an implementation of the allocator could, instead of > calling > >> >> a > >> >> lower-level memory allocator (malloc(), g_malloc(), etc.) for each > >> >> allocation call, allocate larger chunks and parcel out memory from > the > >> >> larger chunks (as the current emem allocator does), if that ends up > >> >> saving > >> >> enough CPU, by making fewer allocate and free calls to the underlying > >> >> memory > >> >> allocator, so as to make it worth whatever wasted memory we have at > the > >> >> ends > >> >> of chunks? > >> >> > >> > > >> > One step further, instead of mempools, I think wireshark could have > >> > great > >> > interest in implementing slabs (slab allocator). Slabs had initially > >> > been > >> > designed for kernel with several advantages over traditional > allocators > >> > in > >> > terms of resources needed to allocate (CPU), (external / internal) > >> > fragmentation and also cache friendliness (most of the traditional > >> > allocators don't care). I've attached some slides about a high-level > >> > description of slab. > >> > > >> > Since then, another paper has been written showing some improvements > and > >> > what it took to write a slab for user-space (libumem). There is > another > >> > well-known exampel out there, called memcache, that implements its own > >> > version (and could be a good intial point for wireshark > implementation, > >> > who > >> > knows? :)) > >> > >> If I understand correctly, a slab allocator provides the most benefit > >> when you have to alloc/free a large number of the same type of object, > > > > you're right, that's where slab is the most efficient at. Although, the > > second paper shows it can be efficient for general purpose allocation > based > > on size and not specific structure. > > > >> but I don't know if this is necessarily the case in Wireshark. There > >> are probably places where it would be useful, but I can't think of any > >> off the top of my head. TVBs maybe? I know emem is currently used all > >> over the place for all sorts of different objects... > > > > I guess the most obvious would be emem_tree (emem_tree_node) might be an > > example used all over and over while dissecting. :) > > There is indeed a bunch of different objects allocated with emem. Also, > it > > might be used to allocate memory for some fragments. > > Ah, yes, the various emem data structures (tree, stack, etc.) would > likely benefit from slab allocators. Converting them to use slabs > would be something to do while porting them from emem to wmem. > > > Since your interface seems to allow it, we could create several slabs > types, > > one for each specific structures that are allocated very frequently > > (emem_tree_node?), others for packets/fragments with some tuned slabs > sizes > > and another with some generic sizes. > > That seems reasonable, presumably with some shared slab code doing the > type-agnostic heavy lifting. I'll have to give a bit of thought to > what the interface for that would be like - if you already have an > interface in mind, please share :) > > Are the slab allocators mentioned "homegrown" or provided by the host OS. If the latter, what platforms are they available on?
___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev@wireshark.org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
