TL;DR - Jakub recently proposed a few changes to emem [1]. While I think they are a very good idea, I believe that in the long term the current emem design has too many fundamental limitations to make it worth adapting for our future needs. I propose that it should be gradually deprecated in favour of something else, and I have written a simple example of what that something else might be.
--- The Long Version --- After my recent adventures with emem, I took some time to work through what features Wireshark will likely want in its memory manager now and in the future. Predicting the future is always a tricky business, but I tried not to stray too far from the obvious: - Arbitrary instances of the current seasonal and ephemeral allocators. These will be necessary if we ever want to support opening multiple files at once. - Thread-safe allocators. Necessary if we ever want to do multi-threaded dissection or re-dissection of a single file. - Allocators with different scopes. I can think of a couple of different places where a new allocator scope might simplify existing code, and I'm sure there are others. With these ideas in mind, I then took a closer look at the current emem design and tried to estimate the amount of work involved in adapting (and maintaining) it going forward. I concluded that in the long run, emem has too many fundamental limitations to make it worth adapting: - Supporting arbitrary seasonal and ephemeral pools would require non-trivial API changes, causing a lot of pain in dependent code. - The current glib and mmap allocators are jammed together, often living side-by-side in the same functions. As I found out recently, trying to tweak either one can have a lot of unexpected side-effects. - Adding a single new scope requires writing a new wrapper for every API function (*_alloc, *_alloc0, *_strdup, *_strndup, etc). At best this is just extra work, but at worst it leads to API inconsistencies, where, for example, there's an ep_strbuf implementation but no equivalent se_ functions (that's a real example by the way). I would like to propose that emem is gradually deprecated in favour of a new memory management framework that is designed with these requirements in mind. A separate new interface will ease the pain of migration by allowing us to maintain emem as a deprecated interface for as long as necessary. In the spirit of backing up ideas with working code, I have written a simple version of what I think such a new framework might look like. It cleanly separates out the allocator back-ends (glib, mmap, etc) and the utility functions (strdup etc.) from the core interface. It also requires that all API functions are explicitly passed the scoped allocator instance they want to use. This makes it trivial to add new scopes or to support multiple instances of the current scopes. Because the allocators are cleanly distinguished, making any one of them thread-safe is a fairly simple operation. I have linked a tarball [2] containing the following files: - wmem_allocator.h - the definition of the allocator interface - wmem_allocator_glib.* - a simple implementation of the allocator interface backed by g_malloc and a singly-linked list. - wmem_core.* - implementations of wmem_alloc() and wmem_alloc0() - wmem_strutl.* - implementations of wmem_strcpy() and wmem_strncpy() - wmem.h - the general header file for inclusion by consumers of wmem, it simply wraps inclusion of wmem_core.h, wmem_strutl.h and any others that might be created The usage might look something like this: wmem_allocator_t *ep_scope = wmem_create_glib_allocator(); doWork(ep_scope); wmem_destroy_glib_allocator(ep_scope); and then in doWork, instead of ep_alloc(numBytes) you would call wmem_alloc(ep_scope, numBytes). Alternatively, if the outer block is in a loop then it doesn't have to create/destroy each time, and can simply call wmem_free_all(ep_scope) between calls to doWork(). Ideas, comments and constructive criticisms are always welcome. What do you think? Evan [1] https://www.wireshark.org/lists/wireshark-dev/201210/msg00116.html [2] https://dl.dropbox.com/u/171647/wmem.tar.gz ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev@wireshark.org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
