Jonathan Adams wrote: > On Fri, Jul 28, 2006 at 06:24:56PM +0100, James Coleman wrote: >> Could someone confirm that I am correct about this that oversize >> allocations are not tracked >> in libumem? > > > They are tracked, but in a different way; you can get all of the oversized > allocations by doing: > > >>*umem_oversize_arena::walk vmem_alloc | ::vmem_seg -v
ohhhh :) more magic. That is quite good. We have one module which is doing practically all those oversize allocations. That oversized list confirms it is using lots of memory but working quite nicely. Now out of a 700M process I can see about 50M tracked in ::umausers and 200M tracked in oversize. Hmmm. I have 450M still to be acoounted for. You'd think it would be easier to see what it was used for!! :) I thought I might see C++ stl doing oversize allocations but I don't. I suspect that is the guilty party. Not sure ... yet. Getting there though. If the missing allocations are not through malloc then it is going to be harder. If I truss then I see mmap on process start .... but not much after. Actually looks like all the mmap calls are invoked by malloc. I see loads of brk calls. Oh dear. Hmmm. Maybe I could write a simple mdb script which would break on every brk call, log details and continue. I don't know much about mmap or sbrk ... I think it's time to find out a bit more. > Which dcmds in particular are you using that you'd like oversize support for? I am using ::umausers alot. We have a process whose behaviour can vary quite a bit. Depending on how it is deployed different protocols are encountered and the behaviour of the process is driven by what it is listening to. The whole system is definately not sentient yet ... but it is ALIVE! :) A very useful thing is to be able to analyze how many and of what type are the objects (c++ or c malloc) that get created in the system. We have counters that track many components of the system but a comprehensive tool that would show a full memory profile without explicitly adding code would be valuable. > > >>It makes sense really. I am thinking it might be possible to >> extend libumem to >> allow tracking of oversize allocations. Would that be a relatively >> straight-forward task? >> I have the source >;) eek! > > > Depending upon the dcmd, it is probably quite possible to extend it to take > oversize allocations into account. I will have to read the source more to grok your umem_oversize_arena walking properly for a start. Thanks again Jonathan. :) James.
