On Mar 6, 2011, at 6:09 AM, Vlad Galu wrote: > > > On Sun, Mar 6, 2011 at 2:06 PM, Vlad Galu <d...@dudu.ro> wrote: > > > On Sun, Mar 6, 2011 at 6:53 AM, Eric Anderson <ander...@ttel.com> wrote: > On Mar 5, 2011, at 10:44 AM, Deomid Ryabkov wrote: > > > On 03/05/2011 04:02 AM, Eric Anderson wrote: > >> Hi all, > >> > >> I have a moderately threaded userland program (all C) I am working on > >> (using pthreads, freebsd 8.1 64bit). It seems to leak memory (using > >> standard malloc/free) badly. > > as opposed to what? OpenBSD? Linux? Windows? why do you think your problem > > is specific to FreeBSD (as evidenced by your post to a FreeBSD-specific > > list) or is related to threaded programs? > > OpenSolaris and Mac OS X. I didn't really assume or state it was specific to > FreeBSD, just that this scenario was on FreeBSD. I happen to do most of the > development and testing on OS X and FreeBSD, and I've enjoyed the FreeBSD > community for a very long time. > > > > > > >> I am using pcap to capture packets and process them. I have a handful of > >> libs statically linked in (pcap is one, the rest don't seem to matter - I > >> can remove them and still see the leak). > >> > >> Does anyone know of issues regarding malloc/free on multithreaded userland > >> apps? > > hell yeah. it goes like this: you malloc() then forget to free() - boom, > > you have a memory leak. > > > > you're welcome. > > > Thanks, very insightful. > > > > > > > > sarcasm aside, those questions still remain: why do you think os/libraries > > are the problem and not your code? > > Because I am tracking all malloc and free calls within the application code > (aside from libraries) and I can account for all malloc'ed memory and freed > memory in both count and by bytes, yet looking at ps output shows a very > different story, and if I leave it run long enough, will consume all memory > and swap in the system and then be killed off. I wrapped malloc/free in a > function, and record all memory alloc'ed and free'd. The only memory I > cannot track is memory alloced and freed by libraries I am pulling in (well, > can't track easily anyway without hacking through all of their source code). > > > > > you can't post all of it, ok, and we don't want all of it either. can you > > isolate a specific example of where valid usage of a library causes a leak? > > > Not really - if I could, I would have fixed it by now. It's a non-trivial > issue - which is why I am beginning to suspect something more complicated > than a "oops I forgot to free" kind of error. Plus, I have seen a few people > elsewhere on various forums/mailing lists with similar issues claiming that > switching to the Hoard allocator fixed the problem (which doesn't seem to be > happy with 32bit FreeBSD - tried it). I have also seen various comments > about pthreads and memory allocators having apparent leaks at some threading > level, but not sure. > > Thanks, > Eric > > > Had there been a memleak in jemalloc, I'm sure more people would have spotted > it by now. How many pcap_t structs do you use in your app? libpcap is not > threadsafe. FWIW I've been running a pcap/threaded application continuously > for the past couple of years and the memory usage has been constant. > > Also, put a couple of printf()s before and after pcap_dispatch()/pcap_loop() > (if you use any of them), to make sure they don't block waiting for your > callback to return (on some platforms it doesn't, so you never get to free > memory in outer frames). This might not necessarily be your case, but it's > worth taking a look at. > > > That should've been pcap_dispatch()/pcap_next().
Hmm - interestingly enough, I use a third party lib that handles some parts of what pcap does.. I'll did into that lib and check these things. Thanks for the idea! Eric _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
