OK. Loaded up JProfiler and confirmed that it’s not LevelDB. This is a non-persistent broker I’m testing on.
Looks like it’s spending all it’s time in CopyOnWriteArrayList.remove and Timer.purge… Which is hopeful because this is ALL due to ActiveMQ internals and in theory LevelDB should perform well if we improve the performance of ActiveMQ internals and fix this lock bug. Which would rock! It should ALSO make queue creation faster. On Sun, Feb 22, 2015 at 4:10 PM, Kevin Burton <bur...@spinn3r.com> wrote: > > > On Sun, Feb 22, 2015 at 3:30 PM, Tim Bain <tb...@alumni.duke.edu> wrote: > >> So if LevelDB cleanup during removeDestination() is the presumed culprit, >> can we spin off the LevelDB cleanup work into a separate thread (better: a >> task object to be serviced by a ThreadPool so you can avoid a fork bomb if >> we remove many destinations at once) so the call to removeDestination() >> can >> return quickly and LevelDB can do its record-keeping in the background >> without blocking message-processing? >> > > Would that be possible? If the delete is pending on ActiveMQ there is a > race where a producer could re-create it unless the lock is held. > > Though I guess if you dispatched to the GC thread WITH the lock still held > you would be ok but I think if we use the existing purge thread then we’re > fine. > > OK. I think I’m wrong about LevelDB being the issue. To be fair I wasn’t > 100% certain before but I should have specified. > > Our current production broker is running with persistent=false.. .and I > just re-ran the tests without disk persistence and it has the same problem. > > So the main issue how is why the heck is ActiveMQ taking SO LONG to GC a > queue. It’s taking about 100ms which is an insane amount of time > considering this is done all in memory. > > Kevin > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> > > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
