On Oct 23, 2010, at 12:40 PM, Ralph Castain wrote: > > On Oct 23, 2010, at 9:42 AM, Nick Mathewson wrote: > >> On Sat, Oct 23, 2010 at 5:14 AM, Ralph Castain <r...@open-mpi.org> wrote: >>> Hi folks >>> >>> I successfully updated our libevent integration in Open MPI, but have >>> encountered a problem with one use-case that used to work and now doesn't. >>> Before proceeding to devise a fix, I just wanted to confirm that I >>> accurately understand the issue. >>> >>> The problem arises from this scenario: >>> >>> 1. we receive a command via a message that we receive in a file descriptor >>> event. We "push" the command message into a timer event (duration zero >>> time) to help break a threading issue, and then return from the file >>> descriptor event. >>> >>> 2. the event library is called with LOOP_ONCE, causing the timer event to >>> fire. >>> >>> 3. from within the timer event, the command causes us to execute a >>> procedure that results in us having to wait for another event to occur. We >>> "block" in that position, running a loop that includes a call to progress >>> the event library (i.e., a call to event_loop(LOOP_ONCE)). >>> >>> This last step used to work just fine, yet now yields the following warning: >>> >>> warn] event_base_loop: reentrant invocation. Only one event_base_loop can >>> run on each event_base at once.: Resource temporarily unavailable >>> >>> and no progression of the event library occurs. >>> >>> I can see a few ways around this problem, but all involve considerable code >>> rewrite. Before I do so, I just wanted to confirm my understanding that >>> this behavior changed (i.e., recursion is no longer allowed), and see if >>> any alternative approaches to revising such codes are recommended. >> >> Hi, Ralph! I'll try to answer in more detail later on -- I can't >> write too much now, but I wanted to drop you a line before you started >> rewriting your code. >> >> I think that the warning you hit may be overzealous. It was meant to >> detect errors that would previously pass silently and produce >> hard-to-diagnose crashes, such as running event_base_dispatch on the >> same event base from multiple threads at once. But I think your use >> case may either a) work right, or b) be trivial to make work right, >> and if so, we should fix Libevent not to warn about it. >> >> I'll look harder at the code here and try to see what's going on here, >> but calling the loop from a callback really was working fine with >> Libevent 1.4, then I think 2.0 should support it. > > FWIW: I decided to just try commenting out the "return" following the warning > in event.c, and everything works fine. > > Not recommending that as a permanent fix, of course - it may just work due to > our peculiar usage. After all, we are running single-threaded, and so you > can't call the dispatch from multiple threads. But thought you might like to > know.
Been thinking about this problem a bit. I wonder if the solution isn't simply to allow recursive calls to event_loop, but for the event library to ignore any event that is currently active? In other words, have the library go ahead and run thru all its events, setting a flag in the event struct when that event handler is called. Subsequent loops check all events -except- those whose flag is set. When an event handler returns, you reset the flag, thus making it active again, or remove the event if it is non-persistent (might be risky in this scenario as popping up the recursion tree could yield stale data - can be handled, but complicated). I haven't thought thru all the thread locking issues around setting/resetting that flag, but I think you'd be okay so long as you refuse to allow an event handler to block while waiting to re-trigger the same event it resides inside (either programmatically somehow or by fiat - i.e., "you will thread lock if you do something this stupid") - perhaps add a lock/condition to the event struct so you can lock only that event when checking/changing the flag. Some performance considerations here, of course. Not sure you can be more re-entrant than that... HTH Ralph > >> >> Thanks for the heads-up, and thanks to everybody else who's hunting >> for regressions and other bugs in 2.0.8-rc. >> >> yrs, >> -- >> Nick >> *********************************************************************** >> To unsubscribe, send an e-mail to majord...@freehaven.net with >> unsubscribe libevent-users in the body. > *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
