On Nov 9, 2010, at 9:03 PM, Nick Mathewson wrote:
> On Tue, Nov 9, 2010 at 3:28 PM, Ralph Castain <r...@open-mpi.org> wrote:
>> Hi folks
>>
>> I'm running into a problem that probably results from my ignorance. So I
>> figured I would ask if someone can tell me what I'm doing wrong.
>>
>> I have a thread that loops the event library with the following call:
>>
>> events += event_loop(mca_oob_tcp_component.event_base, EVLOOP_ONCE);
>>
>> In another thread, I create and add an event that is supposed to occur when
>> a file descriptor is ready for "write":
>>
>> event_set(mca_oob_tcp_component.event_base,
>> &peer->peer_send_event,
>> peer->peer_sd,
>> EV_WRITE|EV_PERSIST,
>> mca_oob_tcp_peer_send_handler,
>> peer);
>> event_add(&peer->peer_send_event, 0);
>>
>> Note that the event_set and event_add can (and in this case, probably do)
>> occur while I am in the event_loop function.
>>
>> What I find is that this event never gets fired. However, events that are
>> set and added -before- going into the loop do fire.
>>
>> So my question is: given that this is a dynamic system, how do I get the
>> event_loop to "see" new events? I want to block in the loop, so I don't
>> really want to set NONBLOCK if I can avoid it (otherwise, I would have to
>> add another blocking call somewhere to keep the thread from endlessly
>> cycling).
>
> So I'm assuming that you've got all the threading callbacks set up
> (probably via evthread_use_pthreads()) before you created the event
> base, so that evthread_make_base_notifiable() was called on the
> event_base when you created it. If that's not the case, that's
> probably the problem there.
I missed that - this may well be my problem. Let me dig into this a little and
get back to you.
FWIW: I don't see myself drop out of the event loop when I add the event. In
fact, just the opposite - I'm stuck in the loop and can't get out. So I'm not
sure the bug mentioned below is accurate.
Let me see what happens when I setup the event base for notification.
Thanks!
>
> Otherwise, I think this one is an honest-to-goodness bug.
>
> Here's what happens when you add an event from another thread.
>
> The event_add locks the event_base's structures, makes the changes
> necessary to add the event, and then notices that it isn't running in
> the main thread, so it needs to "alert" the main thread to exit and
> re-enter the dispatch function[*]. It does this by calling
> evthread_notify_base(), which sends a byte down a pipe (on most Unix)
> or a socketpair (on Windows), or by using an eventfd (on Linux) [**].
> There's an (EVLIST_INTERNAL) event handler for this
> pipe/socketpair/eventfd that notices that we've been "notified" so we
> can drain the pipe/socketpair/eventfd.
>
> Now here's where I think the bug is: that event counts as an active
> event, and processing it gets counted as processing an event, so if
> you're running the loop with EVLOOP_ONCE, the loop will exit right
> after it notices that it should wake up, even though no user callbacks
> were actually entered.
>
> The logic is in event_base_loop():
>
> if (N_ACTIVE_CALLBACKS(base)) {
> event_process_active(base);
> if (!base->event_count_active && (flags & EVLOOP_ONCE))
> done = 1;
> } ...
>
> I think that the second "if" might be wrong. We maybe want to
> continue looping not only if there are active callbacks, but also if
> the only active events that we processed were internal events.
>
> With the current behavior, with EVLOOP_ONCE, I think adding the event
> from another thread will exit the loop right away. Is that consistent
> with the behavior you're seeing? If not, then the bug is possibly in
> the notification stuff, though that part actually is fairly well
> tested.
>
> It's also possible that I'm writing this too late at night for me, and
> I've deeply misanalyzed the code here.
>
> (As an aside, we should probably also continue looping if
> N_ACTIVE_CALLBACKS(base) is nonzero.)
>
> [*] For backends like poll and select, we need to re-enter the
> dispatch function after getting event changes so that we can call
> select/poll with their new arguments. For stuff like epoll and
> kqueue, we need to process the changelist and re-enter the wait
> function.
> [**] Yes, I do indeed think that kqueue should use EVFILT_USER, but
> that's a 2.1 thing.
>
> --
> Nick
> ***********************************************************************
> To unsubscribe, send an e-mail to majord...@freehaven.net with
> unsubscribe libevent-users in the body.
***********************************************************************
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-users in the body.
