Genius! I had forgotten that the FD returned from epoll_create can itself be polled to indicate when any of the currently registered fds are ready to be processed. That simplifies everything for sure.
Given all the above I'm planning to pass the epoll FD to unsafe-fd->evt and then sync on it in a single thread which then calls the C code to process all the fd changes registered with that epoll instance. My only concern with this is whether that single thread might get mildly starved compared to other racket threads given that it technically represents hundreds of 'green threads' inside itself all implemented in C whereas every other racket thread represents one green thread. Is there any way to hint to the thread scheduler that a particular thread needs higher scheduling priority than others? Also, in this scenario would unsafe-poller give any underlying performance benefit compared to using unsafe-fd->evt and sync? On Tue, Aug 4, 2020, 1:34 PM Matthew Flatt <mfl...@cs.utah.edu> wrote: > You're right: I had momentarily confused epoll with poll. You can just > use the single file descriptor for an epoll object with things like > `unsafe-file-descriptor->semaphore`, or you can use `unsafe-poller`. > > I'll note that `unsafe-file-descriptor->semaphore` scales to a large > number of file descriptors as long as you create a thread to block on > each semaphore. Under the hood, creating and triggering fd-semaphores > is implemented by incremental operations on an epoll object (or kqueue > object, etc.), and each blocking thread will be descheduled until a > semaphore post re-schedules it. But if you already have an epoll object > and the code to handle it, then you don't need the semaphore-based > machinery. > > Matthew > > At Tue, 4 Aug 2020 12:54:56 -0600, Robert D Kocisko wrote: > > Thanks Matthew! That was my thought but given the number of fds in play > > and the frequency of adding and removing interest in them (they are added > > dynamically as needed) it seems like that option would be rather > > inefficient. > > > > Is there by chance any 'magic back door' which would allow me to register > > fds directly with Racket's underlying event loop so that I can bypass the > > extra hops of the thread scheduler and the ffi boundaries? Or if not what > > would need to be taken into consideration if I wanted to add such a back > > door via modifying Racket's source? Are there any concerns with starving > > other threads or other timing issues? > > > > Also I can't help but be intrigued by (unsafe-poller). I'm wondering > what > > benefits it might give (if any) in comparison with the approach you > > suggested. > > > > Thanks, > > Bob > > > > On Tue, Aug 4, 2020, 8:08 AM Matthew Flatt <mfl...@cs.utah.edu> wrote: > > > > > Fusing event loops is always tricky, but if you have the option of > > > exposing individual file descriptors to Racket, then `ffi/unsafe/port` > > > is probably the way to go. You can turn a file descriptor into an event > > > using `unsafe-file-descriptor->semaphore` or `unsafe-fd->evt`, and then > > > you can block on a set using `sync`, etc. > > > > > > Matthew > > > > > > At Mon, 3 Aug 2020 23:31:20 -0700 (PDT), Robert D Kocisko wrote: > > > > I have an existing c++ app that is entirely event-loop driven with > epoll > > > > and I am trying to figure out how to integrate Racket in the same > thread > > > > and allow the existing code to work as-is. I have read the docs > about > > > the > > > > C api and the FFI but so far a straightforward and clean option is > not > > > > apparent to me. The 'embedding' examples that I see appear rather > > > opaque > > > > to me and at best seem to be geared towards an external loop based on > > > > select() rather than epoll(), so I'm assuming that the more > > > straightforward > > > > approach would be to add the existing event handlers onto Racket's > event > > > > system. To that end, it seems there should be a way to register the > > > fd's > > > > of interest with Racket and receive a callback when they are > read-ready > > > but > > > > I can't see a way to do that. For example, how would I maintain a > list > > > of > > > > fds to pass to (sync) if I went that route? Or if it's better to > work > > > it > > > > the other way, how would I call (sync) from c code and apply the > list of > > > > fds to it? I vaguely can see a way towards it but it seems at best > > > super > > > > inefficient. Any thoughts or suggestions would be greatly > appreciated! > > > > > > > > Thanks, > > > > Bob Kocisko > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > Groups > > > > "Racket Users" group. > > > > To unsubscribe from this group and stop receiving emails from it, > send > > > an > > > > email to racket-users+unsubscr...@googlegroups.com. > > > > To view this discussion on the web visit > > > > > > > > > > https://groups.google.com/d/msgid/racket-users/46fb3804-396c-4921-849f-71fe5a6f > > > > ac7ao%40googlegroups.com. > > > > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAGF%3DAeSorcVhB3X%3DvrzNFWOiEZAkB%3DfeAB4iGkZwLx6sWC7%3DhA%40mail.gmail.com.
