I submitted the first pull request. It is still a pretty large patch but most of it is defined(FreeBSD) conditionals in the right places.
https://github.com/apple/swift-corelibs-libdispatch/pull/319 kevent is very (very, very, very!) ugly right now and 99% sure broken on Darwin. FreeBSD and Darwin diverge on a ton of type definitions on kevent structures. I had to add a lot of casting to get it working on FreeBSD which almost certainly broken it on Darwin. Maybe it should be split into a kevent_mach for Apple platforms and another for FreeBSD, but that would cause code duplication problems. As said on the pull request, the Xcode project depends on a lot of Apple internal sources which means I simply cannot build it on my Mac to make reviewing and fixing issues on Darwin a lot easier. On Oct 16, 2017, 4:39 PM -0200, Pierre Habouzit <phabou...@apple.com>, wrote: > > On Oct 16, 2017, at 8:03 AM, Rogiel Sulzbach via swift-corelibs-dev > > <swift-corelibs-dev@swift.org> wrote: > > > > Thank you for all the help. I will try to take a look at this during the > > week. > > > > I am not even running it in swift yet, I just got it to compile (without > > actually working) so that I could get Foundation to build without having to > > disable the entire URL system, but I would like to use dispatch. > > > > The reason I am going after runloops is because it was the first test to > > fail and many tests seem to depend on it. It seemed like a logical place to > > start porting. > > It really isn'tm it's a high level construct that needs a TON of dispatch > working for it to work, I'd start with simpler tests first. and sample code > you write here and there, that only use pure dispatch, no runloops. > > Focus on sources and the thread pool, this is where your issues will arise > first. Once that is stable, you can move to runloops as it is built atop of > the rest. > > > > I think i get it now. Ill have to manually pass the fd to the kevent > > implementation. Makes sense. As soon as I get things starting to work i'll > > fork and keep a separate branch with these changes to ease review. > > It's also for your sake, if during review was ask you to do things > differently or find some issue, then it's very likely that there will be > ripple effect in the rest of your porting. > > I encourage you to do focused changes (not small in terms of number of files > touched, but small in terms of scope). It'll also allow us to check that it > doesn't regress other ports which is always a concern ;) > > > > > On Oct 12, 2017, 9:08 PM -0300, Pierre Habouzit via swift-corelibs-dev > > <swift-corelibs-dev@swift.org>, wrote: > > > oh and last, while working on FreeBSD, we are killing support for > > > pthread_workqueue_additem_np, pthread_workqueue_setdispatch_np and > > > similar functions that are ancient and do not support all the required > > > prioirties. in your port, please do not try to repair this support if it > > > is broken, because we're killing it anyway, that would be wasted time. > > > > > > The minimal workqueue implementation that we'll support requires > > > implementation of: > > > - _pthread_workqueue_init > > > - _pthread_workqueue_addthreads > > > > > > Until FreeBSD supports this, the same internal workqueue implementation > > > Linux has can be used. These interfaces above are not that different in > > > terms of kernel requirements from the old interfaces and shouldn't be too > > > difficult to support upstream long term from the original > > > pthread_workqueue_additem_np era APIs FreeBSD had support for originally. > > > > > > Anything older than that is about to be removed. > > > > > > -Pierre > > > > > > > On Oct 12, 2017, at 4:56 PM, Pierre Habouzit via swift-corelibs-dev > > > > <swift-corelibs-dev@swift.org> wrote: > > > > > > > > > On Oct 12, 2017, at 4:52 PM, Pierre Habouzit via swift-corelibs-dev > > > > > <swift-corelibs-dev@swift.org> wrote: > > > > > > > > > > > On Oct 12, 2017, at 7:48 AM, Rogiel Sulzbach <rog...@rogiel.com> > > > > > > wrote: > > > > > > > > > > > > I am trying to get libdispatch working on FreeBSD but I am hitting > > > > > > a wall: I cannot get my head around how the runloop implementation > > > > > > dispatches a event to the kqueue once a new job is submitted. So > > > > > > far I was able to get the library to fully compile but most tests > > > > > > are timing out due to a issue getting the kqueue event handler to > > > > > > awake when the poke method is called. > > > > > > Looking at the Linux implementation > > > > > > (https://github.com/apple/swift-corelibs-libdispatch/blob/master/src/queue.c#L6119) > > > > > > it appears to be creating a new eventfd descriptor and calls > > > > > > eventfd_write on the poke method. A eventfd_read call is only made > > > > > > in the event_epoll.c implementation file BUT I am unable to find > > > > > > where the specific fd (created in queue.c) is forwarded to the > > > > > > epool handler. In other words, how is the event queue > > > > > > implementation informed of that file descriptor? > > > > > > On the Darwin side of things, it appears to use a mix of kqueue (on > > > > > > the event handling side) and mach ports (on the signaling side), > > > > > > but i still cannot understand how the event_kevent.c implementation > > > > > > becomes aware of that port allocated on queue.c. > > > > > > In order do get dispatch working on FreeBSD I need to be able to > > > > > > understand the flow that happens between > > > > > > _dispatch_runloop_queue_class_poke and a event getting fired in the > > > > > > kqueue implementation (which I believe should be happening in > > > > > > _dispatch_kq_poll). > > > > > > > > > > > > Pierre Habouzit replied to me on the bug tracker: > > > > > > > The equivalent of the eventfd on kevent systems is to use an > > > > > > > EVFILT_USER. It is very likely that the current implementation > > > > > > > using kevent is badly broken on FreeBSD it has never run for a > > > > > > > very long time. I have no access to FreeBSD at this time to help > > > > > > > though but I can help with questions on the list. > > > > > > Indeed, the entire library needed changes in 39 files to get > > > > > > compiling, let alone working! But I am getting there! > > > > > > > > > > 39 seems a bit excessive so I'd encourage you when you submit this > > > > > awesome porting effort to try to check in individual "porting fixes" > > > > > in separate branches. > > > > > > > > > > You should also try to get it working at the C level before you try > > > > > to integrate with Swift, runloop queues are not required to get the > > > > > basic library working > > > > > > > > > > > > > > > > > If i am understanding it correctly, I must send a EVFILT_USER and > > > > > > upon receiving that event I get the corresponding unote and call > > > > > > dux_merge_evt. Right? > > > > > > > > > > Hmm actually when answering on Jira I had missed that you were > > > > > talking about runllops. However, the equivalent of eventfd on kevent > > > > > will be EVFILT_USER. > > > > > > > > > > You will need to first work on the implementation of CFRunloop itself > > > > > to understand how it "blocks". It will likely park in kevent() with a > > > > > kqueue fd. > > > > > > > > > > The CFRunloop in question will prepare an EVFILT USER that represents > > > > > "availability" of dispatch items to drain in the runloop queue. the > > > > > difficulty is that unlike epoll of portsets on the Mac, the runloop > > > > > queue has to know your kqueue fd. So you need to devise a way to pass > > > > > it down. > > > > > > > > > > Once you have done that _dispatch_runloop_queue_class_poke() on > > > > > FreeBSD will issue a NOTE_TRIGGER of the EVFILT_USER which will break > > > > > the CFRunloo parked in kqueue() out, and this will recognize the > > > > > EVFILT_USER as being the hint that you need to drain it. > > > > > > > > > > I would suggest that on FreeBSD you add a > > > > > _dispatch_runloop_queue_set_kqueue() which you will store where the > > > > > fd/mach port is stored on other platforms. > > > > > > > > it also means that you likely need to export some function shared > > > > between Swift & Dispatch to "recognize" the knote in question, you'll > > > > need to pick an ident and a udata field. You probably want to have > > > > defines in <dispatch/private.h> when on FreeBSD for this purpose, since > > > > both dispatch & CF/Swift will need to know how to form the knote in > > > > question. > > > > > > > > -Pierre > > > > _______________________________________________ > > > > swift-corelibs-dev mailing list > > > > swift-corelibs-dev@swift.org > > > > https://lists.swift.org/mailman/listinfo/swift-corelibs-dev > > > > > _______________________________________________ > > swift-corelibs-dev mailing list > > swift-corelibs-dev@swift.org > > https://lists.swift.org/mailman/listinfo/swift-corelibs-dev >
_______________________________________________ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
