Re: [swift-corelibs-dev] libdispatch build issue
swift-corelibs-dev-boun...@swift.org wrote on 12/11/2015 03:17:47 AM: > From: f-dz-f via swift-corelibs-dev > To: swift-corelibs-dev@swift.org > Date: 12/11/2015 03:18 AM > Subject: [swift-corelibs-dev] libdispatch build issue > Sent by: swift-corelibs-dev-boun...@swift.org > PS On ubuntu 14.04 project is also generally unbuildable (despite > title of commit:33b735c5330b5a97d029bccfff9a7d78ef1f214 ;)). Hi, Right now the Linux build is assuming you've installed some pre-req packages. In particular, sudo apt-get install libblocksruntime-dev libkqueue-dev libpthread-workqueue-dev I've been building on 15.10; haven't tried 14.04 recently. Maybe try install the pre-reqs on 14.04 and see what happens? I'll put together a quick check sheet with build pre-reqs and instructions and submit a pull request to include it. --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch build issue
swift-corelibs-dev-boun...@swift.org wrote on 12/11/2015 11:19:15 AM: > > The linux port is in progress, and even if it built, right now, it > wouldn’t be functional. > > -Pierre > > Yes. I completely understand that there linux port is not > functional. So I want to contribute to it. But it's practically > impossible since there is no usable build system :(. It's limping along. We've managed to run a few programs successfully. I think the next step is actually to get the test suite to compile & build so we can see what functionally isn't working yet. Pierre and Daniel, we could use some guidance on how to approach this. The tests have quite a few unguarded includes of mach header files, uses of mach types, and includes of Foundation. How would you like this porting to proceed? Hubertus has already done some initial poking at it, but getting direction on the intended outcome would be very helpful. thanks, --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch build issue
phaus...@apple.com wrote on 12/11/2015 12:02:30 PM: > This is only tangentially related: but I was attempting to build > dispatch as well to start work on CFRunLoop and a few other dispatch > bound features in Foundation on linux. I was seeing similar failures > as Dzianis Fedarenka. Is there a branch/fork that I can grab to test > this stuff out w/ bringing stuff up in Foundation? If you grab the master branch it should build libdispatch.so on ubuntu 15.04. I've put a quick cheat sheet in a pull request: https://github.com/apple/swift-corelibs-libdispatch/pull/11 There's a patch from Hubertus to get timers at least partially working that you may want too: https://github.com/apple/swift-corelibs-libdispatch/pull/8 > My question is that of design and status of where things are at; > CFRunLoop requires a method to interoperate with the main dispatch > queue via _dispatch_get_main_queue_port_4CF and > _dispatch_main_queue_callback_4CF. CFRunLoop will need to transact > upon the same event source type as dispatch to ensure the main queue > behaves correctly - so what type of event source should we be using? > A socket? A producer/consumer notification via pthread conditions? Sorry, I don't understand the code well enough yet to answer this one :( I'm sure Pierre or Daniel can. --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch build issue
Thanks Pierre. We made a little progress on getting tests to compile this afternoon: https://github.com/dgrove-oss/swift-corelibs-libdispatch/tree/fixtests Absolutely not ready for a pull request, but mentioning it in case anyone else was starting on the same task and wanted to collaborate. --dave From: Pierre Habouzit To: David P Grove/Watson/IBM@IBMUS Cc: Dzianis Fedarenka , swift-corelibs-dev@swift.org Date: 12/11/2015 12:59 PM Subject:Re: [swift-corelibs-dev] libdispatch build issue Sent by:phabou...@apple.com On Dec 11, 2015, at 8:27 AM, David P Grove via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote: swift-corelibs-dev-boun...@swift.org wrote on 12/11/2015 11:19:15 AM: > > The linux port is in progress, and even if it built, right now, it > wouldn’t be functional. > > -Pierre > > Yes. I completely understand that there linux port is not > functional. So I want to contribute to it. But it's practically > impossible since there is no usable build system :(. It's limping along. We've managed to run a few programs successfully. I think the next step is actually to get the test suite to compile & build so we can see what functionally isn't working yet. Pierre and Daniel, we could use some guidance on how to approach this. The tests have quite a few unguarded includes of mach header files, uses of mach types, and includes of Foundation. How would you like this porting to proceed? Hubertus has already done some initial poking at it, but getting direction on the intended outcome would be very helpful. We do have an internal test-suite that we can’t share because it rely on internal Apple infrastructure. I haven’t closed the loop with Daniel completely on this but our position I think is that these should be made portable even if that means dropping anything mach related stuff by hiding it behind a #if __APPLE__ or similar. -Pierre ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] [swift-users] build libdispatch failed in ubuntu 14.04
swift-corelibs-dev-boun...@swift.org wrote on 03/29/2016 11:58:13 AM: > > Hi, everyone > I follow the command written in INSTALL in > https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL > > sh autogen.sh > > ./configure > > make > > but "make" failed, report is "No rule to make target `all'. " > Is there some optional command I missed? > One way that could happen is if you skipped the step to initialize the libpwq git submodule. The configure script is supposed to detect that you failed to do this, but it appears to not be doing so robustly (I just tried it and it did fail with "No rule to make target `all`" from the libpwq subdir). From the INSTALL file right about the BUILD commands: Initialize git submodules: We are using git submodules to incorporate a specific revision of the upstream pthread_workqueue library into the build. git submodule init git submodule update --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] Setting property "maxConcurrentOperationCount = 1" of NSOperation fails to execute operations using OpenSource Foundation.
I think the intuition about the dispatch overlay being the problem on Linux
is right on. We may have to hack around the problem on the Foundation side
until an improved overlay is available to use.
--dave
From: Philippe Hausler via swift-corelibs-dev
To: Mamatha Busi
Cc: swift-corelibs-dev@swift.org
Date: 05/06/2016 09:43 AM
Subject:Re: [swift-corelibs-dev] Setting property
"maxConcurrentOperationCount = 1" of NSOperation fails to
execute operations using OpenSource Foundation.
Sent by:swift-corelibs-dev-boun...@swift.org
I have a feeling this is associated with the changes for IUO types that
recently landed. I am very worried that this actually could happen on
Darwin targets as well. Perhaps it is the swift overlay for dispatch that
is correcting the failure on Darwin and the dispatch on Linux is missing
that annotation. Worth looking into.
Sent from my iPhone
On May 6, 2016, at 12:16 AM, Mamatha Busi wrote:
@ Philippe: Your right. Converting the 'attr' to an optional did do
the job of creating a serial queue successfully. Thanks for that. I
will create a PR for this.
But this makes me think as to why until now, this was not caught by
the compiler itself?
Regards
Mamatha
- Original message -
From: phaus...@apple.com
To: Mamatha Busi/India/IBM@IBMIN
Cc: swift-corelibs-dev
Subject: Re: [swift-corelibs-dev] Setting property
"maxConcurrentOperationCount = 1" of NSOperation fails to execute
operations using OpenSource Foundation.
Date: Thu, May 5, 2016 10:54 PM
Hmm that seems unfortunate. I wonder if the serial creation is due
to an unwrapped optional? var attr: dispatch_queue_attr_t? instead
might do the trick… or alternatively we could just let the
underlying queue be concurrent all the time and enforce the max ops
via making the semaphore always instantiated (in the case of max ops
being 1) and initializing it to 1 to gate the operations.
On May 5, 2016, at 1:15 AM, Mamatha Busi via
swift-corelibs-dev wrote:
Hello
Code snippet:
———
let operation1 : NSBlockOperation = NSBlockOperation
(block: {
sleep(1)
print("Opertion1")
})
let operation2 : NSBlockOperation = NSBlockOperation
(block: {
sleep(1)
print("Opertion2”)
})
var operations = [NSOperation]()
operations.append(operation1)
operations.append(operation2)
let queue = NSOperationQueue()
queue.maxConcurrentOperationCount = 1
queue.addOperations(operations, waitUntilFinished:
true)
The above code snippet of adding operations to an operation
queue and executing with the property
‘maxConcurrentOperationCount = 1’ fails while executing the
above with the OpenSource Foundation and libDispatch of MAC
inside Xcode.
The error I am seeing is:
fatal error: unexpectedly found nil while unwrapping an
Optional value
Stack trace points to: attr = DISPATCH_QUEUE_SERIAL
which implies that the libDispatch macro is coming as nil
during the creation of the serial queue using libDispatch in
the file NSOperationQueue
The same test-case passes on OSx.
When I do not restrict the serial operation i.e. I remove ‘
queue.maxConcurrentOperationCount = 1’ , test case executes
successfully.
Setting it to a different value other than 1 also causes no
problems.
Am I doing something wrong with the API. If I set the property
‘ queue.maxConcurrentOperationCount = 1’ after adding
operations to the queue, then the problem does not occur. But
then this restricts me to control the operation execution to
be serial. So, I expect the above snippet to work on
OpenSource as well. Any thoughts on this?
Thank you.
Regards
Mamatha
___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift
Re: [swift-corelibs-dev] libdispatch on Linux, or C++ std::async?
Hi, my guess is you missed the git submodule init and update step in setting up your libdispatch checkout. If the libpwq directory is empty, this is certainly what happened (and is why it isn't invoking auotconf recursively on libpwq). Buried at the bottom of the INSTALL file: Initialize git submodules: We are using git submodules to incorporate a specific revision of the upstream pthread_workqueue library into the build. git submodule init git submodule update --dave From: Brian Gesiak via swift-corelibs-dev To: Chris Bailey Cc: Swift Core Libs Date: 05/30/2016 11:51 AM Subject:Re: [swift-corelibs-dev] libdispatch on Linux, or C++ std::async? Sent by:swift-corelibs-dev-boun...@swift.org Thanks for the quick responses! I had tried the experimental/foundation branch as well, with no luck. Here's the full output from a clean build I just attempted: https://gist.github.com/modocache/fcf0c78908b39834448c73ac3f50c110 Prior to the build I removed my entire build directory (which included both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build artifacts from my libdispatch directory (I used `git clean -fxd` -- Make tells me there's no target `distclean`). Looking at the build failure, it appears to be the same one that was reported in https://bugs.swift.org/browse/SR-1242. The author of that report indicates that https://github.com/apple/swift-corelibs-libdispatch/pull/64 fixes the problem. My checkout includes the commits from that pull request, but the problem remains. In any case, thanks for all your help! I'll keep trying. Any and all advice would be much appreciated! - Brian Gesiak On Mon, May 30, 2016 at 10:22 AM, Chris Bailey wrote: I'd certainly recommend picking up the experimental/foundation branch as Rob recommends. Its where we've been doing some additional work to get properly built into the Swift toolchain and to open up some internal APIs needed by Foundation. Whilst I've been building Dispatch either independently or as part of a full toolchain using the buildbot_linux preset (with dispatch and install-dispatch added), I've just checked your build line and its working fine for me. It might be worth cleaning out any partial build you might have of libdispatch in your build directory, and running 'make distclean' from your libdispatch source directory. FYI, we've got some further changes to the experimental/foundation branch of libdispatch with should land in the next couple of days. This fixes some issues with the Swift overlay, and resolves a crash that we've found in the underlying libkqueue implementation. Chris From: Rob Allen via swift-corelibs-dev < swift-corelibs-dev@swift.org> To: Brian Gesiak Cc: Swift Core Libs Date: 30/05/2016 08:08 Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++ std::async? Sent by: swift-corelibs-dev-boun...@swift.org Hi, For what it's worth, I can't get the master branch to build either, but I can get the experimental/foundation branch to build. Regards, Rob On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote: Hello all! I'm working on https://bugs.swift.org/browse/SR-710, which involves porting SourceKit to Linux. SourceKit contains a handful of references to libdispatch symbols. I have two questions: 1. Could someone share a set of instructions to build swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have installed [the dependencies listed here]( https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html ). I have tried using both `swift/utils/build-script --libdispatch` as well as the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both fail with the [same errors]( https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11). Any tips? 2. I don't know much about C++, but from what I understand the C++ stdlib provides tools for asynchronous programming. I list SourceKit's uses of libdispatch in a comment on https://bugs.swift.org/browse/SR-1639. For those familiar with both C++ and libdispatch: do you think it's a good idea to migrate to C++, at least for now? Any advice would be appreciated. Thanks!! - Brian Gesiak -- Development thoughts at http://akrabat.com Daily Jotter for Mac OS X at http://dailyjotter.com ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev ___ swift-corelibs-dev mailing list swift-corelibs
Re: [swift-corelibs-dev] libdispatch/GCD for Swift 3.0 on Linux
Hi Joe, TL;DR: right now the Swift 3.0 overlay for libdispatch is not available on Linux. But, a variety of activities are kicking off to get it there. There should be visible progress (and opportunities to help if you are so inclined) over the next few weeks. > > While working to convert some NSThread code over to GCD on Linux I > noticed that (to my dismay) GCD wasn't included in the Swift 3.0 > preview build, nor is it apart of the buildbot for Linux. Scouring > through the repository and mailing lists I can see where it is > possible to build, but there are no downstream instructions on how > to make a line like this: > > let queue = DispatchQueue(label: "com.app.queue") > > work with the swiftc compiler on Linux. > > A couple of questions: > > * are there any timelines that the principals of GCD with Swift 3 on > Linux can share; i.e., is the plan to have it included in time for > the Swift 3 GA release later this year? I certainly hope so; but cannot speak for official plans. > * are there any follow-up instructions past getting it built on how > to use it (or will make install DESTDIR place it in the right spots > in the Swift 3 toolchain hierarchy?) > I expect when things are done, it will be similar to how building the current Swift-2 dispatch overlay works on Linux. To build from source you use swift/utils/build-script with --libdispatch and it takes care of all the details (ie, it just works...). --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch/GCD for Swift 3.0 on Linux
Joseph Bell wrote on 06/21/2016 09:15:00 AM: > > Thanks for the details, I appreciate it. I have seen the term > "Swift overlay" used, particularly in the context of libdispatch. > What does that mean exactly in this regard (searching for it returns > tutorials on overlay UIViews which I doubt is appropriate here). > Hi, There's a layer of Swift code that sits on top of the non-Swift implementation of libdispatch to provide the Swift-level API for the library. This is called the overlay. In Swift 2, the overlay for libdispatch was relatively thin. In Swift 3 it became thicker and on Darwin platforms more reliant on compiler support for importing Objective-C API declarations in a "Swifty" way. The main work item for getting the libdispatch Swift 3 APIs on Linux is to compensate for the lack of Objective-C by manually writing a layer (in Swift) that sits between the basic C-level APIs libdispatch provides on Linux and the desired user-visible Swift-3 APIs. --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch/GCD for Swift 3.0 on Linux
Hi, I've been making some progress on the Swift 3 overlay for dispatch on a branch [1]. It builds, but doesn't do much more beyond that yet. I expect to raise a pull request within a week (once basic programs are working). --dave [1] https://github.com/dgrove-oss/swift-corelibs-libdispatch/tree/wrapping-overlay-stage-1 From: Joseph Bell To: David P Grove/Watson/IBM@IBMUS, Chris Bailey Cc: swift-corelibs-dev@swift.org Date: 06/26/2016 05:30 PM Subject:Re: [swift-corelibs-dev] libdispatch/GCD for Swift 3.0 on Linux David, Chris, Thanks again for the responses regarding libdispatch (GCD) on Linux with Swift 3.0. I took a stab at building what was there and indeed, I see there are missing components to the overlay. For example, I see where Dispatch.swift contains things like "public extension DispatchGroup" but there is no actual DispatchGroup defined anywhere (just extensions to it). The same goes for DispatchSemaphore, DispatchQueue, DispatchWorkItem, etc. Unless I am offbase and that is defined (I certainly couldn't find it). At any rate, thanks again for the work you're doing bringing GCD to Linux; I'm looking forward to it. Joe On Tue, Jun 21, 2016 at 10:24 AM, David P Grove wrote: Joseph Bell wrote on 06/21/2016 09:15:00 AM: > > Thanks for the details, I appreciate it. I have seen the term > "Swift overlay" used, particularly in the context of libdispatch. > What does that mean exactly in this regard (searching for it returns > tutorials on overlay UIViews which I doubt is appropriate here). > Hi, There's a layer of Swift code that sits on top of the non-Swift implementation of libdispatch to provide the Swift-level API for the library. This is called the overlay. In Swift 2, the overlay for libdispatch was relatively thin. In Swift 3 it became thicker and on Darwin platforms more reliant on compiler support for importing Objective-C API declarations in a "Swifty" way. The main work item for getting the libdispatch Swift 3 APIs on Linux is to compensate for the lack of Objective-C by manually writing a layer (in Swift) that sits between the basic C-level APIs libdispatch provides on Linux and the desired user-visible Swift-3 APIs. --dave -- Joseph Bell http://dev.iachieved.it/iachievedit/ @iachievedit ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] Troubleshooting libdispatch build failure
Hi, There has been some churn over the last few days, but I am successfully using util/build-script to build libdispatch with the current tip of the master branch of libdispatch (and a swift checkout of the master branch from yesterday morning). Maybe updating the checkout of libdispatch would help? --dave From: Jean-Pierre Simard To: swift-corelibs-dev@swift.org Cc: David P Grove/Watson/IBM@IBMUS Date: 07/06/2016 04:28 PM Subject:Troubleshooting libdispatch build failure Up until a few days ago, running 'util/build-script --libdispatch' completed successfully. Now it fails, generating the following errors. Any ideas on how to resolve these? Or other ways people are using to build libdispatch at the moment? Thanks! /bin/bash ../libtool --tag=CXX --mode=compile /build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang++ -DHAVE_CONFIG_H -I. -I//swift-corelibs-libdispatch/src -I../config -I.. -I//swift-corelibs-libdispatch -I//swift-corelibs-libdispatch/private -DDISPATCH_USE_DTRACE=0 -I//swift-corelibs-libdispatch/libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer -I/usr/include/kqueue -isystem /usr/include/bsd -DLIBBSD_OVERLAY -fblocks -std=gnu++11 -fno-exceptions -g -O2 -c -o swift/libdispatch_la-DispatchStubs.lo `test -f 'swift/DispatchStubs.cc' || echo '//swift-corelibs-libdispatch/src/'`swift/DispatchStubs.cc libtool: compile: /build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang++ -DHAVE_CONFIG_H -I. -I//swift-corelibs-libdispatch/src -I../config -I.. -I//swift-corelibs-libdispatch -I//swift-corelibs-libdispatch/private -DDISPATCH_USE_DTRACE=0 -I//swift-corelibs-libdispatch/libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer -I/usr/include/kqueue -isystem /usr/include/bsd -DLIBBSD_OVERLAY -fblocks -std=gnu++11 -fno-exceptions -g -O2 -c //swift-corelibs-libdispatch/src/swift/DispatchStubs.cc -fPIC -DPIC -o swift/.libs/libdispatch_la-DispatchStubs.o /build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc -frontend -c //swift-corelibs-libdispatch/src/swift/Block.swift //swift-corelibs-libdispatch/src/swift/Data.swift //swift-corelibs-libdispatch/src/swift/Dispatch.swift //swift-corelibs-libdispatch/src/swift/IO.swift //swift-corelibs-libdispatch/src/swift/Private.swift //swift-corelibs-libdispatch/src/swift/Queue.swift //swift-corelibs-libdispatch/src/swift/Source.swift //swift-corelibs-libdispatch/src/swift/Time.swift //swift-corelibs-libdispatch/src/swift/Wrapper.swift -primary-file //swift-corelibs-libdispatch/src/swift/Block.swift \ -Xcc -fmodule-map-file=//swift-corelibs-libdispatch/dispatch/module.modulemap -I//swift-corelibs-libdispatch -Xcc -fblocks -module-name Dispatch -module-link-name dispatch \ -o /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o -emit-module-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.~partial.swiftmodule \ -emit-module-doc-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.~partial.swiftdoc -emit-dependencies-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.d \ -emit-reference-dependencies-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.swiftdeps \ -module-cache-path .. :1:10: note: in file included from :1: #include "dispatch.h" ^ //swift-corelibs-libdispatch/dispatch/dispatch.h:99:10: error: 'os/object.h' file not found #include ^ //swift-corelibs-libdispatch/src/swift/Block.swift:13:8: error: could not build Objective-C module 'CDispatch' import CDispatch ^ Makefile:956: recipe for target '/build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o' failed ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] Troubleshooting libdispatch build failure
ok, thanks for the details. I'll grab the latest myself tomorrow morning and see if I can reproduce. --dave From: Jean-Pierre Simard To: David P Grove/Watson/IBM@IBMUS Cc: swift-corelibs-dev@swift.org Date: 07/06/2016 06:21 PM Subject:Re: Troubleshooting libdispatch build failure Thanks for getting back to me so quickly Dave. This worked last week, but didn't yesterday when I tried again. My "last few days" guess might actually just be since yesterday morning. Maybe something's changed since your last checkouts from yesterday morning? Running these commands on a fresh Ubuntu 16.04 machine (i.e. no build cache) gives me the output I shared above. apt-get update && \ apt-get install -y \ autoconf \ clang \ cmake \ git \ icu-devtools \ libblocksruntime-dev \ libbsd-dev \ libedit-dev \ libicu-dev \ libkqueue-dev \ libncurses5-dev \ libpython-dev \ libsqlite3-dev \ libtool \ libxml2-dev \ ninja-build \ pkg-config \ python \ swig \ uuid-dev git clone https://github.com/apple/swift.git cd swift utils/update-checkout --clone utils/build-script --libdispatch I suspect that the Xcode 8 beta 2 set of changes that landed yesterday may have caused this. There were patches on all sorts of repos (swift-llvm, swift-clang, swift, etc). On Wed, Jul 6, 2016 at 4:52 PM, David P Grove wrote: Hi, There has been some churn over the last few days, but I am successfully using util/build-script to build libdispatch with the current tip of the master branch of libdispatch (and a swift checkout of the master branch from yesterday morning). Maybe updating the checkout of libdispatch would help? --dave Inactive hide details for Jean-Pierre Simard ---07/06/2016 04:28:46 PM---Up until a few days ago, running 'util/build-script --Jean-Pierre Simard ---07/06/2016 04:28:46 PM---Up until a few days ago, running 'util/build-script --libdispatch' completed successfully. From: Jean-Pierre Simard To: swift-corelibs-dev@swift.org Cc: David P Grove/Watson/IBM@IBMUS Date: 07/06/2016 04:28 PM Subject: Troubleshooting libdispatch build failure Up until a few days ago, running 'util/build-script --libdispatch' completed successfully. Now it fails, generating the following errors. Any ideas on how to resolve these? Or other ways people are using to build libdispatch at the moment? Thanks! /bin/bash ../libtool --tag=CXX --mode=compile /build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang++ -DHAVE_CONFIG_H -I. -I//swift-corelibs-libdispatch/src -I../config -I.. -I//swift-corelibs-libdispatch -I//swift-corelibs-libdispatch/private -DDISPATCH_USE_DTRACE=0 -I//swift-corelibs-libdispatch/libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer -I/usr/include/kqueue -isystem /usr/include/bsd -DLIBBSD_OVERLAY -fblocks -std=gnu++11 -fno-exceptions -g -O2 -c -o swift/libdispatch_la-DispatchStubs.lo `test -f 'swift/DispatchStubs.cc' || echo '//swift-corelibs-libdispatch/src/'`swift/DispatchStubs.cc libtool: compile: /build/Ninja-DebugAssert/llvm-linux-x86_64/bin/clang++ -DHAVE_CONFIG_H -I. -I//swift-corelibs-libdispatch/src -I../config -I.. -I//swift-corelibs-libdispatch -I//swift-corelibs-libdispatch/private -DDISPATCH_USE_DTRACE=0 -I//swift-corelibs-libdispatch/libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer -I/usr/include/kqueue -isystem /usr/include/bsd -DLIBBSD_OVERLAY -fblocks -std=gnu++11 -fno-exceptions -g -O2 -c //swift-corelibs-libdispatch/src/swift/DispatchStubs.cc -fPIC -DPIC -o swift/.libs/libdispatch_la-DispatchStubs.o /build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc -frontend -c //swift-corelibs-libdispatch/src/swift/Block.swift //swift-corelibs-libdispatch/src/swift/Data.swift //swift-corelibs-libdispatch/src/swift/Dispatch.swift //swift-corelibs-libdispatch/src/swift/IO.swift //swift-corelibs-libdispatch/src/swift/Private.swift //swift-corelibs-libdispatch/src/swift/Queue.swift //swift-corelibs-libdispatch/src/swift/Source.swift //swift-corelibs-libdispatch/src/swift/Time.swift //swift-corelibs-libdispatch/src/swift/Wrapper.swift -primary-file //swift-corelibs-libdispatch/src/swift/Block.swift \ -Xcc -fmodule-map-file=//swift-corelibs-libdispatch/dispatch/module.modulemap -I//swift-corelibs-libdispatch -Xcc -fblocks -module-name Dispatch -module-link-name dispatch \ -o /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o -emit-module-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.~partial.swiftmodule \ -emit-module-doc-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.~partial.swiftdoc -emit-dependencies-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src/swift/Block.o.d \ -emit-reference-dependencies-path /build/Ninja-DebugAssert/libdispatch-linux-x86_64/src
Re: [swift-corelibs-dev] Wrapping up Swift 3 for swift-corelibs
Tony Parker wrote on 07/28/2016 01:41:55 PM: > > 1. Integrate swift-corelibs-dispatch into Foundation. Hi Tony, Hopefully this is on the task list already, but if it isn't we should add it before it gets to be too late to change the compiler... When compiling a Swift program on Linux that imports Dispatch (or Foundation once the integration is done), the user has to give the extra compilation flags -Xcc -fblocks to enable block support. We really need to land a change somewhere so that either (1) blocks support is always on for Linux or (2) importing Dispatch or Foundation automatically turns on blocks support. I have some time today and tomorrow that I could use to work on this if no one is handling it already, but I'm not sure how best to tackle the problem. Suggestions? --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] Wrapping up Swift 3 for swift-corelibs
Bug filed: https://bugs.swift.org/browse/SR-2309 I've prototyped potential solution #2 this morning (embedding yet another copy of the blocks runtime into libdispatch).I need to do a little cleanup of the build plumbing, but it is workable. I have time to work on it this week once there is a consensus on the approach. --dave From: Daniel Dunbar To: Chris Bailey Cc: David P Grove/Watson/IBM@IBMUS, "swift-corelibs-dev@swift.org" Date: 08/03/2016 02:10 PM Subject:Re: [swift-corelibs-dev] Wrapping up Swift 3 for swift-corelibs Sent by:daniel_dun...@apple.com Can someone file a bug to track how we are going to resolve this issue, and outline our options? I want to make sure we have enough eyes on it to make sure we get what we need for Swift 3. - Daniel On Aug 1, 2016, at 3:08 AM, Chris Bailey wrote: I had a quick conversation with Daniel Dunbar on this. Adding -Xcc -fblocks as a default option to Swift PM (on Linux). The wrinkle is the availability of the blocks runtime - it would either need to become a pre-req for Swift, or needs one to be built into the package. Chris From: David P Grove via swift-corelibs-dev < swift-corelibs-dev@swift.org> To:Swift corelibs dev Date:28/07/2016 18:50 Subject:Re: [swift-corelibs-dev] Wrapping up Swift 3 for swift-corelibs Sent by:swift-corelibs-dev-boun...@swift.org Tony Parker wrote on 07/28/2016 01:41:55 PM: > > 1. Integrate swift-corelibs-dispatch into Foundation. Hi Tony, Hopefully this is on the task list already, but if it isn't we should add it before it gets to be too late to change the compiler... When compiling a Swift program on Linux that imports Dispatch (or Foundation once the integration is done), the user has to give the extra compilation flags -Xcc -fblocks to enable block support. We really need to land a change somewhere so that either (1) blocks support is always on for Linux or (2) importing Dispatch or Foundation automatically turns on blocks support. I have some time today and tomorrow that I could use to work on this if no one is handling it already, but I'm not sure how best to tackle the problem. Suggestions? --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] libdispatch Arch Linux
Ryan Lovelett wrote on 08/14/2016 05:10:34 PM: > > However, the `dispatch_io` test is still failing. I've added the > test-suite.log once more. > Hi Ryan, The test that is failing creates a temporary directory, sets it to be read-only, and then attempts to write into the directory. It is expecting the write to fail with errno value EACESS (13). Instead, the write attempt is succeeding (errno is 0). I'd guess that either you are running the test as root/sudo or the chmod operation on the temporary directory to make it read-only failed. --dave [BEGIN] error from write to write protected directory Actual: 0 Expected: 13Permission denied [FAIL] error from write to write protected directory (dispatch_io.c:611) dispatch_io.c:611 ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift libdispatch Incremental RA - Ubuntu 14.04 (master) #183
I've never seen this test fail before. We haven't changed the kqueue git submodule version since July 20. It looks like the next 20 runs of this CI build didn't hit this test failure either. I took a quick scan of the kqueue tests and like some of the other libdispatch test cases, the kqueue test is not bulletproof if the system on which it is run is under very unusual load. For example, there are tests of timer events and if the expected timers are more than a second late, the testcase is marked as a failure. Fairly reasonable for a testcase run on a development machine before committing code. Not great for running hundreds of times in a less controlled environment. We will get occasional spurious failures. As part of getting the dispatch tests running in the CI environment, we've turned off about 20% of the test cases because they were having non-reproducible occasional failures on the CI machines. This probably ties back to the discussion a few days ago on this list about the lldb tests. What's the best way for us to structure testcases that test end-to-end properties or timeliness expectations for highly concurrent code that won't cause unacceptable levels of spurious failures in the CI environment? --dave From: "Daniel A. Steffen" To: Jordan Rose , David P Grove/Watson/IBM@IBMUS, Hubertus Franke/Watson/IBM@IBMUS Cc: swift-corelibs-dev , swift-dev Date: 08/24/2016 10:11 PM Subject:Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift libdispatch Incremental RA - Ubuntu 14.04 (master) #183 Sent by:dstef...@apple.com looks like a libkqueue test failed dave / hubertus may have seen this before ? make check-TESTS make[2]: Entering directory `/home/buildnode/disk2/workspace/oss-swift-incremental-RA-libdispatch-linux-ubuntu-14_04/Ninja-ReleaseAssert/libdispatch-linux-x86_64/libkqueue' make[3]: Entering directory `/home/buildnode/disk2/workspace/oss-swift-incremental-RA-libdispatch-linux-ubuntu-14_04/Ninja-ReleaseAssert/libdispatch-linux-x86_64/libkqueue' FAIL: kqtest On Aug 24, 2016, at 16:05, Jordan Rose via swift-dev < swift-...@swift.org> wrote: Not it. Dispatch folks, any idea what's up? On Aug 24, 2016, at 16:04, no-re...@swift.org wrote: [FAILURE] oss-swift-incremental-RA-libdispatch-linux-ubuntu-14_04 [#183] Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-libdispatch-linux-ubuntu-14_04/183/ Project: oss-swift-incremental-RA-libdispatch-linux-ubuntu-14_04 Date of Wed, 24 Aug 2016 15:36:44 -0700 build: Build28 min duration: Changes Commit 9fab7cb15f5584ce49b49f73e0e8ac277d7c2192 by jordan_rose: Fix s390x Enum load/storeMultiPayloadValue (#4461) edit: stdlib/public/runtime/Enum.cpp ___ swift-dev mailing list swift-...@swift.org https://lists.swift.org/mailman/listinfo/swift-dev ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
Re: [swift-corelibs-dev] [swift-dev] Swift CI PR builder dispatch linux failure
The order may need to vary by platform. On Linux, the DIspatch Swift
overlay code lives in swift-corelibs-libdispatch/src/swift. So it works to
build (1) Swift (2) libdispatch (both C and Swift) (3) Foundation.
--dave
From: Jordan Rose
To: David P Grove/Watson/IBM@IBMUS
Cc: "Daniel A. Steffen" , swift-dev
, swift-corelibs-dev
Date: 09/23/2016 06:34 PM
Subject:Re: [swift-dev] Swift CI PR builder dispatch linux failure
Sent by:jordan_r...@apple.com
I think the right order to build things is:
1. libdispatch (C)
2. Swift (compiler + stdlib + Dispatch overlay)
3. Foundation
Otherwise we need to build Swift, then build libdispatch, then go back to
"Swift" to build the overlay, and only finally get to Foundation.
Jordan
On Sep 23, 2016, at 14:32, David P Grove via swift-dev <
swift-...@swift.org> wrote:
I'm playing with something. Should have a PR ready shortly.
--dave
"Daniel A. Steffen via swift-dev" ---09/23/2016 04:45:06
PM---I don’t know how these dependencies are expressed in CMake but
it doesn’t seem very difficult to do
From: "Daniel A. Steffen via swift-dev"
To: Ted Kremenek
Cc: swift-dev
Date: 09/23/2016 04:45 PM
Subject: Re: [swift-dev] Swift CI PR builder dispatch linux failure
Sent by: swift-dev-boun...@swift.org
I don’t know how these dependencies are expressed in CMake but it
doesn’t seem very difficult to do so in the autotools buildsystem, we
have
AC_ARG_WITH([swift-toolchain],
[AS_HELP_STRING([--with-swift-toolchain], [Specify path to Swift
toolchain])],
[swift_toolchain_path=${withval}
AC_DEFINE(HAVE_SWIFT, 1, [Define if building for Swift])
SWIFTC="$swift_toolchain_path/bin/swiftc”
in configure.ac which ends up as a SWIFTC makefile variable, so as
long as that argument is passed as an absolute path, it should work
as a dependency in the build rules
$(abs_builddir)/swift/%.o: $(abs_srcdir)/swift/%.swift
$(SWIFTC) -frontend -c $(SWIFT_ABS_SRC_FILES) -primary-file $< \
Is there a JIRA ticket for this ? the IBM folks should probably be
the ones making these changes since we still don’t have the whitelist
in place for Apple contributors to do so…
Daniel
On Sep 23, 2016, at 13:00, Ted Kremenek <
kreme...@apple.com> wrote:
+ Daniel
Daniel: how hard would it be for the Dispatch project to
be modified to add the swiftc binary used to build the
project as a dependency?
On Sep 23, 2016, at 12:55 PM, Ted Kremenek
via swift-dev wrote:
Dispatch is a separate project. There isn’t a
dependency in the Dispatch project to tell it
is building with a new compiler. The project
would need to be modified to add a dependency
on the swiftc binary being used to compile
this project.
On Sep 21, 2016, at 9:39 AM,
Mishal Shah via swift-dev <
swift-...@swift.org> wrote:
We should look into why it's not
rebuilding dependencies. Do we
need to fix CMake logic? Can you
please file a bug?
Thanks,
Mishal Shah
On Sep 21, 2016, at 9:28 AM,
Michael Gottesman <
mgottes...@apple.com> wrote:
+CC Mishal
On Sep
21, 2016,
at 6:59
AM, David
P Grove
via
swift-dev
<
swift-...@swift.org
> wrote:
I don't
Re: [swift-corelibs-dev] [swift-dev] Swift CI PR builder dispatch linux failure
We think that the libdispatch incremental rebuilds didn't know that if
swiftc changed it needed to invalidate the compilation of the swift
dispatch overlay files (missing dependency in the make rule). A localized
change that could be enough to fix the problem is libdispatch PR #178. If
that isn't enough, we might have to do a more ambitious restructuring of
the libdispatch build.
--dave
From: Jordan Rose
To: David P Grove/Watson/IBM@IBMUS
Cc: "Daniel A. Steffen" , swift-dev
, swift-corelibs-dev
Date: 09/26/2016 01:07 PM
Subject:Re: [swift-dev] Swift CI PR builder dispatch linux failure
Sent by:jordan_r...@apple.com
Oh, I didn’t realize we had a separate copy of the overlay code (almost
certainly the right thing to do at this point). But in that case, why are
we seeing any of these errors?
Jordan
On Sep 25, 2016, at 11:38, David P Grove wrote:
The order may need to vary by platform. On Linux, the DIspatch Swift
overlay code lives in swift-corelibs-libdispatch/src/swift. So it
works to build (1) Swift (2) libdispatch (both C and Swift) (3)
Foundation.
--dave
Jordan Rose ---09/23/2016 06:34:46 PM---I think the
right order to build things is: 1. libdispatch (C)
From: Jordan Rose
To: David P Grove/Watson/IBM@IBMUS
Cc: "Daniel A. Steffen" , swift-dev <
swift-...@swift.org>, swift-corelibs-dev <
swift-corelibs-dev@swift.org>
Date: 09/23/2016 06:34 PM
Subject: Re: [swift-dev] Swift CI PR builder dispatch linux failure
Sent by: jordan_r...@apple.com
I think the right order to build things is:
1. libdispatch (C)
2. Swift (compiler + stdlib + Dispatch overlay)
3. Foundation
Otherwise we need to build Swift, then build libdispatch, then go
back to "Swift" to build the overlay, and only finally get to
Foundation.
Jordan
On Sep 23, 2016, at 14:32, David P Grove via swift-dev <
swift-...@swift.org> wrote:
I'm playing with something. Should have a PR ready
shortly.
--dave
"Daniel A. Steffen via swift-dev"
---09/23/2016 04:45:06 PM---I don’t know how these
dependencies are expressed in CMake but it doesn’t seem
very difficult to do
From: "Daniel A. Steffen via swift-dev" <
swift-...@swift.org>
To: Ted Kremenek
Cc: swift-dev
Date: 09/23/2016 04:45 PM
Subject: Re: [swift-dev] Swift CI PR builder dispatch
linux failure
Sent by: swift-dev-boun...@swift.org
I don’t know how these dependencies are expressed in
CMake but it doesn’t seem very difficult to do so in the
autotools buildsystem, we have
AC_ARG_WITH([swift-toolchain],
[AS_HELP_STRING([--with-swift-toolchain], [Specify path
to Swift toolchain])],
[swift_toolchain_path=${withval}
AC_DEFINE(HAVE_SWIFT, 1, [Define if building for Swift])
SWIFTC="$swift_toolchain_path/bin/swiftc”
in configure.ac which ends up as a SWIFTC makefile
variable, so as long as that argument is passed as an
absolute path, it should work as a dependency in the
build rules
$(abs_builddir)/swift/%.o: $(abs_srcdir)/swift/%.swift
$(SWIFTC) -frontend -c $(SWIFT_ABS_SRC_FILES)
-primary-file $< \
Is there a JIRA ticket for this ? the IBM folks should
probably be the ones making these changes since we still
don’t have the whitelist in place for Apple contributors
to do so…
Daniel
On Sep 23, 2016, at 13:00, Ted
Kremenek
wrote:
+ Daniel
Daniel: how hard would it be for
the Dispatch project to be
modified to add the swiftc binary
used to build the project as a
dependency?
On Sep
23, 2016,
at 12:55
PM, Ted
Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568
I think the root problem is that some of the dispatch tests, including this one, are trying to test that timers/events happen within a "reasonable" interval of time from when they are supposed to. If deadlines are missed too badly, then the test is marked as failing. If we completely disabled all such tests, we run the risk of breaking the timer/event subsystem of libdispatch and not noticing. However, on the CI system there is unpredictable load, so "reasonable" is hard to specify and with some probability, deadlines will be missed no matter how loosely they are set. If this test is failing frequently and the infrastructure does not support rerunning or ignoring a flagged "flakey" test then disabling it is probably the expedient option. Probably what we need to do is strip down to (1) a very reliable subset of the dispatch tests that are run on most CI jobs and a (2) more comprehensive set of tests that is run in a separate job just for libdispatch testing. Job (2) can afford to include tests with low probability failures because only the dispatch team would need to pay attention to them and decide if they were real failures or not. Is something like this possible to set up? --dave From: Doug Coleman via swift-corelibs-dev To: Jordan Rose Cc: swift-dev , swift-corelibs-dev Date: 11/02/2016 01:48 PM Subject:Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568 Sent by:swift-corelibs-dev-boun...@swift.org It has also been reported as a dup here: https://bugs.swift.org/browse/SR-3110 It’s important that we fix this one so we stop wasting five minutes of many peoples’ time several times a day. Maybe we should disable it until it’s fixed? On Nov 2, 2016, at 10:46 AM, Jordan Rose wrote: Adding swift-corelibs-dev, since that’s where the Dispatch folks hang out. On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev < swift-...@swift.org> wrote: https://bugs.swift.org/browse/SR-2931 On Nov 2, 2016, at 10:43 AM, no-re...@swift.org wrote: [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568] Build URL: https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ Project: oss-swift-incremental-RA-linux-ubuntu-15_10 Date of Wed, 02 Nov 2016 10:17:41 -0700 build: Build25 min duration: Tests: Name: Swift(linux-x86_64) Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s) Name: Swift-Unit Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s) Changes Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift: [swiftc (94 vs. 5180)] Add crasher in ? add: validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift ___ swift-dev mailing list swift-...@swift.org https://lists.swift.org/mailman/listinfo/swift-dev ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://l
Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Still Failing: 0. OSS - Swift Incremental RA - Ubuntu 16.10 (master) #3698
swift-dev-boun...@swift.org wrote on 05/17/2017 02:04:23 PM: > > Failure in dispatch-starfish. This test looks timing-sensitive. > Boris, is this unlikely to be related to your changes? Does anyone > familiar with dispatch have suggestions, such as increasing the > latency threshold? > > [BEGIN] Latency > Actual: 1553 > Expected: <1000 > [FAIL] Latency (dispatch_starfish.c:82) > dispatch_starfish.c:82 > lap: 9 > count: 1000 > delta: 675640 ns > math: 3374.825175 ns / lap > > [BEGIN] Latency > Actual: 3374 > Expected: <1000 > [FAIL] Latency (dispatch_starfish.c:82) > dispatch_starfish.c:82 > The test is subject to failing spuriously if the machine is badly overloaded. One could increase the threshold by changing the #define on line 43 of dispatch_starfish.c There's a trade-off between spurious failures due to overloads on build machines vs. missing performance regressions by making the latency thresholds much too lenient. We've been running with the current threshold in dispatch_starfish since we began running the dispatch tests in the CI system last summer. So if this isn't a one-off failure caused by CI load, it might be pointing to an underlying performance regression. --dave ___ swift-corelibs-dev mailing list swift-corelibs-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
