Hi all,
The current build process for the overall Swift project (i.e., the compiler +
associated projects like Foundation, XCTest, and SwiftPM) relies on each
project having dependencies on the built artifacts of previously built
projects. Those dependencies are currently communicated to each project in the
chain through an ad hoc set of arguments to the individual project's build
process. This has been painful to maintain, and makes it hard to reason about
the environment that each of the associated projects are building within.
Instead, I would like to move towards what I have been calling a
"toolchain-based" build process.
In this model:
1. The entire build process will be organized as a sequential, incremental
construction of a complete toolchain.
- Each individual project will build and copy its products into a
staging area.
2. The build script will always create a composed toolchain.
- It will start with an empty toolchain, and merge in the content from
each project as it builds.
- This will use rsync & hard-links to avoid needing to stay fast.
3. Each individual project build will just use the composed toolchain to build.
- This will replace the grab bag of options we use to communicate
between the projects.
4. At the end of the build, we will have constructed a complete toolchain which
can be installed (or used with Xcode).
Aside from simplifying the overall build process, this has a couple very nice
upsides:
1. At the end of the build, the user has a complete toolchain. They can install
it, or use it as they would a distributed snapshot. This is very beneficial for
people who are only building the Swift project to get a more recent version of
the compiler.
2. Each individual project can be built using the "official" build process with
a downloaded snapshot (assuming it is of a new enough version). This is very
beneficial for easing contribution to projects like Foundation, XCTest, and
SwiftPM which are pure Swift and have fast build times, but which currently
build in Swift CI using a very different process from the snapshot-based
workflow.
Concrete details:
1. I do not plan to change the actual install process in the short term. The
actual install process used today relies on the CMake-based install process for
some projects (most importantly Swift) and isn't suitable for use in this
fashion (where incremental development speed is of high importance). Instead,
my plan is to teach the build script itself how to assemble the staging area
for each individual project, with the long term goal of using that for the
official install process instead of the CMake-based process.
2. My plan is that the build script would only support building one "primary
product" (i.e. toolchain). That product may itself be a complete cross compiler
toolchain with support for multiple platforms, but the expectation is that
users would invoke the build script multiple times if building multiple
toolchains. However, to support Canadian Cross [1] build scenarios the build
script may need to manage the construction of two products, the primary
toolchain and the intermediate (cross compiling) toolchain used to build the
artifacts in the primary toolchain.
3. As a concrete example of a problem this solves, SwiftPM currently doesn't
test its `swift-test` product in Swift CI, because that product requires using
all of the toolchain stack (swift, Foundation, and XCTest), but the product
itself is only intended to be used as part of a concrete toolchain. As such, it
doesn't know how to operate correctly when given the piecemeal build products
for each of those projects, and teaching it to do so would be very cumbersome
for us to maintain.
Feedback welcome!
- Daniel
[1] https://en.wikipedia.org/wiki/Cross_compiler#Canadian_Cross
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
