A C++ change will end up triggering 38 builds (C++, Python, Ruby, R)...
Regards Antoine. Le 11/11/2019 à 12:25, Wes McKinney a écrit : > That's too many for C++ and Python changes in my opinion. Let's focus > on getting the PR merged and then we can address this problem > > On Mon, Nov 11, 2019 at 3:21 AM Krisztián Szűcs > <szucs.kriszt...@gmail.com> wrote: >> >> It depends on the change: >> >> - [C++]: 20 >> >> https://github.com/kszucs/arrow/commit/5654da07c21a7cf6a0e08894f4bf4e7aaa70ed26/checks?check_suite_id=304713473 >> - [Python]: 8 >> >> https://github.com/kszucs/arrow/commit/5084c23f4d8b0fe05f1056808e7066aadbf0033e/checks?check_suite_id=304716138 >> - [Ruby]: 5 >> >> https://github.com/kszucs/arrow/commit/e90e92514f69b0734e8126a949858476bfb70c1a/checks?check_suite_id=304717369 >> - [JS]: 6 >> >> https://github.com/kszucs/arrow/commit/f27d6500f628c152c732f2372beae15a34f088e9/checks?check_suite_id=304718837 >> - [R]: 5 >> >> https://github.com/kszucs/arrow/commit/eaec66cbde923687beebe31114dea6d0410043c6/checks?check_suite_id=304719993 >> - [Rust]: 6 >> >> https://github.com/kszucs/arrow/commit/0c52ad15a54e78fdd1fe797668ebf0cdfbf6f4ab/checks?check_suite_id=304720884 >> - [Go]: 7 >> >> https://github.com/kszucs/arrow/commit/30ab7f5e8cf6373774ce6ea75b0473da5ec808a3/checks?check_suite_id=304722182 >> >> >> On Sat, Nov 9, 2019 at 2:19 AM Wes McKinney <wesmck...@gmail.com> wrote: >>> >>> Just to be sure, if this PR is merged, how many GHA tasks will be run >>> on each commit to master? >>> >>> On Fri, Nov 8, 2019 at 12:07 PM Krisztián Szűcs >>> <szucs.kriszt...@gmail.com> wrote: >>>> >>>> I've trimmed down the number of triggered builds on pull requests by >>>> converting them to run on master only or cron builds. Alsoe added >>>> the action filters including the changed path patterns. I've also >>>> collected ~30 follow up JIRAs aggregating the problems I came across >>>> during the refactor and possible further improvements and optimizations >>>> like caching. >>>> >>>> The PR is ready to be merged [1]. I expect multiple issues after merging >>>> the PR, despite that I was trying to port everything thoroughly. >>>> The changeset is simply too big, but we need to start somewhere, and >>>> sooner is better. >>>> >>>> [1]: https://github.com/apache/arrow/pull/5589 >>>> >>>> On Thu, Oct 31, 2019 at 10:21 PM Wes McKinney <wesmck...@gmail.com> wrote: >>>>> >>>>> hi Krisz -- I just left comments on the PR. This definitely looks like >>>>> a good step forward. My main comment is that I think there are too >>>>> many C++/Python tasks to run on an each-commit basis. Ideally many of >>>>> these would be run nightly. There is also a certain amount of >>>>> redundancy in rebuilding the C++ library multiple times before running >>>>> each dependent set of tests, whereas in Travis we build the C++ >>>>> library once then test both C++ and Python. If there is sufficient >>>>> number of builders then perhaps it doesn't matter so much >>>>> >>>>> It seems there are a few things, like action filtering (similar to >>>>> "detect-changes.py") based on what was changed that would need to get >>>>> done before this can be merged. >>>>> >>>>> - Wes >>>>> >>>>> On Fri, Oct 25, 2019 at 7:25 PM Krisztián Szűcs >>>>> <szucs.kriszt...@gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> During the release of 0.15.1-RC0 I literally had to wait days >>>>>> to ensure that the Travis, Appveyor and Crossbow builds >>>>>> were all passing for the release branch. Additionally each >>>>>> newly added patch was delaying the process by 8 hrs or so >>>>>> (actually felt like 16). >>>>>> >>>>>> Recently I've been working on to incorporate the advantages >>>>>> of the Buildbot setup into our current docker-compose >>>>>> configuration, including support for multiple architectures >>>>>> and platforms, reusing docker images and caching dependency >>>>>> installation steps. It tries to follow the semantics of ursabot, >>>>>> but using only docker-compose and tiny shell scripts. >>>>>> >>>>>> This refactoring also includes GitHub Actions workflows for >>>>>> Windows and macOS as well, reusing the same (bash) builds >>>>>> scripts. The docker configuration and the scripts are CI agnostic. >>>>>> Last but not least, I've managed to clean up a lot of things >>>>>> including every travis builds, and three Appveyor builds. >>>>>> As an example the ci [3] and dev [4] folders got much cleaner. >>>>>> >>>>>> The majority of the builds are passing [2], but due to the size >>>>>> of the pull request [1] reviews for relevant workflows like the >>>>>> JavaScript, C#, Rust, JNI, etc. would be much appreciated. >>>>>> I'll be on vacation until Wednesday, but will try to respond on >>>>>> both GH and the ML. >>>>>> >>>>>> Thanks, Krisztian >>>>>> >>>>>> [1]: https://github.com/apache/arrow/pull/5589 >>>>>> [2]: https://github.com/apache/arrow/runs/275685241 >>>>>> [3]: >>>>>> https://github.com/apache/arrow/tree/9c7e7289b9c9486c13a02e7cb5682a0f9f274ec6/ci >>>>>> [4]: >>>>>> https://github.com/apache/arrow/tree/9c7e7289b9c9486c13a02e7cb5682a0f9f274ec6/dev
