Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Krisztián Szűcs
We can reduce the number of builds further by running them as cron tasks. In the beginning I'd rather keep them to keep regressions earlier. The PR has an approval, so I'll merge it tomorrow morning (CET) if there are no objections. Thanks, Krisztian On Mon, Nov 11, 2019 at 12:26 PM Wes McKinney

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Krisztián Szűcs
Note that 20 builds are running on master only, including windows, macos, and documentation builds. On Mon, Nov 11, 2019, 12:33 PM Antoine Pitrou wrote: > > Ah, no, sorry, I read wrongly. It's 20 builds total. That's still a > lot, IMO. > > Regards > > Antoine. > > > Le 11/11/2019 à 12:32, Ant

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Antoine Pitrou
Ah, no, sorry, I read wrongly. It's 20 builds total. That's still a lot, IMO. Regards Antoine. Le 11/11/2019 à 12:32, Antoine Pitrou a écrit : > > A C++ change will end up triggering 38 builds (C++, Python, Ruby, R)... > > Regards > > Antoine. > > > Le 11/11/2019 à 12:25, Wes McKinney

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Antoine Pitrou
A C++ change will end up triggering 38 builds (C++, Python, Ruby, R)... Regards Antoine. Le 11/11/2019 à 12:25, Wes McKinney a écrit : > That's too many for C++ and Python changes in my opinion. Let's focus > on getting the PR merged and then we can address this problem > > On Mon, Nov 11, 2

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Wes McKinney
That's too many for C++ and Python changes in my opinion. Let's focus on getting the PR merged and then we can address this problem On Mon, Nov 11, 2019 at 3:21 AM Krisztián Szűcs wrote: > > It depends on the change: > > - [C++]: 20 > > https://github.com/kszucs/arrow/commit/5654da07c21a7cf6a0

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-11 Thread Krisztián Szűcs
It depends on the change: - [C++]: 20 https://github.com/kszucs/arrow/commit/5654da07c21a7cf6a0e08894f4bf4e7aaa70ed26/checks?check_suite_id=304713473 - [Python]: 8 https://github.com/kszucs/arrow/commit/5084c23f4d8b0fe05f1056808e7066aadbf0033e/checks?check_suite_id=304716138 - [Ruby]: 5 h

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-08 Thread Wes McKinney
Just to be sure, if this PR is merged, how many GHA tasks will be run on each commit to master? On Fri, Nov 8, 2019 at 12:07 PM Krisztián Szűcs wrote: > > I've trimmed down the number of triggered builds on pull requests by > converting them to run on master only or cron builds. Alsoe added > the

Re: [CI] Docker-compose refactor and GitHub Actions

2019-11-08 Thread Krisztián Szűcs
I've trimmed down the number of triggered builds on pull requests by converting them to run on master only or cron builds. Alsoe added the action filters including the changed path patterns. I've also collected ~30 follow up JIRAs aggregating the problems I came across during the refactor and possi

Re: [CI] Docker-compose refactor and GitHub Actions

2019-10-31 Thread Wes McKinney
hi Krisz -- I just left comments on the PR. This definitely looks like a good step forward. My main comment is that I think there are too many C++/Python tasks to run on an each-commit basis. Ideally many of these would be run nightly. There is also a certain amount of redundancy in rebuilding the

[CI] Docker-compose refactor and GitHub Actions

2019-10-25 Thread Krisztián Szűcs
Hi, During the release of 0.15.1-RC0 I literally had to wait days to ensure that the Travis, Appveyor and Crossbow builds were all passing for the release branch. Additionally each newly added patch was delaying the process by 8 hrs or so (actually felt like 16). Recently I've been working on to