Random question - could Github be reached out to for the limits to be increased/investigated in the mean time, given this is an opensource project?
On Wed, 6 Jan 2021 at 06:57, Andrew Lamb <al...@influxdata.com> wrote: > Makes sense -- thank you > > On Tue, Jan 5, 2021 at 5:38 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > There are a lot of companies now with a business interest in the success > of > > Apache Arrow. We should do a back of the envelope estimate of the number > of > > compute hours we use per year and arrive at an annual cost estimate. My > > guess (and maybe I’m wildly off) is that it’s somewhere around > > $50,000-100,000/year, which shouldn’t be difficult to come by. Costs > could > > also be mitigated by organizations hooking up physical on-prem build > > machines. > > > > On Tue, Jan 5, 2021 at 4:29 PM Andrew Lamb <al...@influxdata.com> wrote: > > > > > One thing that is not clear to me is who will pay for the dedicated > > > infrastructure costs with something like Build Kite? > > > > > > I don't fully understand how moving to self hosted (and thus also > > > presumably limited) arrow-only infrastructure will reduce build queue > > > times, unless there is a deep pocketed sponsor. I can certainly see how > > > moving to self hosted infrastructure will increase the amount of > control > > / > > > visibility the project would have > > > > > > Andrew > > > > > > > > > > > > On Tue, Jan 5, 2021 at 12:46 PM Jorge Cardoso Leitão < > > > jorgecarlei...@gmail.com> wrote: > > > > > > > Krisztián, > > > > > > > > I agree with you that there is an ongoing problem with the queue. > > Thanks > > > > for the tips wrt to WIP. > > > > > > > > Wes, I would be up for moving Rust workflows to Buildkite. Is the > > > > integration in place wrt to reporting to triggers and reporting back > to > > > > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is > > there > > > > any PMC-specific activity that is blocking us from working on this? > > > > > > > > Best, > > > > Jorge > > > > > > > > > > > > > > > > > > > > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <wesmck...@gmail.com> > > wrote: > > > > > > > > > At the risk of sounding like a broken record -- we are almost > > > > > certainly going to have to move our builds to dedicated > > infrastructure > > > > > that this community has complete agency over sometime between now > and > > > > > 2025. Maybe it will be this year, maybe next year, but to me it is > an > > > > > inevitability. I spent quite a bit of time getting Apache Infra to > > > > > enable Buildkite for apache/* repositories precisely so that we > have > > a > > > > > coherent path. I hope to be able to have folks on my team focus on > > > > > this at some point and contribute it to the community once we have > > the > > > > > bandwidth. > > > > > > > > > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs > > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > > > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <jeroeno...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs > > > > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I'm concerned about the overall feedback time we have on pull > > > > > requests. > > > > > > > > I have a simple PR to make the comment bot working again, but > > no > > > > > > > > builds are running even after 30 minutes. > > > > > > > > > > > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have > > the > > > > > > > same problem sometimes with pull requests taking long to build, > > but > > > > > > > this usually happens when there are large jobs running under my > > > name > > > > > > > in completely unrelated projects. Also note that if you use > e.g. > > > > > > > scheduled (cron) actions, then the jobs are executed in the > queue > > > of > > > > > > > the github user who last touched the yml file for that > workflow. > > > > > > Based on the "Actions" tab the queueing seems chronological to > me, > > > > > > although I can imagine the behaviour you described. > > > > > > > > > > > > Has anyone experienced similar clogging on apache/arrow's gha > queue > > > > > lately? > > > > > > > > > > > > > > >
