Re: [Hosting] Network Issues

2021-02-09 Thread Lance Albertson
Further updates from the outage from our upstream provider (all times PST): As a precautionary measure against potential equipment failure, Link Oregon > will replace one of its core routers in our Portland Pittock hub starting > on Wednesday (02/10/2021) at 17:30. This change is to remediate rou

Re: GA workflow cancellation

2021-02-09 Thread Jarek Potiuk
I updated the docs slightly: * https://github.com/potiuk/cancel-workflow-runs#the-action-target-workflow -> description about the workflow the action acts on * https://github.com/potiuk/cancel-workflow-runs#tackling-the-high-queue-strain-situation -> I added "Tackling the high queue strain situati

Re: GA workflow cancellation

2021-02-09 Thread Jarek Potiuk
> Er... so "source" workflow is the triggering workflow, and "target" workflow is the cancelling workflow? Or is it something else? I'm afraid all those explanations are a bit confusing to me :-) Precisely. Unfortunately GA's features for the "special" workflows are as powerful as they are compl

Re: [Hosting] Mail Relay Migration (smtp3) - Mon, Feb 8, 2021, 2:15-4:45pm PST (2215-0045 UTC)

2021-02-09 Thread Lance Albertson
It was discovered this morning that a few domains were failing with "Relay access denied". Unfortunately, it looks like we had missed about a half dozen domains when we migrated to the new server. I just deployed a fix for this on smtp3. Apologies if you had any issues with that. We'll continue to

Re: GA workflow cancellation

2021-02-09 Thread Antoine Pitrou
Le 09/02/2021 à 19:28, Jarek Potiuk a écrit : > Yes. I noticed this misleading line and I will update the docs shortly. It > should be Cancels duplicate runs from all running workflow *runs of the > workflow the action acts on*.""" > > The action is written in the way that it always acts on

Re: GA workflow cancellation

2021-02-09 Thread Jarek Potiuk
Yes. I noticed this misleading line and I will update the docs shortly. It should be Cancels duplicate runs from all running workflow *runs of the workflow the action acts on*.""" The action is written in the way that it always acts on a single workflow. By default this is indeed the same wor

Re: GA again unreasonably slow (again)

2021-02-09 Thread Jarek Potiuk
The reasoning for selective checks here: https://github.com/apache/airflow/blob/master/PULL_REQUEST_WORKFLOW.rst (correct link) On Tue, Feb 9, 2021 at 7:05 PM Jarek Potiuk wrote: > | The real hard problem is knowing when a change requires full regression > and integration testing of all possible

Re: GA again unreasonably slow (again)

2021-02-09 Thread Jarek Potiuk
| The real hard problem is knowing when a change requires full regression and integration testing of all possible platforms. And here I absolutely agree too. Even more than that - I am a hard practitioner of that. This is what we already implemented in Airflow (the whole reasoning how and why it i

Re: GA again unreasonably slow (again)

2021-02-09 Thread Dave Fisher
The real hard problem is knowing when a change requires full regression and integration testing of all possible platforms. I think projects are allowing lazy engineering if those making changes don’t know the level of testing needed for their changes. Now with easy lightweight branches all bein

Re: GA again unreasonably slow (again)

2021-02-09 Thread Jarek Potiuk
Absolutely agree Matt. Throwing more hardware at "all of the projects" is definitely not going to help - I was telling that from the beginning - it is like building free motorways - the more you build, the more traffic flows and the traffic jams remain. That's why I think reasonable self-hosted sol

Re: [Hosting] Network Issues

2021-02-09 Thread Lance Albertson
Here is an update from the network issues we were having last night. According to our monitoring everything is back to normal as of the time mentioned below (which is in PST). Please let us know if you're still having issues as of this morning. Thanks for your patience. > > A routing anomaly occu

Re: GA again unreasonably slow (again)

2021-02-09 Thread Matt Sicker
To be honest, this sounds exactly like the usual CI problem on every platform. As your project scales up, CI becomes a Hard Problem. I don’t think throwing hardware at it indefinitely works, though your research here is finding most of the useful things. On Tue, Feb 9, 2021 at 02:21 Jarek Potiuk

Re: GA workflow cancellation

2021-02-09 Thread Antoine Pitrou
Hi Jarek, Le 09/02/2021 à 09:21, Jarek Potiuk a écrit : > And we all together try to help each other (for > example I helped yesterday the Pulsar team to implement most aggressive way > of cancelling their workflows https://github.com/apache/pulsar/pull/9503 > (you can find pretty good explanati

Re: GA again unreasonably slow (again)

2021-02-09 Thread Jarek Potiuk
The report shows only top contenders. And yes - we know it is flawed - because it shows workflows not jobs (if you read the disclaimers - we simply have not enough API calls quota to get detailed information for all projects). So this is anecdotal. I also get no queue when I submit PR at 11 pm. Ac