The GitHub Actions job "Tests" on airflow.git/trigger-comms-paths has failed. Run started by GitHub user ashb (triggered by ashb).
Head commit for run: 38c3c691713155bf07be6cf016fb50f514b641f8 / Ash Berlin-Taylor <[email protected]> Change Trigger<->Supervisor communication to avoid mixing messages The TriggerRunnerSupervisor previoulsy sent messges to the Triggerer directly when a trigger needed to be created or cancelled, and this initially worked fine, but when we later added the ability for Triggers to access Connections and Variables this ended up causing problems. The issue was that we broke one of the "rules" of comms, which is even documents as such in `airflow/sdk/exeuction_time/comms.py`: > * No messages are sent to task process except in response to a request. > (This is because the task process will be running user's code, so we can't > read from stdin until we enter our code, such as when requesting an XCom ) value etc.) The net result of this was that since the supervisor sent messages un-solicited, if things were timed badly, then the trigger would send a request for a Conn, and the supervisor would reply with a trigger to create etc. This sort of instability was very likely the cause of our flakeyness in the `test_trigger_can_access_variables_connections_and_xcoms` test. This fixes things by making the TriggerRunnerSupervisor collect the triggers to create/cancel in a deqeue in memory, and respond to the TriggerStateChanges messages with TriggerStateSync with the info it would have sent on an as-needed basis. In order to make sure that we only send+receive one message at a time I have swapped from the `threading.lock` to `aiologic.Lock` which allows for better locking/waiting behaviour between threads (which is what Triggers will use via the `sync_to_async`) and the main TriggerRunner. Report URL: https://github.com/apache/airflow/actions/runs/14312848363 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
