This is great! well done! On Thu, Apr 23, 2026 at 4:42 PM Amogh Desai <[email protected]> wrote:
> Hey All, > > Wanted to call out a new hook we added to BaseTrigger — `on_kill()`, PR: > https://github.com/apache/airflow/pull/65590. > > *Why does it exist?* > > If your trigger polls an external job (BigQuery, Databricks, Dataproc, > etc.), there was no clean way to cancel that job when > a user kills the deferred task, either by marking it as success, failure, > or clearing it. `cleanup()` wasn't the right place — > it fires on every trigger exit including triggerer restarts and rolling > deploys, so putting cancellation there would kill in-flight > work on every redeploy. > > `on_kill()` solves this. It only fires when a user explicitly acts on the > task. Not on restart, not on redistribution. > > *How it works* > > The triggerer passes a private sentinel via `asyncio.Task.cancel(msg)` to > distinguish user kills from other exits — no DB > round-trip needed. There's a configurable timeout (*[triggerer] > on_kill_timeout, default 30s*) so a slow external API call > cannot block the triggerer. > > Databricks is updated as a reference: > https://github.com/apache/airflow/pull/65672 > > Both `DatabricksExecutionTrigger` and > `DatabricksSQLStatementExecutionTrigger` now implement it. > > *Who benefits from this?* > > If you are someone building or maintaining a trigger that manages external > work, you can consider adopting to this hook > and implementing it for your trigger class, Airflow 3.3 will launch this. > > No compat guard is needed, Airflow < 3.2 the method just never gets called. > > *Next steps* > > I pulled some data from the repo and these are candidates for this > migration. > > Google Provider: DataprocSubmitTrigger and DataprocSubmitJobDirectTrigger > in the google provider. Currently catches > CancelledError in run() and calls hook.cancel_job(job_id) after checking > safe_to_cancel(). > > Amazon Provider: EmrServerlessStartJobTrigger. Same pattern. CancelledError > handler with safe_to_cancel() calling > hook.conn.cancel_job_run(). Straightforward migration. > > CNCF provider: KubernetesPodTrigger in cncf/kubernetes. Has an > on_kill_action parameter controlling whether to > delete/patch the pod. The cancellation logic lives in a CancelledError > handler. Move it to on_kill(). > > Happy to help with reviews for these. > > > Thanks & Regards, > Amogh Desai >
