Hi all,
In https://github.com/apache/airflow/pull/67947#discussion_r3569126675
Ash suggested taking the naming question raised there to the dev list
for a wider discussion. Starting that thread here, since it currently
slows down the whole `tasks` command family — and since renames are
only cheap while we are on 0.x. The sooner a rule is agreed, the less
inconsistency friction accumulates for contributors and early adopters
alike, and the smoother the eventual path to 1.0.
**The question**
airflowctl currently has no written rule for how user-facing command
groups and names are chosen. Two principles are competing in practice,
and I'd like us to pick one:
- **Option A — REST resource alignment**: commands are named after the
Public API resource they operate on, using the plural URL segment as
the group (`POST /dags/{id}/dagRuns` → `dagruns trigger`).
- **Option B — airflow CLI parity**: any command that replaces an
`airflow` CLI remote command keeps its old group + name (`airflow
tasks clear` → `airflowctl tasks clear`); commands with no old-CLI
counterpart fit into the same old-CLI-style groups.
The merged surface today mixes both: `dags trigger` follows B
(deliberately moved out of the `dagrun` group after the 1.0.0rc2 vote
feedback, although it calls `POST /dags/{id}/dagRuns`), while `dagrun
get/list/delete` roughly follows A — with a singular group name that
matches neither the URL segment (`/dagRuns`) nor any old-CLI group.
Meanwhile `dagrun list` and the old CLI's `dags list-runs` name the
same thing differently, and neither was chosen on purpose.
**Why decide now**
Seven open PRs adding task-level commands (#67947, #69320, #69366,
#69397, #66852, #66213, #68776) each had to pick names independently:
some would surface these commands as `tasks ...`, others as
`taskinstances ...`. The diffs conflict with each other, and in #67947
the same code was renamed back and forth between reviews pulling in
opposite directions ("match core CLI naming" vs "these are
TaskInstance operations"). Three of these PRs are approved and
mergeable but stalled. Every future command family will replay this
argument until a rule is written down.
>From a user's perspective (I run Airflow in production and am
evaluating airflowctl for remote task-instance operations — I filed
#69947 for `tasks logs`): today you cannot predict whether the command
you need will be called `tasks clear` or `taskinstances clear`.
**The two options side by side**
```
operation | A: REST resources | B: airflow CLI parity
---------------------------+----------------------------+----------------------------------
trigger a dag run | dagruns trigger | dags trigger
(today)
list runs of a dag | dagruns list | dags
list-runs (old CLI)
get one dag run | dagruns get | dags state
(today)
delete a dag run | dagruns delete | needs a new
name under `dags`
list tasks of a dag | tasks list | tasks list
all TI states of a run | taskinstances list | tasks
states-for-dag-run
clear task instances | taskinstances clear | tasks clear
single TI state | taskinstances get | tasks state
task instance logs (new) | taskinstances logs | tasks logs
TI failed dependencies | taskinstances failed-deps | tasks failed-deps
```
**Why I would not formalize the middle path**
There is a third possible outcome: codify what current practice has
drifted into — migrated commands keep their old-CLI names (B),
API-native commands follow REST (A). I want to argue explicitly
against this hybrid, even though it looks like the path of least
resistance, because I think it is the worst of the three for the long
run:
- It makes names unpredictable *by design*. To guess a command's name
you first have to know whether it happened to exist in the
pre-airflowctl CLI — that is knowledge about the tool's history, not
about the task at hand. Existing users get partial familiarity; new
users (eventually the majority) get no benefit at all, yet pay the
full cost of two conventions.
- It splits one resource across groups. Task-instance operations would
land partly under `tasks` (the migrated clear/state) and partly under
whatever the REST rule produces (e.g. a new logs command) — related
commands stop being discoverable side by side in `--help`.
- It never converges. As the old CLI is deprecated and new API
capabilities keep arriving, the REST-named share of the surface keeps
growing around a frozen old-CLI core. B's familiarity benefit decays
to zero once the `airflow` CLI is gone, but the inconsistency it bakes
in is permanent. There is no future release in which the command
surface ever looks uniform again.
- It doesn't even end the per-PR debates. "Did this exist in the old
CLI?" is blurrier than it sounds (`dags list-runs` vs `dagrun list` is
the same feature in two shapes), so contributors and reviewers still
have to argue category membership for every new command family — which
is exactly the friction we are trying to remove.
- And it compounds the maintenance burden: docs, the `airflow ↔
airflowctl` mapping table, help texts and any lint/prek enforcement
all have to encode two conventions plus an exception list, forever.
Today's confusion is not the absence of the hybrid — it is the
hybrid's output. Whichever of A or B people prefer, I believe one
principle applied everywhere is worth more than either principle's
specific merits.
**Compatibility, if Option A wins**
A implies renaming the commands that currently follow B (`dags
trigger`, `dags state`), plus fixing `dagrun` → `dagruns`. I would
lean against adding any compatibility shim for the old names:
airflowctl is a new tool on 0.x — breaking changes are exactly what
this window is for — and AIP-94 itself only requires *functional*
parity plus a documented `airflow ↔ airflowctl` command mapping, not
identical names. A clearly documented mapping table (which AIP-94's
migration tooling plans anyway) is enough for users to migrate by, and
it keeps the command surface clean and the long-term maintenance
burden low.
**Either way**
Whichever option is chosen: write it into the contributing docs,
enforce it with a prek check where possible, and rename the existing
non-compliant commands while we are still on 0.x.
I'm happy to turn the outcome into the docs PR plus the corresponding
rename PRs. Curious what people think — especially Bugra and Ash,
whose review positions in #67947 this is meant to unblock.
Kind regards,
Bob Du
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
