Possibly late but here's me - +1 (binding) on spread(across=N).
I put this in the discussion thread by mistake on Wednesday, so repeating it here where it counts. - On the keyword *across=* over *across_n_tasks=*. - On the feature name, I'd also suggest we don't change it and keep it like earlier. Thanks & Regards, Amogh Desai On Fri, Sep 25, 2026 at 1:01 AM constance.astronomer.io via dev < [email protected]> wrote: > +1 (non-binding) on spread.(across=N) > > I actually don’t think you need to change the name of the feature to Task > Spreading though, it’s fine as is. There’s plenty of times where the > feature name and the action or verb itself is named something different. > > Oh, then: > > distribute (Ash’s new one) > scatter > parallelize > … with a strong > dislike of unfold > shard > batch > partition. > > You’re doing a great job wrangling cats :) > > > > > On Sep 24, 2026, at 3:13 AM, Shahar Epstein <[email protected]> wrote: > > +1 (binding) > > > > spread > scatter > unfold > partition > shard > batch > > > > > > Shahar > > > > On Thu, Sep 24, 2026 at 10:01 AM Blain David <[email protected]> > > wrote: > > > >> Hi all, > >> > >> The discussion thread [1] about the name of the method that distributes > an > >> iterable over a fixed number of mapped task instances brought up several > >> candidates without converging on one, so I am opening a vote to settle > it > >> before PR #62922 [2] merges. > >> What the method does > >> `task.<name>(N).iterate(url=urls)` creates N mapped task instances > through > >> Dynamic Task Mapping, and each instance iterates over its share of > `urls` > >> in a single task instance using Iterable Tasks. Items are distributed > round > >> robin: item `i` goes to task instance `i % N`. N is the number of task > >> instances, not a chunk length. That is why `batch(size=N)` was found > >> misleading: it reads like `itertools.batched`, which does the opposite > >> (chunk length given, number of chunks derived). > >> Candidates > >> In the order they came up in the discussion, with the spelling each > would > >> take: > >> > >> 1. > >> batch - `.batch(tasks=N)` "Task Batching" (original name in the AIP) > >> 2. > >> spread - `.spread(across=N)` "Task Spreading" (Shahar, Constance; > >> currently on the AIP page) > >> 3. > >> scatter - `.scatter(across=N)` "Task Scattering" (Jarek) > >> 4. > >> unfold - `.unfold(into=N)` "Task Unfolding" (Jarek) > >> 5. > >> shard - `.shard(into=N)` "Task Sharding" > >> 6. > >> partition - `.partition(into=N)` "Task Partitioning" > >> > >> The keyword spellings are indicative. The vote is about the verb; the > >> exact keyword is settled in the PR once the verb is chosen. > >> Arguments raised so far > >> > >> * > >> `batch` and `shard` are semantically overloaded (Shahar). Naming it > after > >> * > >> `itertools.batched` is confusing because the semantics are the opposite > >> (Ash). > >> * > >> `shard` and `partition` carry data-engineering connotations that suggest > >> contiguous > >> * > >> ranges as much as `batch` does. Partitions are now an Airflow feature, > so > >> `partition` > >> would introduce a name clash (Constance). > >> * > >> `spread` is hard to misread because of the preposition: "spread across > 17" > >> cannot be > >> * > >> read as "17 each" (Constance). > >> * > >> Jarek pointed out that the stdlib settled on a positional `n` for > >> `batched(it, n)`, > >> * > >> which is a precedent to match or consciously break if `batch` is kept. > >> > >> Regardless of the outcome > >> > >> * > >> The parameter is keyword-only. `spread(17)` is as ambiguous as > `batch(17)` > >> and is rejected. > >> * > >> Passing `size=` raises a clear error instead of being accepted silently. > >> * > >> The docs explain the round-robin distribution and the reason for it. > >> * > >> The AIP-104 page [3], the Task SDK docs and PR #62922 are updated to the > >> winning name > >> * > >> before the PR merges. > >> > >> How to vote > >> Reply to this thread with your preferences ranked from most to least > >> preferred, for example: > >> > >> spread > scatter > batch > >> > >> You do not have to rank all six. The result is decided by instant > runoff: > >> the option with the fewest first preferences is eliminated and its votes > >> move to the next preference on each ballot, until one option holds a > >> majority. > >> > >> Everyone is welcome to vote. Votes from committers and PMC members are > >> binding. > >> The vote stays open for at least 72 hours and closes on Monday 28 > >> September 2026 at 12:00 CEST. I will post the result in this thread. > >> > >> [1] https://lists.apache.org/thread/6qlbvvnt76514rstzk447pvs4784wcj2 > >> [2] https://github.com/apache/airflow/pull/62922 > >> [3] > >> > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/421954527/AIP-104+Iterable+Tasks+IT+and+Task+Spreading+TS > >> > >> Thanks, > >> David > >> > >> > >> General (Internal Property) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
