I could imagine a use case where pools are far more important during business hours and allowing backfills to ignore pools outside business hours would be very useful.
On Mon, Oct 21, 2024 at 3:58 PM Daniel Standish <daniel.stand...@astronomer.io.invalid> wrote: > This was discussed in open source slack and I have documented the > discussion on github here > https://github.com/apache/airflow/discussions/43235. > > Formally, the proposal here is, in new airflow 3.0 backfill (AIP-78), we > will not provide a mechanism to override the pool set on the dag. At > least, for now we won't -- not that we can't change minds in future. > > Some background. Pre-airflow 3.0, backfill actually *completely > ignored *pools. > And it *only* scheduled tasks in a specific pool if you provided an > override when running backfill command. And in that case, all tasks in the > dag would be given that pool. > > If no one objects, this vote will pass 3 days from now, 1pm pacific > Thursday Oct 24, 2024. > > Ordinarily I would have created a discuss thread first, but since there > seemed to be unanimity on the side of removing pool override as an option, > I went straight to vote. >
