Hi Ryan, No worry, Pools can be adjusted during runtime. We also have such requirement today. We have an automation (better to say… DAG!) that changes pool sizes based on the time of day via API. I think there is no special complexity needed in core for this (at least not in scope of this consensus) here.
Jens Sent from my Smartphone > On 21. Oct 2024, at 22:51, Ryan Hatter <ryan.hat...@astronomer.io.invalid> > wrote: > > I could imagine a use case where pools are far more important during > business hours and allowing backfills to ignore pools outside business > hours would be very useful. > >> On Mon, Oct 21, 2024 at 3:58 PM Daniel Standish >> <daniel.stand...@astronomer.io.invalid> wrote: >> >> This was discussed in open source slack and I have documented the >> discussion on github here >> https://github.com/apache/airflow/discussions/43235. >> >> Formally, the proposal here is, in new airflow 3.0 backfill (AIP-78), we >> will not provide a mechanism to override the pool set on the dag. At >> least, for now we won't -- not that we can't change minds in future. >> >> Some background. Pre-airflow 3.0, backfill actually *completely >> ignored *pools. >> And it *only* scheduled tasks in a specific pool if you provided an >> override when running backfill command. And in that case, all tasks in the >> dag would be given that pool. >> >> If no one objects, this vote will pass 3 days from now, 1pm pacific >> Thursday Oct 24, 2024. >> >> Ordinarily I would have created a discuss thread first, but since there >> seemed to be unanimity on the side of removing pool override as an option, >> I went straight to vote. >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
