Apologies if my earlier message wasn't clear! What I was trying to convey is that, since we're discussing pool and deferrable tasks. Yes, there's also been some discussion around managing deferred tasks at the DAG level, which might be relevant here. The PR I mentioned (https://github.com/apache/airflow/pull/41560) aims to introduce the ability to control deferred tasks on the DAG level, as highlighted in issue #40528. I wanted to bring this up because it might impact or complement the decision we're making on whether tasks should always take up a pool slot or not during deferral.
In short, my intention was to share this related work for consideration, as it might influence how we think about modeling deferrable tasks and pool management. Sorry if you think this is irrelevant here please ignore this. Regards, Pavan On Wed, Oct 9, 2024 at 11:31 PM Daniel Standish <daniel.stand...@astronomer.io.invalid> wrote: > Pavan, I didn't understand... If you have a position on what I am > proposing, can you try to clarify it? Or maybe you are just trying to let > people know that there's a PR that is related to the general topic of > deferrables and concurrency? >
