t4n1o commented on issue #19192: URL: https://github.com/apache/airflow/issues/19192#issuecomment-1003724797
I'm not sure how airflow is intended to be useful, but sometimes people find other use cases for a tool they haven't designed. We run a task that can take a few hours to collect all the historical data and process it. And then we want the task to run once per day. It appears, from my side, that the airflow server UI can't contact the scheduler while the long task is running, and other DAGs can't be run. Perhaps the scheduler wants my code to yield control back to it frequently (once per day of data, for example), but I prefer to let my own code manage the date ranges, because that's where the unit tests are, and all the heavy lifting is in rust anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
