Re: [DISCUSS] Asynchronous SQLAlchemy

Ah that makes sense thank you. I've not tried 1000s (isn't the default limit 1000 tasks per triggerer process?) but I have not seen any performance issues yet on our vms scaling up yet, tested up to nearly 100 AsyncSqlSensors plus 200 other sensors. Damian From:

Re: [DISCUSS] Asynchronous SQLAlchemy

What you propose is exactly what `sync_to_async` does implicitly (check source code ), and as we mentioned before, this solution would work, but coroutines are lightweight compared to threads,

Re: [VOTE] Airflow Providers prepared on April 10, 2024

+1 non binding. All AWS system tests are running successfully against apache-airflow-providers-amazon==8.20.0rc1 with the exception of example_bedrock that is failing due to a bug in the test itself (fix here: https://github.com/apache/airflow/pull/38887). You can see the results here: https://

[VOTE] Airflow Providers prepared on April 10, 2024

Hey all, I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 72 hours - which means that it will end on April 13, 2024 16:35 PM UTC and until 3 binding +1 votes have been received. Consider this my (binding) +1. Airflow Provi

RE: [DISCUSS] Asynchronous SQLAlchemy

I know this is a different use case, but I maintain an internal Airflow provider that creates a SQLAlchemy connection type so I can create an AsyncSqlSensor that uses SQLAlchemy to manage the connections in the triggerer, (I keep a global engine per connection id so that there can be a global c