Hello everyone,

I had a short discussion with Elad and unless there is any objection, we
decided to bring back support for "min-airlfow-version" support for
pre-installed providers (common.sql, ftp, imap, http, sqlite). We will have
to release them ad-hoc now and `yank` the latest versions of those released
yesterday,

PR here: https://github.com/apache/airflow/pull/31469

Some context:

In the last wave of providers we bumped min-airlfow-version to 2.4 and
added mechanism to verify min-airflow version is ok while importing, but it
turned out that there are cases where installing just old version of
airflow (with no constraints) brings the latest version of those providers
and causes new installation of airflow to fail. This is far too common to
ignore or require to use constraints, unfortunately

We do not have min-airlfow-version in the preinstalled providers for one
reason only. For some tools that are NOT conforming to standards (such as
bazel), having min-airflow-version for those providers causes circular
dependency (even though technically dependencies in PyPI can - and often
are - circular, because dependencies in Python are not DAG and can contain
cycles.

This was in response to this issue in 2021
https://github.com/apache/airflow/issues/17795

However - this is really a bazel issue, and on top of it - it's recognized
as such and being fixed very recently in
https://github.com/bazelbuild/rules_python/issues/1188 because there are
other packages that have similar problems (pytorch and triton being
popular). Also Bazel is not that popular in the Python world.

Therefore - rather than trying to workaround the problem of bazel, we will
encourage them to merge and release the fix (I already did
https://github.com/bazelbuild/rules_python/pull/1166#issuecomment-1558690218)
and call it out in our installation instructions, that bazel installation
might lead to problems like that.

If bazel does not fix it, this will only be a problem for Future
installations of airflow in a few months most likely. It will not impact
current bazel users installing old versions of Airflow (actually they might
start having problems now if we do not fix it and yank the 5 providers
released yesterday)

Please let us know if you have any comments, and shout if you think this is
a bad idea.


J.

Reply via email to