Hello everyone, I have two important topics to discuss regarding Provider <> Airflow version support rules.
*1) What is the "min Airflow version" support while we are transitioning to Airflow 3? * So far we had the [1] rule: > The default support timespan for the minimum version of Airflow (there could be justified exceptions) is that we increase the minimum Airflow version to the next MINOR release, when 12 months passed since the first release for the MINOR version of Airflow. > For example this means that by default we upgrade the minimum version of Airflow supported by providers to 2.8.0 in the first Provider's release after 18th of December 2024. 18th of December 2023 is the date when the first PATCHLEVEL of 2.8 (2.8.0) has been released. The next cutoff time (2.9.0 + 12 months) will be in April 2025, And the next one - 2.10.0 - assuming we release 2.10.0 in August - for min airflow version = 2.10.0 will be in August 2025. If we have "bridge" 2.11.0 release (very likely) - say in December, then min_version = 2.11.0 will be in December 2025. Assuming that we release Airflow 3.0.0 in March 2025, if we follow this rule, we will be able to drop Airflow 2 support in providers in March 2026 - so we give about a year to Airflow 2 users to migrate to Airflow 3 - if they will want new features and bug-fixes from new providers. Also it means that we have 1.5 year of supporting both Airflow 2 and Airflow 3 in providers (that for example includes the old Templated Field back-compatibility support assuming that we will want to go ahead with the change) That provides (as was the original idea of this rolling min-version support) additional incentive for users to migrate (want new features from providers - migrate to latest Airflow). With Airflow 3 that will be a more difficult decision to migrate, so I wonder if this rule should be somewhat adapted. *2) Should we introduce a rule for a "rolling" min version Provider's support in Airflow 3?* In Airflow 2 we never had a rule describing "how old" providers are still supported in newer versions of Airflow. For example we never said you need to have at least "2.10.0" version of a google provider. This was never a huge problem, because vast majority of users were upgrading providers with Airflow and only occasionally downgraded providers or kept them with old versions because of some breaking changes.That had never caused a big problem as far as I know and remember - and we can safely assume that "really old" providers might not work with "new" airflow even if we had no formal rule for it. However that required some back-compatibility code sometimes - like this code removal that started breaking our main after removing old "imports" [2]. This means that some very old SB providers will not work with the new common.sql provider when released (and they also will not work with Airflow 3). I think it will never cause a "real" serious issue - but would be great to formalize it to be able to set expectations of our users and have some ways to respond to potential issues they might raise. Should we formalize it somehow? For example add a rule that providers will be guaranteed to support Airflow versions released up to 12 months after the provider was released? We could potentially add tests to check it in the future (though it would be rather complex and time consuming). But I think we should start with the rule first. Example to illustrate it (hypothetical): * Google provider 11.0.4 is released in December 2024 - just before Airflow 2.11.0 * Airflow 3.0 - 3.4 are released between December 2024 and December 2025 - all of them support google provider 2.11.0 (and if not - this is a bug that needs to be fixed) * Airflow 3.5 is released in January 2026 - and if someone wants to install Airflow 3.5 and use Google provider 11.0.4 or below - they might, or might not work. But if they don't we do not treat it as a bug and we recommend the users to upgrade to a newer Google provider version. I think we should not make "intentional" changes to break those providers, unless we have good reasons, and it's mostly about some weird edge cases, where people have new Airflow and old providers, but I think it's worth introducing and documenting some rules about it. Let me know what you think on both accounts. J. [1] https://github.com/apache/airflow/blob/main/PROVIDERS.rst#upgrading-minimum-supported-version-of-airflow [2] https://github.com/apache/airflow/pull/41461
