> We currently guarantee that the minimum Airflow version supported by a provider is the release date of the next minor version, plus 12 months. You’re asking if this should be adjusted for Airflow 3, correct?
Yes > Are you asking for guarantees the other way around, so Airflow version -> guaranteed provider package min version support? Yes. On Wed, Aug 14, 2024 at 12:04 PM Bas Harenslak <b...@astronomer.io.invalid> wrote: > Sorry I’m a bit lost in the long text. Is my understanding of the 2 > questions correct? > > We currently guarantee that the minimum Airflow version supported by a > provider is the release date of the next minor version, plus 12 months. > You’re asking if this should be adjusted for Airflow 3, correct? > Are you asking for guarantees the other way around, so Airflow version -> > guaranteed provider package min version support? > > Bas > > > On 14 Aug 2024, at 11:22, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > Hello everyone, > > > > I have two important topics to discuss regarding Provider <> Airflow > > version support rules. > > > > *1) What is the "min Airflow version" support while we are transitioning > to > > Airflow 3? * > > > > So far we had the [1] rule: > > > >> The default support timespan for the minimum version of Airflow (there > > could be justified exceptions) is that we increase the minimum Airflow > > version to the next MINOR release, when 12 months passed since the first > > release for the MINOR version of Airflow. > > > >> For example this means that by default we upgrade the minimum version of > > Airflow supported by providers to 2.8.0 in the first Provider's release > > after 18th of December 2024. 18th of December 2023 is the date when the > > first PATCHLEVEL of 2.8 (2.8.0) has been released. > > > > The next cutoff time (2.9.0 + 12 months) will be in April 2025, And the > > next one - 2.10.0 - assuming we release 2.10.0 in August - for min > airflow > > version = 2.10.0 will be in August 2025. If we have "bridge" 2.11.0 > release > > (very likely) - say in December, then min_version = 2.11.0 will be in > > December 2025. > > > > Assuming that we release Airflow 3.0.0 in March 2025, if we follow this > > rule, we will be able to drop Airflow 2 support in providers in March > 2026 > > - so we give about a year to Airflow 2 users to migrate to Airflow 3 - > if > > they will want new features and bug-fixes from new providers. Also it > means > > that we have 1.5 year of supporting both Airflow 2 and Airflow 3 in > > providers (that for example includes the old Templated Field > > back-compatibility support assuming that we will want to go ahead with > the > > change) > > > > That provides (as was the original idea of this rolling min-version > > support) additional incentive for users to migrate (want new features > from > > providers - migrate to latest Airflow). With Airflow 3 that will be a > more > > difficult decision to migrate, so I wonder if this rule should be > somewhat > > adapted. > > > > *2) Should we introduce a rule for a "rolling" min version Provider's > > support in Airflow 3?* > > > > In Airflow 2 we never had a rule describing "how old" providers are still > > supported in newer versions of Airflow. For example we never said you > need > > to have at least "2.10.0" version of a google provider. This was never a > > huge problem, because vast majority of users were upgrading providers > with > > Airflow and only occasionally downgraded providers or kept them with old > > versions because of some breaking changes.That had never caused a big > > problem as far as I know and remember - and we can safely assume that > > "really old" providers might not work with "new" airflow even if we had > no > > formal rule for it. > > > > However that required some back-compatibility code sometimes - like this > > code removal that started breaking our main after removing old "imports" > > [2]. This means that some very old SB providers will not work with the > new > > common.sql provider when released (and they also will not work with > Airflow > > 3). I think it will never cause a "real" serious issue - but would be > great > > to formalize it to be able to set expectations of our users and have some > > ways to respond to potential issues they might raise. > > > > Should we formalize it somehow? For example add a rule that providers > will > > be guaranteed to support Airflow versions released up to 12 months after > > the provider was released? We could potentially add tests to check it in > > the future (though it would be rather complex and time consuming). But I > > think we should start with the rule first. > > > > Example to illustrate it (hypothetical): > > > > * Google provider 11.0.4 is released in December 2024 - just before > > Airflow 2.11.0 > > * Airflow 3.0 - 3.4 are released between December 2024 and December 2025 > - > > all of them support google provider 2.11.0 (and if not - this is a bug > that > > needs to be fixed) > > * Airflow 3.5 is released in January 2026 - and if someone wants to > install > > Airflow 3.5 and use Google provider 11.0.4 or below - they might, or > might > > not work. But if they don't we do not treat it as a bug and we recommend > > the users to upgrade to a newer Google provider version. > > > > I think we should not make "intentional" changes to break those > providers, > > unless we have good reasons, and it's mostly about some weird edge cases, > > where people have new Airflow and old providers, but I think it's worth > > introducing and documenting some rules about it. > > > > > > Let me know what you think on both accounts. > > > > J. > > > > [1] > > > https://github.com/apache/airflow/blob/main/PROVIDERS.rst#upgrading-minimum-supported-version-of-airflow > > [2] https://github.com/apache/airflow/pull/41461 > >
