Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Jarek Potiuk]

A draft PR here (i will split it as there are few unrelated changes)
https://github.com/apache/airflow/pull/49103 -> but it shows how we can
automatically set the min versions in "apache-airflow" meta-package (with
6-months-old provider versions).

That will quite likely help to get the "Resolution too deep" error solved,
and I think it has a nice property - that we can treat it as "recommended"
range of versions for providers - while giving all the flexibility to the
user to keep their old versions or downgrade.


J.

On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk  wrote:

> Actually what I see when I attempted to do it - is that even if we
> introduce it - it is not going to be a hard requirement. It is just a soft
> limit when installing full apache-airflow[with extras].
>
> It will be more of a default behaviour to limit the providers when extras
> are used but will be easy to override and it will not apply when you
> install airflow-core /task-sdk/ providers directly. So yes - I think no
> real drawback
>
> This limits will only be on extras of the 'apache-airflow' and in case
> someone used the extras so far - the were not able to control the version
> of provider that will be used anyway - and this only used a lower version
> of provider than latest I'd some other package was installed at the same
> time that was conflicting with latest provider..
>
> Thoss limits are not going to be persistent, they are going to simply be
> taken into account when apache-airflow{provider} is used and only for the
> time of the installation. It will not prevent the user from
> upgrading/downgrading the provider later. And also user still be able to
> remove extra and use the provider directly in the same `pip install`
> command
>
> I have a draft change that sets the lower bound to now() -6 months +
> manual overrides if we need to get some new provider limited to latest
> version this way for example. I am still coming back from US and will be
> home later today and try it and I think some form of it will be good to
> have  - 6 months seems reasonable and it will have the 'self-maintenance'
> capability.
>
> J.
>
> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor 
> napisał:
>
>> There absolutely is a downside to the users — “forcing" users to upgrade
>> providers along with Airflow version makes upgrades a much more daunting
>> experience.
>>
>> > On 10 Apr 2025, at 21:31, Maciej Obuchowski 
>> wrote:
>> >
>> > For OpenLineage we'd definitely like something like this.
>> > IMO there is no downside to restricting the providers to the latest
>> > released version.
>> > We could go with the chicken-egg for the next release - that would add
>> at
>> > least one useful
>> > feature authored by Kacper Muda -
>> > https://github.com/apache/airflow/pull/48941
>> > but even without it it's not broken, just missing that feature.
>> >
>> >
>> > czw., 10 kwi 2025 o 21:49 Jarek Potiuk  napisał(a):
>> >
>> >> Accidentally It turned out - with today's investigation of why RC
>> images
>> >> are not building that it is VERY needed. It seems that `pip` resolution
>> >> works currently with our meta-package in a very bad way. If in our
>> >> pyproject.toml we have just
>> >> "amazon' =  "apache-airflow-providers-amazon", "google" =
>> >> "apache-airflow-providers-google" 
>> >>
>> >> then
>> >>
>> >> apache-airflow[google,amazon] - will lead pip to download and try to
>> >> resolve ALL VERSIONS or apparently ALL providers that have ever been
>> >> released. That means we have 100 amazon, 100 google - and if you have
>> more
>> >> extras - 10s or 100s of other providers to consider. And essentially
>> what
>> >> pip tries to do is - try all combinations of those to find out which
>> ones
>> >> are good. Or at least very, very big subset of those providers. Which
>> >> actually explains the "Resolution Too Deep" problem we tried to
>> diagnose
>> >> and investigate over the last few weeks.
>> >>
>> >> So summarising:
>> >> * resolution too deep was actually "real" issue
>> >> * it was caused by no lower-bounds in providers in meta package
>> >> * we will have to set those lower-binds to something reasonable (and I
>> >> might start later today with simply latest released versions if i have
>> >> internet on my flight back from the US)
>> >> * and we will have to keep those min versions updated and bumped
>> >> periodically
>> >>
>> >> The last case is interesting - because it will finally force us to
>> >> introduce some policy on how old providers we support when we release
>> >> airflow. With Airflow 3 I think we are good with basically "latest"
>> >> versions (and maybe get a few versions back for crucial providers).
>> But we
>> >> can discuss it later.
>> >>
>> >> J.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Apr 10, 2025 at 2:49 PM Vincent Beck 
>> wrote:
>> >>
>> >>> I assume this is not necessary when the constraint is already set on
>> the
>> >>> provider side? For example, FAB provider has a dependency on Airflow 3

Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Ash Berlin-Taylor]

Oh hang on wait a moment.

Is aiobotocore a new extra that is not available in 2.10.5 or any early 
versions?

If that is the case then I’d instead vote that we don’t add new extras to 
`apache-airflow`, and we instead remove it. I grudgingly accept that having 
`apache-airflow[amazon]` make sense, but `aiobotocore` isn’t the name of the 
provider, and you need to know what it means anyway, so I don’t think we should 
have it as an extra anywhere other than on the provider.

-ash 

> On 11 Apr 2025, at 08:40, Jarek Potiuk  wrote:
> 
> A draft PR here (i will split it as there are few unrelated changes)
> https://github.com/apache/airflow/pull/49103 -> but it shows how we can
> automatically set the min versions in "apache-airflow" meta-package (with
> 6-months-old provider versions).
> 
> That will quite likely help to get the "Resolution too deep" error solved,
> and I think it has a nice property - that we can treat it as "recommended"
> range of versions for providers - while giving all the flexibility to the
> user to keep their old versions or downgrade.
> 
> 
> J.
> 
> On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk  wrote:
> 
>> Actually what I see when I attempted to do it - is that even if we
>> introduce it - it is not going to be a hard requirement. It is just a soft
>> limit when installing full apache-airflow[with extras].
>> 
>> It will be more of a default behaviour to limit the providers when extras
>> are used but will be easy to override and it will not apply when you
>> install airflow-core /task-sdk/ providers directly. So yes - I think no
>> real drawback
>> 
>> This limits will only be on extras of the 'apache-airflow' and in case
>> someone used the extras so far - the were not able to control the version
>> of provider that will be used anyway - and this only used a lower version
>> of provider than latest I'd some other package was installed at the same
>> time that was conflicting with latest provider..
>> 
>> Thoss limits are not going to be persistent, they are going to simply be
>> taken into account when apache-airflow{provider} is used and only for the
>> time of the installation. It will not prevent the user from
>> upgrading/downgrading the provider later. And also user still be able to
>> remove extra and use the provider directly in the same `pip install`
>> command
>> 
>> I have a draft change that sets the lower bound to now() -6 months +
>> manual overrides if we need to get some new provider limited to latest
>> version this way for example. I am still coming back from US and will be
>> home later today and try it and I think some form of it will be good to
>> have  - 6 months seems reasonable and it will have the 'self-maintenance'
>> capability.
>> 
>> J.
>> 
>> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor 
>> napisał:
>> 
>>> There absolutely is a downside to the users — “forcing" users to upgrade
>>> providers along with Airflow version makes upgrades a much more daunting
>>> experience.
>>> 
 On 10 Apr 2025, at 21:31, Maciej Obuchowski 
>>> wrote:
 
 For OpenLineage we'd definitely like something like this.
 IMO there is no downside to restricting the providers to the latest
 released version.
 We could go with the chicken-egg for the next release - that would add
>>> at
 least one useful
 feature authored by Kacper Muda -
 https://github.com/apache/airflow/pull/48941
 but even without it it's not broken, just missing that feature.
 
 
 czw., 10 kwi 2025 o 21:49 Jarek Potiuk  napisał(a):
 
> Accidentally It turned out - with today's investigation of why RC
>>> images
> are not building that it is VERY needed. It seems that `pip` resolution
> works currently with our meta-package in a very bad way. If in our
> pyproject.toml we have just
> "amazon' =  "apache-airflow-providers-amazon", "google" =
> "apache-airflow-providers-google" 
> 
> then
> 
> apache-airflow[google,amazon] - will lead pip to download and try to
> resolve ALL VERSIONS or apparently ALL providers that have ever been
> released. That means we have 100 amazon, 100 google - and if you have
>>> more
> extras - 10s or 100s of other providers to consider. And essentially
>>> what
> pip tries to do is - try all combinations of those to find out which
>>> ones
> are good. Or at least very, very big subset of those providers. Which
> actually explains the "Resolution Too Deep" problem we tried to
>>> diagnose
> and investigate over the last few weeks.
> 
> So summarising:
> * resolution too deep was actually "real" issue
> * it was caused by no lower-bounds in providers in meta package
> * we will have to set those lower-binds to something reasonable (and I
> might start later today with simply latest released versions if i have
> internet on my flight back from the US)
> * and we will have to keep those min versions updated and bumped
>

Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Ash Berlin-Taylor]

Just checked, no this was in 2.10 releases. NM then.

Still, I think we should not add new extras to `apache-airflow` in general, and 
definitely not for anything other than direct provider “short-names”.


-ash

> On 11 Apr 2025, at 09:02, Ash Berlin-Taylor  wrote:
> 
> Oh hang on wait a moment.
> 
> Is aiobotocore a new extra that is not available in 2.10.5 or any early 
> versions?
> 
> If that is the case then I’d instead vote that we don’t add new extras to 
> `apache-airflow`, and we instead remove it. I grudgingly accept that having 
> `apache-airflow[amazon]` make sense, but `aiobotocore` isn’t the name of the 
> provider, and you need to know what it means anyway, so I don’t think we 
> should have it as an extra anywhere other than on the provider.
> 
> -ash 
> 
>> On 11 Apr 2025, at 08:40, Jarek Potiuk  wrote:
>> 
>> A draft PR here (i will split it as there are few unrelated changes)
>> https://github.com/apache/airflow/pull/49103 -> but it shows how we can
>> automatically set the min versions in "apache-airflow" meta-package (with
>> 6-months-old provider versions).
>> 
>> That will quite likely help to get the "Resolution too deep" error solved,
>> and I think it has a nice property - that we can treat it as "recommended"
>> range of versions for providers - while giving all the flexibility to the
>> user to keep their old versions or downgrade.
>> 
>> 
>> J.
>> 
>> On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk  wrote:
>> 
>>> Actually what I see when I attempted to do it - is that even if we
>>> introduce it - it is not going to be a hard requirement. It is just a soft
>>> limit when installing full apache-airflow[with extras].
>>> 
>>> It will be more of a default behaviour to limit the providers when extras
>>> are used but will be easy to override and it will not apply when you
>>> install airflow-core /task-sdk/ providers directly. So yes - I think no
>>> real drawback
>>> 
>>> This limits will only be on extras of the 'apache-airflow' and in case
>>> someone used the extras so far - the were not able to control the version
>>> of provider that will be used anyway - and this only used a lower version
>>> of provider than latest I'd some other package was installed at the same
>>> time that was conflicting with latest provider..
>>> 
>>> Thoss limits are not going to be persistent, they are going to simply be
>>> taken into account when apache-airflow{provider} is used and only for the
>>> time of the installation. It will not prevent the user from
>>> upgrading/downgrading the provider later. And also user still be able to
>>> remove extra and use the provider directly in the same `pip install`
>>> command
>>> 
>>> I have a draft change that sets the lower bound to now() -6 months +
>>> manual overrides if we need to get some new provider limited to latest
>>> version this way for example. I am still coming back from US and will be
>>> home later today and try it and I think some form of it will be good to
>>> have  - 6 months seems reasonable and it will have the 'self-maintenance'
>>> capability.
>>> 
>>> J.
>>> 
>>> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor 
>>> napisał:
>>> 
 There absolutely is a downside to the users — “forcing" users to upgrade
 providers along with Airflow version makes upgrades a much more daunting
 experience.
 
> On 10 Apr 2025, at 21:31, Maciej Obuchowski 
 wrote:
> 
> For OpenLineage we'd definitely like something like this.
> IMO there is no downside to restricting the providers to the latest
> released version.
> We could go with the chicken-egg for the next release - that would add
 at
> least one useful
> feature authored by Kacper Muda -
> https://github.com/apache/airflow/pull/48941
> but even without it it's not broken, just missing that feature.
> 
> 
> czw., 10 kwi 2025 o 21:49 Jarek Potiuk  napisał(a):
> 
>> Accidentally It turned out - with today's investigation of why RC
 images
>> are not building that it is VERY needed. It seems that `pip` resolution
>> works currently with our meta-package in a very bad way. If in our
>> pyproject.toml we have just
>> "amazon' =  "apache-airflow-providers-amazon", "google" =
>> "apache-airflow-providers-google" 
>> 
>> then
>> 
>> apache-airflow[google,amazon] - will lead pip to download and try to
>> resolve ALL VERSIONS or apparently ALL providers that have ever been
>> released. That means we have 100 amazon, 100 google - and if you have
 more
>> extras - 10s or 100s of other providers to consider. And essentially
 what
>> pip tries to do is - try all combinations of those to find out which
 ones
>> are good. Or at least very, very big subset of those providers. Which
>> actually explains the "Resolution Too Deep" problem we tried to
 diagnose
>> and investigate over the last few weeks.
>> 
>> So summarising:
>> * 

[DISCUSSION] Proposal to Add Couchbase Provider to Apache Airflow

[Shyam Rajamannar]

Couchbase has customers already using Apache Airflow, and having an officially 
supported provider within the ecosystem would simplify integration, improve 
reliability, and encourage adoption.

We previously released 
airflow-providers-couchbase
 through an external repository also listed under airflow ecosystem. However, 
it has been less visible to our customers and the broader Airflow community. 
Contributing this provider directly to the official Airflow ecosystem will 
ensure better visibility, easier adoption, and tighter integration with 
Airflow’s architecture and release processes.

As part of the initial implementation, we have created a Couchbase hook that 
exposes the Couchbase client, scope, and collection, allowing users to easily 
interact with Couchbase within Airflow tasks. The provider also includes 
functionality to customize connection field behaviour, adhering to Airflow’s 
connection management standards.

Our initial scope is deliberately minimal—focused on foundational operations. 
While we may introduce CRUD functionality later, we expect the provider to 
remain stable with minimal and infrequent changes.

We are committed to maintaining the provider, ensuring compatibility with 
future Couchbase API changes, and responding to community feedback—similar to 
the stewardship shown by other officially supported database providers.

If maintenance is a concern, we are fully willing to act as active maintainers, 
ensuring the provider remains well-supported, reliable, and aligned with 
Airflow’s evolution.

Regards,
Shyam Venkat

Re: [DISCUSSION] Proposal to Add Couchbase Provider to Apache Airflow

[Jarek Potiuk]

I am not convinced.

First of all what do you mean by visibility ? I see no problem with the
visibility of your current provider - couchbase seems to have well
established ecosystem with multiple plugins and extensions -
https://github.com/Couchbase-Ecosystem and I would say it's quite likely
Couchbase users will be looking for Airflow integration among those. Also
you will have (as you have now) freedom to release whatever changes and
improvements needed as Couchbase evolves and if you already commit to
maintain and keep the couchbase integration by Couchbase team (you wrote it
above) - you can equally well maintain it in and release it in your repo.

You already seem to be a good steward of that integration and by keeping
your ecosystem healthy you do a good job, I do not see a particular reason
why you'd want to contribute it to Airflow. We have no knowledge of
Couchbase itself here, you are the experts and you can decide what is the
best integration level you provide, the set of operators and hooks are best
for you - and I think it's better if you listen to the Couchbase community
feedback and get your community to contribute there - Airflow provides a
nice set of interfaces for you to adhere, and that's basically all that
(from Airflow Community point of view) you need to know and follow - and
you already do it apparently.

Couchbase is "source-available" database
https://www.couchbase.com/blog/couchbase-adopts-bsl-license/ - which is not
an OSI compliant open-source produict, which makes it even more suitable
for your company to keep maintaining the integration. Of course eventually
it's a community decision whether to accept such a provider or not, but I
would be rather against it - just for that reason.

J.



On Fri, Apr 11, 2025 at 11:17 AM Shyam Rajamannar
 wrote:

> Couchbase has customers already using Apache Airflow, and having an
> officially supported provider within the ecosystem would simplify
> integration, improve reliability, and encourage adoption.
>
> We previously released airflow-providers-couchbase<
> https://github.com/Couchbase-Ecosystem/airflow-providers-couchbase>
> through an external repository also listed under airflow ecosystem.
> However, it has been less visible to our customers and the broader Airflow
> community. Contributing this provider directly to the official Airflow
> ecosystem will ensure better visibility, easier adoption, and tighter
> integration with Airflow’s architecture and release processes.
>
> As part of the initial implementation, we have created a Couchbase hook
> that exposes the Couchbase client, scope, and collection, allowing users to
> easily interact with Couchbase within Airflow tasks. The provider also
> includes functionality to customize connection field behaviour, adhering to
> Airflow’s connection management standards.
>
> Our initial scope is deliberately minimal—focused on foundational
> operations. While we may introduce CRUD functionality later, we expect the
> provider to remain stable with minimal and infrequent changes.
>
> We are committed to maintaining the provider, ensuring compatibility with
> future Couchbase API changes, and responding to community feedback—similar
> to the stewardship shown by other officially supported database providers.
>
> If maintenance is a concern, we are fully willing to act as active
> maintainers, ensuring the provider remains well-supported, reliable, and
> aligned with Airflow’s evolution.
>
> Regards,
> Shyam Venkat
>
>
>

Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Kaxil Naik]

Agreed on not having extras apart from providers.

On Fri, 11 Apr 2025 at 13:36, Ash Berlin-Taylor  wrote:

> Just checked, no this was in 2.10 releases. NM then.
>
> Still, I think we should not add new extras to `apache-airflow` in
> general, and definitely not for anything other than direct provider
> “short-names”.
>
>
> -ash
>
> > On 11 Apr 2025, at 09:02, Ash Berlin-Taylor  wrote:
> >
> > Oh hang on wait a moment.
> >
> > Is aiobotocore a new extra that is not available in 2.10.5 or any early
> versions?
> >
> > If that is the case then I’d instead vote that we don’t add new extras
> to `apache-airflow`, and we instead remove it. I grudgingly accept that
> having `apache-airflow[amazon]` make sense, but `aiobotocore` isn’t the
> name of the provider, and you need to know what it means anyway, so I don’t
> think we should have it as an extra anywhere other than on the provider.
> >
> > -ash
> >
> >> On 11 Apr 2025, at 08:40, Jarek Potiuk  wrote:
> >>
> >> A draft PR here (i will split it as there are few unrelated changes)
> >> https://github.com/apache/airflow/pull/49103 -> but it shows how we can
> >> automatically set the min versions in "apache-airflow" meta-package
> (with
> >> 6-months-old provider versions).
> >>
> >> That will quite likely help to get the "Resolution too deep" error
> solved,
> >> and I think it has a nice property - that we can treat it as
> "recommended"
> >> range of versions for providers - while giving all the flexibility to
> the
> >> user to keep their old versions or downgrade.
> >>
> >>
> >> J.
> >>
> >> On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk  wrote:
> >>
> >>> Actually what I see when I attempted to do it - is that even if we
> >>> introduce it - it is not going to be a hard requirement. It is just a
> soft
> >>> limit when installing full apache-airflow[with extras].
> >>>
> >>> It will be more of a default behaviour to limit the providers when
> extras
> >>> are used but will be easy to override and it will not apply when you
> >>> install airflow-core /task-sdk/ providers directly. So yes - I think no
> >>> real drawback
> >>>
> >>> This limits will only be on extras of the 'apache-airflow' and in case
> >>> someone used the extras so far - the were not able to control the
> version
> >>> of provider that will be used anyway - and this only used a lower
> version
> >>> of provider than latest I'd some other package was installed at the
> same
> >>> time that was conflicting with latest provider..
> >>>
> >>> Thoss limits are not going to be persistent, they are going to simply
> be
> >>> taken into account when apache-airflow{provider} is used and only for
> the
> >>> time of the installation. It will not prevent the user from
> >>> upgrading/downgrading the provider later. And also user still be able
> to
> >>> remove extra and use the provider directly in the same `pip install`
> >>> command
> >>>
> >>> I have a draft change that sets the lower bound to now() -6 months +
> >>> manual overrides if we need to get some new provider limited to latest
> >>> version this way for example. I am still coming back from US and will
> be
> >>> home later today and try it and I think some form of it will be good to
> >>> have  - 6 months seems reasonable and it will have the
> 'self-maintenance'
> >>> capability.
> >>>
> >>> J.
> >>>
> >>> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor 
> >>> napisał:
> >>>
>  There absolutely is a downside to the users — “forcing" users to
> upgrade
>  providers along with Airflow version makes upgrades a much more
> daunting
>  experience.
> 
> > On 10 Apr 2025, at 21:31, Maciej Obuchowski 
>  wrote:
> >
> > For OpenLineage we'd definitely like something like this.
> > IMO there is no downside to restricting the providers to the latest
> > released version.
> > We could go with the chicken-egg for the next release - that would
> add
>  at
> > least one useful
> > feature authored by Kacper Muda -
> > https://github.com/apache/airflow/pull/48941
> > but even without it it's not broken, just missing that feature.
> >
> >
> > czw., 10 kwi 2025 o 21:49 Jarek Potiuk 
> napisał(a):
> >
> >> Accidentally It turned out - with today's investigation of why RC
>  images
> >> are not building that it is VERY needed. It seems that `pip`
> resolution
> >> works currently with our meta-package in a very bad way. If in our
> >> pyproject.toml we have just
> >> "amazon' =  "apache-airflow-providers-amazon", "google" =
> >> "apache-airflow-providers-google" 
> >>
> >> then
> >>
> >> apache-airflow[google,amazon] - will lead pip to download and try to
> >> resolve ALL VERSIONS or apparently ALL providers that have ever been
> >> released. That means we have 100 amazon, 100 google - and if you
> have
>  more
> >> extras - 10s or 100s of other providers to consider. And essentially
>  what
> >> pip

Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Jarek Potiuk]

That's a bit of a different topic :)  - I was only asking about the minimum
provider versions.

And I think no-one wants to add new extras - but  we do have a bunch of
them already. What we have now is that (see pyproject-toml files in both) :

* `apache-airflow-core` has a few of those historically to add "features"
to the core , but some (say kerberos) - does not have providers.
airflow-core currently has NO provider extras - they are only defined in
the meta-package
* `apache-airflow`  meta- on the other hand - has "pass-through" core
extras (same as "airflow-core") for back compatibility + it has provider
extras

Possibly we should also remove some of the "core" extras (for example I
think async, cgroups and a few others might not make sense there. But I
think this is another discussion . I will start it separately

J.


On Fri, Apr 11, 2025 at 12:30 PM Kaxil Naik  wrote:

> Agreed on not having extras apart from providers.
>
> On Fri, 11 Apr 2025 at 13:36, Ash Berlin-Taylor  wrote:
>
> > Just checked, no this was in 2.10 releases. NM then.
> >
> > Still, I think we should not add new extras to `apache-airflow` in
> > general, and definitely not for anything other than direct provider
> > “short-names”.
> >
> >
> > -ash
> >
> > > On 11 Apr 2025, at 09:02, Ash Berlin-Taylor  wrote:
> > >
> > > Oh hang on wait a moment.
> > >
> > > Is aiobotocore a new extra that is not available in 2.10.5 or any early
> > versions?
> > >
> > > If that is the case then I’d instead vote that we don’t add new extras
> > to `apache-airflow`, and we instead remove it. I grudgingly accept that
> > having `apache-airflow[amazon]` make sense, but `aiobotocore` isn’t the
> > name of the provider, and you need to know what it means anyway, so I
> don’t
> > think we should have it as an extra anywhere other than on the provider.
> > >
> > > -ash
> > >
> > >> On 11 Apr 2025, at 08:40, Jarek Potiuk  wrote:
> > >>
> > >> A draft PR here (i will split it as there are few unrelated changes)
> > >> https://github.com/apache/airflow/pull/49103 -> but it shows how we
> can
> > >> automatically set the min versions in "apache-airflow" meta-package
> > (with
> > >> 6-months-old provider versions).
> > >>
> > >> That will quite likely help to get the "Resolution too deep" error
> > solved,
> > >> and I think it has a nice property - that we can treat it as
> > "recommended"
> > >> range of versions for providers - while giving all the flexibility to
> > the
> > >> user to keep their old versions or downgrade.
> > >>
> > >>
> > >> J.
> > >>
> > >> On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk 
> wrote:
> > >>
> > >>> Actually what I see when I attempted to do it - is that even if we
> > >>> introduce it - it is not going to be a hard requirement. It is just a
> > soft
> > >>> limit when installing full apache-airflow[with extras].
> > >>>
> > >>> It will be more of a default behaviour to limit the providers when
> > extras
> > >>> are used but will be easy to override and it will not apply when you
> > >>> install airflow-core /task-sdk/ providers directly. So yes - I think
> no
> > >>> real drawback
> > >>>
> > >>> This limits will only be on extras of the 'apache-airflow' and in
> case
> > >>> someone used the extras so far - the were not able to control the
> > version
> > >>> of provider that will be used anyway - and this only used a lower
> > version
> > >>> of provider than latest I'd some other package was installed at the
> > same
> > >>> time that was conflicting with latest provider..
> > >>>
> > >>> Thoss limits are not going to be persistent, they are going to simply
> > be
> > >>> taken into account when apache-airflow{provider} is used and only for
> > the
> > >>> time of the installation. It will not prevent the user from
> > >>> upgrading/downgrading the provider later. And also user still be able
> > to
> > >>> remove extra and use the provider directly in the same `pip install`
> > >>> command
> > >>>
> > >>> I have a draft change that sets the lower bound to now() -6 months +
> > >>> manual overrides if we need to get some new provider limited to
> latest
> > >>> version this way for example. I am still coming back from US and will
> > be
> > >>> home later today and try it and I think some form of it will be good
> to
> > >>> have  - 6 months seems reasonable and it will have the
> > 'self-maintenance'
> > >>> capability.
> > >>>
> > >>> J.
> > >>>
> > >>> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor <
> a...@apache.org>
> > >>> napisał:
> > >>>
> >  There absolutely is a downside to the users — “forcing" users to
> > upgrade
> >  providers along with Airflow version makes upgrades a much more
> > daunting
> >  experience.
> > 
> > > On 10 Apr 2025, at 21:31, Maciej Obuchowski <
> mobuchow...@apache.org>
> >  wrote:
> > >
> > > For OpenLineage we'd definitely like something like this.
> > > IMO there is no downside to restricting the providers to the latest
> > > released ve

Re: [DISCUSS] Minimum versions of providers for Airflow 3

[Kaxil Naik]

Sounds good

On Fri, 11 Apr 2025 at 16:16, Jarek Potiuk  wrote:

> That's a bit of a different topic :)  - I was only asking about the minimum
> provider versions.
>
> And I think no-one wants to add new extras - but  we do have a bunch of
> them already. What we have now is that (see pyproject-toml files in both) :
>
> * `apache-airflow-core` has a few of those historically to add "features"
> to the core , but some (say kerberos) - does not have providers.
> airflow-core currently has NO provider extras - they are only defined in
> the meta-package
> * `apache-airflow`  meta- on the other hand - has "pass-through" core
> extras (same as "airflow-core") for back compatibility + it has provider
> extras
>
> Possibly we should also remove some of the "core" extras (for example I
> think async, cgroups and a few others might not make sense there. But I
> think this is another discussion . I will start it separately
>
> J.
>
>
> On Fri, Apr 11, 2025 at 12:30 PM Kaxil Naik  wrote:
>
> > Agreed on not having extras apart from providers.
> >
> > On Fri, 11 Apr 2025 at 13:36, Ash Berlin-Taylor  wrote:
> >
> > > Just checked, no this was in 2.10 releases. NM then.
> > >
> > > Still, I think we should not add new extras to `apache-airflow` in
> > > general, and definitely not for anything other than direct provider
> > > “short-names”.
> > >
> > >
> > > -ash
> > >
> > > > On 11 Apr 2025, at 09:02, Ash Berlin-Taylor  wrote:
> > > >
> > > > Oh hang on wait a moment.
> > > >
> > > > Is aiobotocore a new extra that is not available in 2.10.5 or any
> early
> > > versions?
> > > >
> > > > If that is the case then I’d instead vote that we don’t add new
> extras
> > > to `apache-airflow`, and we instead remove it. I grudgingly accept that
> > > having `apache-airflow[amazon]` make sense, but `aiobotocore` isn’t the
> > > name of the provider, and you need to know what it means anyway, so I
> > don’t
> > > think we should have it as an extra anywhere other than on the
> provider.
> > > >
> > > > -ash
> > > >
> > > >> On 11 Apr 2025, at 08:40, Jarek Potiuk  wrote:
> > > >>
> > > >> A draft PR here (i will split it as there are few unrelated changes)
> > > >> https://github.com/apache/airflow/pull/49103 -> but it shows how we
> > can
> > > >> automatically set the min versions in "apache-airflow" meta-package
> > > (with
> > > >> 6-months-old provider versions).
> > > >>
> > > >> That will quite likely help to get the "Resolution too deep" error
> > > solved,
> > > >> and I think it has a nice property - that we can treat it as
> > > "recommended"
> > > >> range of versions for providers - while giving all the flexibility
> to
> > > the
> > > >> user to keep their old versions or downgrade.
> > > >>
> > > >>
> > > >> J.
> > > >>
> > > >> On Fri, Apr 11, 2025 at 7:01 AM Jarek Potiuk 
> > wrote:
> > > >>
> > > >>> Actually what I see when I attempted to do it - is that even if we
> > > >>> introduce it - it is not going to be a hard requirement. It is
> just a
> > > soft
> > > >>> limit when installing full apache-airflow[with extras].
> > > >>>
> > > >>> It will be more of a default behaviour to limit the providers when
> > > extras
> > > >>> are used but will be easy to override and it will not apply when
> you
> > > >>> install airflow-core /task-sdk/ providers directly. So yes - I
> think
> > no
> > > >>> real drawback
> > > >>>
> > > >>> This limits will only be on extras of the 'apache-airflow' and in
> > case
> > > >>> someone used the extras so far - the were not able to control the
> > > version
> > > >>> of provider that will be used anyway - and this only used a lower
> > > version
> > > >>> of provider than latest I'd some other package was installed at the
> > > same
> > > >>> time that was conflicting with latest provider..
> > > >>>
> > > >>> Thoss limits are not going to be persistent, they are going to
> simply
> > > be
> > > >>> taken into account when apache-airflow{provider} is used and only
> for
> > > the
> > > >>> time of the installation. It will not prevent the user from
> > > >>> upgrading/downgrading the provider later. And also user still be
> able
> > > to
> > > >>> remove extra and use the provider directly in the same `pip
> install`
> > > >>> command
> > > >>>
> > > >>> I have a draft change that sets the lower bound to now() -6 months
> +
> > > >>> manual overrides if we need to get some new provider limited to
> > latest
> > > >>> version this way for example. I am still coming back from US and
> will
> > > be
> > > >>> home later today and try it and I think some form of it will be
> good
> > to
> > > >>> have  - 6 months seems reasonable and it will have the
> > > 'self-maintenance'
> > > >>> capability.
> > > >>>
> > > >>> J.
> > > >>>
> > > >>> czw., 10 kwi 2025, 22:46 użytkownik Ash Berlin-Taylor <
> > a...@apache.org>
> > > >>> napisał:
> > > >>>
> > >  There absolutely is a downside to the users — “forcing" users to
> > > upgrade
> > >  providers along with Airflow version makes

[ANNOUNCE] Airflow main is now targeting 3.1

[Kaxil Naik]

Hey everyone,

As we approach the final stages of the Airflow 3.0.0 release, the main
branch is now officially targeting Airflow 3.1.

*What this means*:
• PR authors and contributors should continue working as usual -- nothing
changes in how PRs are submitted or merged.
• However, starting now, all new changes merged into main will be
considered part of 3.1.
• If a change should be included in 3.0.0, I’ll cherry-pick it into the
v3-0-test branch or rebase to main in case there are no 3.1 changes and all
changes merged-to-main are bugfixes and crucial for 3.0.0 - and I’ll reach
out to authors directly if anything needs clarification.

*Why now?*

We are at a stage where stability matters more than velocity. I expect RC2
to help surface any remaining issues, and I’d like RC3 to be our final RC.
Having a tighter grip on post-RC2 changes gives us a better shot at getting
there without another RC cycle.

I’ll actively sync main and v3-0-test multiple times a day and compare
diffs between them to ensure no important changes are missed. If anything
seems off, I’ll flag it in #contributors or #internal-airflow-ci-cd on
Slack. If you find something that is missed, please reach out to me or on
#airflow-3-dev Slack channel.

*Why not wait?*

Right now, the main and v3-0-tests branches are still very close, which
makes it easier to maintain control and reduce risk.

Using labels or backport flags doesn’t provide the same level of review or
assurance as I want to opt-in to reviewing each change then the opposite.

This also gives us a chance to gradually clean up our `-stable` vs `-test`
branching process, which has drifted a bit in recent releases.

I’m open to refining this if needed, but wanted to set expectations and
keep things moving cleanly into the next cycle.

I have created https://github.com/apache/airflow/pull/49119 in preparation
for that.

Once 3.0.0 is officially released, we’ll return to a more explicit and
automated process for backporting, similar to what we followed during the
2.x cycle. This means PRs intended for the 3.0.x line should either be
labeled for backporting (e.g. `backport-to-v3-0-test`) or added to the
relevant bugfix milestone. I will share more details and reminders once we
are at that point.

Thanks everyone!

Regards,
Kaxil

Re: [ANNOUNCE] Airflow main is now targeting 3.1

[Jarek Potiuk]

Just one comment - following our discussion.. I perfectly understand we
want to take more control now on what is merged. And if you want to take
responsibility there then I am fine ( but also do  not envy you :)

I think we just need to clarify who and  when should add the backport-to
labels PRS (if at all). Just to understand it. Because I understood it a
bit differently this morning.

I understand that you would prefer no one to set the  backport label at all
and you will identify what to cherry pick and do it ? I am talking about
changes to 'airflow sources'. Do I understand correctly?

I guess we allow some exceptions.

I think (from past experience) would like to treat the CI / breeze related
changes a bit differently - things tend to decay there pretty quickly - we
still have some chicken/egg providers there and we have likely some doc
things to fix etc.etc.and it is far easier to cherry-pick all those changes
rather than subset of those.

I will for sure want to keep on backporting (with label) all necessary CI
/breeze /dependency changes from main to keep builds green. Happy to wait
for your merges there, but I thi k also iterating on those backporting PR
to make the latest v3-0-test green will make your life easier and I am
happy to help with that.

Not sure if there are other exceptions that are similar ?

Does all I wrote make sense :) ?

J.

pt., 11 kwi 2025, 14:05 użytkownik Kaxil Naik  napisał:

> Hey everyone,
>
> As we approach the final stages of the Airflow 3.0.0 release, the main
> branch is now officially targeting Airflow 3.1.
>
> *What this means*:
> • PR authors and contributors should continue working as usual -- nothing
> changes in how PRs are submitted or merged.
> • However, starting now, all new changes merged into main will be
> considered part of 3.1.
> • If a change should be included in 3.0.0, I’ll cherry-pick it into the
> v3-0-test branch or rebase to main in case there are no 3.1 changes and all
> changes merged-to-main are bugfixes and crucial for 3.0.0 - and I’ll reach
> out to authors directly if anything needs clarification.
>
> *Why now?*
>
> We are at a stage where stability matters more than velocity. I expect RC2
> to help surface any remaining issues, and I’d like RC3 to be our final RC.
> Having a tighter grip on post-RC2 changes gives us a better shot at getting
> there without another RC cycle.
>
> I’ll actively sync main and v3-0-test multiple times a day and compare
> diffs between them to ensure no important changes are missed. If anything
> seems off, I’ll flag it in #contributors or #internal-airflow-ci-cd on
> Slack. If you find something that is missed, please reach out to me or on
> #airflow-3-dev Slack channel.
>
> *Why not wait?*
>
> Right now, the main and v3-0-tests branches are still very close, which
> makes it easier to maintain control and reduce risk.
>
> Using labels or backport flags doesn’t provide the same level of review or
> assurance as I want to opt-in to reviewing each change then the opposite.
>
> This also gives us a chance to gradually clean up our `-stable` vs `-test`
> branching process, which has drifted a bit in recent releases.
>
> I’m open to refining this if needed, but wanted to set expectations and
> keep things moving cleanly into the next cycle.
>
> I have created https://github.com/apache/airflow/pull/49119 in preparation
> for that.
>
> Once 3.0.0 is officially released, we’ll return to a more explicit and
> automated process for backporting, similar to what we followed during the
> 2.x cycle. This means PRs intended for the 3.0.x line should either be
> labeled for backporting (e.g. `backport-to-v3-0-test`) or added to the
> relevant bugfix milestone. I will share more details and reminders once we
> are at that point.
>
> Thanks everyone!
>
> Regards,
> Kaxil
>

Re: [VOTE] Airflow Providers prepared on April 10, 2025

[Vincent Beck]

+1 non binding for apache-airflow-providers-amazon==9.6.0rc1. All AWS system 
tests are passing against apache-airflow-providers-amazon==9.6.0rc1. You can 
see the results here: 
https://aws-mwaa.github.io/#/open-source/system-tests/version/6bd535471819074f74b0c3a3251a823be24c61b0_9.6.0rc1.html.



On 2025/04/10 14:49:17 Elad Kalif wrote:
> Hey all,
> 
> I have just cut the new wave Airflow Providers packages. This email is
> calling a vote on the release,
> which will last for 72 hours - which means that it will end on April 13,
> 2025 14:45 PM UTC and until 3 binding +1 votes have been received.
> 
> Consider this my (binding) +1.
> 
> Airflow Providers are available at:
> https://dist.apache.org/repos/dist/dev/airflow/providers/
> 
> *apache-airflow-providers--*.tar.gz* are the binary
>  Python "sdist" release - they are also official "sources" for the Provider
> distributions.
> 
> *apache_airflow_providers_-*.whl are the binary
>  Python "wheel" release.
> 
> The test procedure for PMC members is described in
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-pmc-members
> 
> The test procedure for and Contributors who would like to test this RC is
> described in:
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-contributors
> 
> 
> Public keys are available at:
> https://dist.apache.org/repos/dist/release/airflow/KEYS
> 
> Please vote accordingly:
> 
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
> 
> Only votes from PMC members are binding, but members of the community are
> encouraged to test the release and vote with "(non-binding)".
> 
> Please note that the version number excludes the 'rcX' string.
> This will allow us to rename the artifact without modifying
> the artifact checksums when we actually release.
> 
> The status of testing the providers by the community is kept here:
> https://github.com/apache/airflow/issues/49066
> 
> The issue is also the easiest way to see important PRs included in the RC
> candidates.
> Detailed changelog for the providers will be published in the documentation
> after the
> RC candidates are released.
> 
> You can find the RC packages in PyPI following these links:
> 
> https://pypi.org/project/apache-airflow-providers-amazon/9.6.0rc1/
> https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.8.0rc1/
> https://pypi.org/project/apache-airflow-providers-apache-livy/4.3.1rc1/
> https://pypi.org/project/apache-airflow-providers-apache-spark/5.2.0rc1/
> https://pypi.org/project/apache-airflow-providers-celery/3.10.6rc1/
> https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.4.2rc1/
> https://pypi.org/project/apache-airflow-providers-common-io/1.5.3rc1/
> https://pypi.org/project/apache-airflow-providers-common-messaging/1.0.0rc2/
> https://pypi.org/project/apache-airflow-providers-databricks/7.3.1rc1/
> https://pypi.org/project/apache-airflow-providers-discord/3.9.5rc1/
> https://pypi.org/project/apache-airflow-providers-docker/4.3.1rc1/
> https://pypi.org/project/apache-airflow-providers-fab/2.0.0rc4/
> https://pypi.org/project/apache-airflow-providers-git/0.0.1rc1/
> https://pypi.org/project/apache-airflow-providers-google/15.0.1rc1/
> https://pypi.org/project/apache-airflow-providers-mysql/6.2.1rc1/
> https://pypi.org/project/apache-airflow-providers-openlineage/2.1.3rc1/
> https://pypi.org/project/apache-airflow-providers-sftp/5.2.0rc1/
> https://pypi.org/project/apache-airflow-providers-slack/9.0.4rc1/
> https://pypi.org/project/apache-airflow-providers-snowflake/6.2.1rc1/
> https://pypi.org/project/apache-airflow-providers-sqlite/4.0.2rc1/
> https://pypi.org/project/apache-airflow-providers-standard/0.4.0rc1/
> 
> 
> Cheers,
> Elad Kalif
> 

-
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org

Re: [ANNOUNCE] Airflow main is now targeting 3.1

[Kaxil Naik]

Yeah, I removed that label, too, to avoid any confusion. Once 3.0.0 is
officially released, we’ll return to `backport-to-v3-0-test` for automatic
backport -- where the PR author or a committer could do that -- but we can
discuss than once we reach there.

I think we just need to clarify who and  when should add the backport-to
> labels PRS (if at all). Just to understand it. Because I understood it a
> bit differently this morning.
> I understand that you would prefer no one to set the  backport label at all
> and you will identify what to cherry pick and do it ? I am talking about
> changes to 'airflow sources'. Do I understand correctly?



And yeah absolutely about the CI-related changes. I am going to keep them
updated and ask for help for sure if needed.

On Fri, 11 Apr 2025 at 18:30, Jarek Potiuk  wrote:

> Just one comment - following our discussion.. I perfectly understand we
> want to take more control now on what is merged. And if you want to take
> responsibility there then I am fine ( but also do  not envy you :)
>
> I think we just need to clarify who and  when should add the backport-to
> labels PRS (if at all). Just to understand it. Because I understood it a
> bit differently this morning.
>
> I understand that you would prefer no one to set the  backport label at all
> and you will identify what to cherry pick and do it ? I am talking about
> changes to 'airflow sources'. Do I understand correctly?
>
> I guess we allow some exceptions.
>
> I think (from past experience) would like to treat the CI / breeze related
> changes a bit differently - things tend to decay there pretty quickly - we
> still have some chicken/egg providers there and we have likely some doc
> things to fix etc.etc.and it is far easier to cherry-pick all those changes
> rather than subset of those.
>
> I will for sure want to keep on backporting (with label) all necessary CI
> /breeze /dependency changes from main to keep builds green. Happy to wait
> for your merges there, but I thi k also iterating on those backporting PR
> to make the latest v3-0-test green will make your life easier and I am
> happy to help with that.
>
> Not sure if there are other exceptions that are similar ?
>
> Does all I wrote make sense :) ?
>
> J.
>
> pt., 11 kwi 2025, 14:05 użytkownik Kaxil Naik 
> napisał:
>
> > Hey everyone,
> >
> > As we approach the final stages of the Airflow 3.0.0 release, the main
> > branch is now officially targeting Airflow 3.1.
> >
> > *What this means*:
> > • PR authors and contributors should continue working as usual -- nothing
> > changes in how PRs are submitted or merged.
> > • However, starting now, all new changes merged into main will be
> > considered part of 3.1.
> > • If a change should be included in 3.0.0, I’ll cherry-pick it into the
> > v3-0-test branch or rebase to main in case there are no 3.1 changes and
> all
> > changes merged-to-main are bugfixes and crucial for 3.0.0 - and I’ll
> reach
> > out to authors directly if anything needs clarification.
> >
> > *Why now?*
> >
> > We are at a stage where stability matters more than velocity. I expect
> RC2
> > to help surface any remaining issues, and I’d like RC3 to be our final
> RC.
> > Having a tighter grip on post-RC2 changes gives us a better shot at
> getting
> > there without another RC cycle.
> >
> > I’ll actively sync main and v3-0-test multiple times a day and compare
> > diffs between them to ensure no important changes are missed. If anything
> > seems off, I’ll flag it in #contributors or #internal-airflow-ci-cd on
> > Slack. If you find something that is missed, please reach out to me or on
> > #airflow-3-dev Slack channel.
> >
> > *Why not wait?*
> >
> > Right now, the main and v3-0-tests branches are still very close, which
> > makes it easier to maintain control and reduce risk.
> >
> > Using labels or backport flags doesn’t provide the same level of review
> or
> > assurance as I want to opt-in to reviewing each change then the opposite.
> >
> > This also gives us a chance to gradually clean up our `-stable` vs
> `-test`
> > branching process, which has drifted a bit in recent releases.
> >
> > I’m open to refining this if needed, but wanted to set expectations and
> > keep things moving cleanly into the next cycle.
> >
> > I have created https://github.com/apache/airflow/pull/49119 in
> preparation
> > for that.
> >
> > Once 3.0.0 is officially released, we’ll return to a more explicit and
> > automated process for backporting, similar to what we followed during the
> > 2.x cycle. This means PRs intended for the 3.0.x line should either be
> > labeled for backporting (e.g. `backport-to-v3-0-test`) or added to the
> > relevant bugfix milestone. I will share more details and reminders once
> we
> > are at that point.
> >
> > Thanks everyone!
> >
> > Regards,
> > Kaxil
> >
>

Re: [DISCUSSION] Proposal to Add Couchbase Provider to Apache Airflow

[Vikram Koka]

Shyam,

First off, I'm really glad to hear that Couchbase is interested in a deeper
integration with Airflow and that Couchbase users are already finding value
in it.

Just to follow up with a couple of specific questions around your request:
Could you share more about what's missing today with the integration being
maintained in the Couchbase repo, versus having it packaged within the
Airflow repo?

You mentioned benefits like "better visibility, easier adoption, and
tighter integration with Airflow's architecture and release processes."
If you could expand on those - especially in terms of what you see changing
before vs. after, and ideally any metrics or examples - that context would
be very helpful.

Best regards,
Vikram


On Fri, Apr 11, 2025 at 3:02 AM Jarek Potiuk  wrote:

> I am not convinced.
>
> First of all what do you mean by visibility ? I see no problem with the
> visibility of your current provider - couchbase seems to have well
> established ecosystem with multiple plugins and extensions -
> https://github.com/Couchbase-Ecosystem and I would say it's quite likely
> Couchbase users will be looking for Airflow integration among those. Also
> you will have (as you have now) freedom to release whatever changes and
> improvements needed as Couchbase evolves and if you already commit to
> maintain and keep the couchbase integration by Couchbase team (you wrote it
> above) - you can equally well maintain it in and release it in your repo.
>
> You already seem to be a good steward of that integration and by keeping
> your ecosystem healthy you do a good job, I do not see a particular reason
> why you'd want to contribute it to Airflow. We have no knowledge of
> Couchbase itself here, you are the experts and you can decide what is the
> best integration level you provide, the set of operators and hooks are best
> for you - and I think it's better if you listen to the Couchbase community
> feedback and get your community to contribute there - Airflow provides a
> nice set of interfaces for you to adhere, and that's basically all that
> (from Airflow Community point of view) you need to know and follow - and
> you already do it apparently.
>
> Couchbase is "source-available" database
> https://www.couchbase.com/blog/couchbase-adopts-bsl-license/ - which is
> not
> an OSI compliant open-source produict, which makes it even more suitable
> for your company to keep maintaining the integration. Of course eventually
> it's a community decision whether to accept such a provider or not, but I
> would be rather against it - just for that reason.
>
> J.
>
>
>
> On Fri, Apr 11, 2025 at 11:17 AM Shyam Rajamannar
>  wrote:
>
> > Couchbase has customers already using Apache Airflow, and having an
> > officially supported provider within the ecosystem would simplify
> > integration, improve reliability, and encourage adoption.
> >
> > We previously released airflow-providers-couchbase<
> > https://github.com/Couchbase-Ecosystem/airflow-providers-couchbase>
> > through an external repository also listed under airflow ecosystem.
> > However, it has been less visible to our customers and the broader
> Airflow
> > community. Contributing this provider directly to the official Airflow
> > ecosystem will ensure better visibility, easier adoption, and tighter
> > integration with Airflow’s architecture and release processes.
> >
> > As part of the initial implementation, we have created a Couchbase hook
> > that exposes the Couchbase client, scope, and collection, allowing users
> to
> > easily interact with Couchbase within Airflow tasks. The provider also
> > includes functionality to customize connection field behaviour, adhering
> to
> > Airflow’s connection management standards.
> >
> > Our initial scope is deliberately minimal—focused on foundational
> > operations. While we may introduce CRUD functionality later, we expect
> the
> > provider to remain stable with minimal and infrequent changes.
> >
> > We are committed to maintaining the provider, ensuring compatibility with
> > future Couchbase API changes, and responding to community
> feedback—similar
> > to the stewardship shown by other officially supported database
> providers.
> >
> > If maintenance is a concern, we are fully willing to act as active
> > maintainers, ensuring the provider remains well-supported, reliable, and
> > aligned with Airflow’s evolution.
> >
> > Regards,
> > Shyam Venkat
> >
> >
> >
>

Re: [VOTE] Airflow Providers prepared on April 10, 2025

[Jens Scheffler]

+1 (binding) - Checked SVN, Check in Docker, Reproducible package build,
Licenses, Signatures (All except edge)

Note: I was surprised that I found edge provider in the delivered
package though I assume (again, but no explicit list given) that edge is
not voted on. Reproducibility check with edge failed, the DIFF was that
the SVN package in provider.yaml was marked "not-ready" but in git is is
marked "ready".

Maybe it would be good if providers are released and VOTE is requested
to explicitly list which packages it is voted about.

Thanks for makign it happen! Also FAB is now correct! And due to the
exclusion of node_modules the package is much smaller.

On 11.04.25 17:18, Vincent Beck wrote:

+1 non binding for apache-airflow-providers-amazon==9.6.0rc1. All AWS system 
tests are passing against apache-airflow-providers-amazon==9.6.0rc1. You can 
see the results here: 
https://aws-mwaa.github.io/#/open-source/system-tests/version/6bd535471819074f74b0c3a3251a823be24c61b0_9.6.0rc1.html.



On 2025/04/10 14:49:17 Elad Kalif wrote:

Hey all,

I have just cut the new wave Airflow Providers packages. This email is
calling a vote on the release,
which will last for 72 hours - which means that it will end on April 13,
2025 14:45 PM UTC and until 3 binding +1 votes have been received.

Consider this my (binding) +1.

Airflow Providers are available at:
https://dist.apache.org/repos/dist/dev/airflow/providers/

*apache-airflow-providers--*.tar.gz* are the binary
  Python "sdist" release - they are also official "sources" for the Provider
distributions.

*apache_airflow_providers_-*.whl are the binary
  Python "wheel" release.

The test procedure for PMC members is described in
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-pmc-members

The test procedure for and Contributors who would like to test this RC is
described in:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-contributors


Public keys are available at:
https://dist.apache.org/repos/dist/release/airflow/KEYS

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Please note that the version number excludes the 'rcX' string.
This will allow us to rename the artifact without modifying
the artifact checksums when we actually release.

The status of testing the providers by the community is kept here:
https://github.com/apache/airflow/issues/49066

The issue is also the easiest way to see important PRs included in the RC
candidates.
Detailed changelog for the providers will be published in the documentation
after the
RC candidates are released.

You can find the RC packages in PyPI following these links:

https://pypi.org/project/apache-airflow-providers-amazon/9.6.0rc1/
https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.8.0rc1/
https://pypi.org/project/apache-airflow-providers-apache-livy/4.3.1rc1/
https://pypi.org/project/apache-airflow-providers-apache-spark/5.2.0rc1/
https://pypi.org/project/apache-airflow-providers-celery/3.10.6rc1/
https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.4.2rc1/
https://pypi.org/project/apache-airflow-providers-common-io/1.5.3rc1/
https://pypi.org/project/apache-airflow-providers-common-messaging/1.0.0rc2/
https://pypi.org/project/apache-airflow-providers-databricks/7.3.1rc1/
https://pypi.org/project/apache-airflow-providers-discord/3.9.5rc1/
https://pypi.org/project/apache-airflow-providers-docker/4.3.1rc1/
https://pypi.org/project/apache-airflow-providers-fab/2.0.0rc4/
https://pypi.org/project/apache-airflow-providers-git/0.0.1rc1/
https://pypi.org/project/apache-airflow-providers-google/15.0.1rc1/
https://pypi.org/project/apache-airflow-providers-mysql/6.2.1rc1/
https://pypi.org/project/apache-airflow-providers-openlineage/2.1.3rc1/
https://pypi.org/project/apache-airflow-providers-sftp/5.2.0rc1/
https://pypi.org/project/apache-airflow-providers-slack/9.0.4rc1/
https://pypi.org/project/apache-airflow-providers-snowflake/6.2.1rc1/
https://pypi.org/project/apache-airflow-providers-sqlite/4.0.2rc1/
https://pypi.org/project/apache-airflow-providers-standard/0.4.0rc1/


Cheers,
Elad Kalif


-
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org

Re: [VOTE] Airflow Providers prepared on April 10, 2025

[Elad Kalif]

During release I manually mark edge as not ready so it won't create docs
for it. The fact that we can't push it to PyPi doesnt mean we don't test it
hence I pushed it to SVN.

We vote only on the providers rc listed in this email

בתאריך שבת, 12 באפר׳ 2025, 00:35, מאת Jens Scheffler
‏:

> +1 (binding) - Checked SVN, Check in Docker, Reproducible package build,
> Licenses, Signatures (All except edge)
>
> Note: I was surprised that I found edge provider in the delivered
> package though I assume (again, but no explicit list given) that edge is
> not voted on. Reproducibility check with edge failed, the DIFF was that
> the SVN package in provider.yaml was marked "not-ready" but in git is is
> marked "ready".
>
> Maybe it would be good if providers are released and VOTE is requested
> to explicitly list which packages it is voted about.
>
> Thanks for makign it happen! Also FAB is now correct! And due to the
> exclusion of node_modules the package is much smaller.
>
> On 11.04.25 17:18, Vincent Beck wrote:
> > +1 non binding for apache-airflow-providers-amazon==9.6.0rc1. All AWS
> system tests are passing against apache-airflow-providers-amazon==9.6.0rc1.
> You can see the results here:
> https://aws-mwaa.github.io/#/open-source/system-tests/version/6bd535471819074f74b0c3a3251a823be24c61b0_9.6.0rc1.html
> .
> >
> >
> >
> > On 2025/04/10 14:49:17 Elad Kalif wrote:
> >> Hey all,
> >>
> >> I have just cut the new wave Airflow Providers packages. This email is
> >> calling a vote on the release,
> >> which will last for 72 hours - which means that it will end on April 13,
> >> 2025 14:45 PM UTC and until 3 binding +1 votes have been received.
> >>
> >> Consider this my (binding) +1.
> >>
> >> Airflow Providers are available at:
> >> https://dist.apache.org/repos/dist/dev/airflow/providers/
> >>
> >> *apache-airflow-providers--*.tar.gz* are the binary
> >>   Python "sdist" release - they are also official "sources" for the
> Provider
> >> distributions.
> >>
> >> *apache_airflow_providers_-*.whl are the binary
> >>   Python "wheel" release.
> >>
> >> The test procedure for PMC members is described in
> >>
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-pmc-members
> >>
> >> The test procedure for and Contributors who would like to test this RC
> is
> >> described in:
> >>
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-contributors
> >>
> >>
> >> Public keys are available at:
> >> https://dist.apache.org/repos/dist/release/airflow/KEYS
> >>
> >> Please vote accordingly:
> >>
> >> [ ] +1 approve
> >> [ ] +0 no opinion
> >> [ ] -1 disapprove with the reason
> >>
> >> Only votes from PMC members are binding, but members of the community
> are
> >> encouraged to test the release and vote with "(non-binding)".
> >>
> >> Please note that the version number excludes the 'rcX' string.
> >> This will allow us to rename the artifact without modifying
> >> the artifact checksums when we actually release.
> >>
> >> The status of testing the providers by the community is kept here:
> >> https://github.com/apache/airflow/issues/49066
> >>
> >> The issue is also the easiest way to see important PRs included in the
> RC
> >> candidates.
> >> Detailed changelog for the providers will be published in the
> documentation
> >> after the
> >> RC candidates are released.
> >>
> >> You can find the RC packages in PyPI following these links:
> >>
> >> https://pypi.org/project/apache-airflow-providers-amazon/9.6.0rc1/
> >> https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.8.0rc1/
> >> https://pypi.org/project/apache-airflow-providers-apache-livy/4.3.1rc1/
> >>
> https://pypi.org/project/apache-airflow-providers-apache-spark/5.2.0rc1/
> >> https://pypi.org/project/apache-airflow-providers-celery/3.10.6rc1/
> >>
> https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.4.2rc1/
> >> https://pypi.org/project/apache-airflow-providers-common-io/1.5.3rc1/
> >>
> https://pypi.org/project/apache-airflow-providers-common-messaging/1.0.0rc2/
> >> https://pypi.org/project/apache-airflow-providers-databricks/7.3.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-discord/3.9.5rc1/
> >> https://pypi.org/project/apache-airflow-providers-docker/4.3.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-fab/2.0.0rc4/
> >> https://pypi.org/project/apache-airflow-providers-git/0.0.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-google/15.0.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-mysql/6.2.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-openlineage/2.1.3rc1/
> >> https://pypi.org/project/apache-airflow-providers-sftp/5.2.0rc1/
> >> https://pypi.org/project/apache-airflow-providers-slack/9.0.4rc1/
> >> https://pypi.org/project/apache-airflow-providers-snowflake/6.2.1rc1/
> >> https://pypi.org/project/apache-airflow-providers-sqlite/4.0.2rc1/
> >> htt