Re: 📢 Apache Airflow 3.0.0beta4 is available for testing! Final beta!! 🎉

[Amogh Desai]

Nice! Good job to the entire team, this is an impressive milestone.

Thanks Kaxil, Jed and everyone who helped in creating the release.

Thanks & Regards,
Amogh Desai


On Fri, Mar 21, 2025 at 3:33 AM Pavankumar Gopidesu 
wrote:

> Whoohoo, Great work everyone :)
>
> Pavan
>
> On Thu, Mar 20, 2025 at 9:58 PM Buğra Öztürk 
> wrote:
>
> > Amazing news! Kudos to everyone who contributed and made this happen!
> >
> > On Thu, Mar 20, 2025 at 9:52 PM Kaxil Naik  wrote:
> >
> > > Docker image is published, use the following:
> > >
> > > docker pull apache/airflow:3.0.0b4
> > >
> > > On Fri, 21 Mar 2025 at 01:58, Vikram Koka  >
> > > wrote:
> > >
> > > > Awesome!
> > > >
> > > > Thank you Kaxil and everyone who contributed to this release!
> > > >
> > > > Vikram
> > > >
> > > > On Thu, Mar 20, 2025 at 12:55 PM Jarek Potiuk 
> > wrote:
> > > >
> > > > > Wowowowowwowow
> > > > >
> > > > > On Thu, Mar 20, 2025 at 8:54 PM Kaxil Naik 
> > > wrote:
> > > > >
> > > > > > Dear Airflow Community,
> > > > > >
> > > > > > I am thrilled to announce the availability of Apache Airflow
> > > > 3.0.0.beta4
> > > > > > for testing!
> > > > > >
> > > > > > Airflow 3.0 marks a significant milestone as the first major
> > > > > > release in over four years, introducing improvements that enhance
> > > user
> > > > > > experience, task execution, and system scalability.
> > > > > >
> > > > > > First, a few caveats:
> > > > > >
> > > > > > This is a beta release, so do not run it in production. It may
> > > contain
> > > > > > significant issues, and you will likely need to reset your
> database
> > > > > between
> > > > > > this and subsequent release candidate versions. (Consider
> yourself
> > > > > > warned!)
> > > > > >
> > > > > > This release is intended for Airflow developers only to test the
> > > build
> > > > > and
> > > > > > start preparing for Airflow 3.0.0. This is not an official
> > > release—that
> > > > > > will happen when we create a release candidate and hold a vote.
> The
> > > > > > expected timeline for the first release candidate is the week of
> > > > > > 2025-03-31, but we encourage early feedback to help stabilize the
> > > > > release.
> > > > > >
> > > > > > Where to get it? The beta snapshot is available at:
> > > > > > https://dist.apache.org/repos/dist/dev/airflow/3.0.0b4
> > > > > >
> > > > > >- apache-airflow-3.0.0b3-bin.tar.gz: Binary Python "sdist"
> > > snapshot.
> > > > > >- apache_airflow-3.0.0b3-py3-none-any.whl: Binary Python wheel
> > > > > snapshot.
> > > > > >
> > > > > > This snapshot has also been published to PyPI at
> > > > > > https://pypi.org/project/apache-airflow/3.0.0b4/
> > > > > >
> > > > > > Also present are beta releases for various other packages which
> you
> > > > might
> > > > > > also need for 3.0.0b4 to work. These have also been published to
> > > PyPI.:
> > > > > >
> > > > > >- apache-airflow-task-sdk
> > > > > >- apache-airflow-providers-celery
> > > > > >- apache-airflow-providers-cncf-kubernetes
> > > > > >- apache-airflow-providers-common-compat
> > > > > >- apache-airflow-providers-common-io
> > > > > >- apache-airflow-providers-common-messaging
> > > > > >- apache-airflow-providers-common-sql
> > > > > >- apache-airflow-providers-fab
> > > > > >- apache-airflow-providers-openlineage
> > > > > >- apache-airflow-providers-standard
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Public Keys & Verification*
> > > > > >
> > > > > > Public keys for verification are available at:
> > > > > > https://www.apache.org/dist/airflow/KEYS
> > > > > >
> > > > > > Jed has signed the artifacts.
> > > > > >
> > > > > > *Instructions*:
> > > > > >
> > > > > > You can build a virtualenv that installs this beta, and other
> > > required
> > > > > > packages (e.g. task sdk), like this:
> > > > > >
> > > > > > ```
> > > > > > uv venv
> > > > > >
> > > > > > uv pip install \
> > > > > > apache-airflow==3.0.0b4 \
> > > > > > apache-airflow-providers-celery==3.10.4b1 \
> > > > > > apache-airflow-providers-cncf-kubernetes==10.4.0b1 \
> > > > > > apache-airflow-providers-common-compat==1.6.0b1 \
> > > > > > apache-airflow-providers-common-io==1.5.2b1 \
> > > > > > apache-airflow-providers-common-sql==1.24.1b1 \
> > > > > > apache-airflow-providers-fab==2.0.0b1 \
> > > > > > apache-airflow-providers-openlineage==2.1.2b1 \
> > > > > > apache-airflow-providers-standard==0.2.0b1 \
> > > > > > apache-airflow-task-sdk==1.0.0b4
> > > > > > ```
> > > > > >
> > > > > > The Dockerfiles will also be published in an hour. Once they are
> > > done,
> > > > I
> > > > > > will reply to this email.
> > > > > >
> > > > > >
> > > > > > *What's new in Airflow 3?*
> > > > > > Airflow 3.0.0 introduces significant enhancements and breaking
> > > changes.
> > > > > >
> > > > > > Notable Features
> > > > > >
> > > > > > DAG versioning & Bundles
> > > > > >
> > > > > > Airflow now tracks DAG versions, offering better visibility i

Re: Airflow Mentors for Summer 2025 MLH Fellowship

[Amogh Desai]

Hey Everyone,

I also worked with Alex and his team during the fall of 2023 and it was an
amazing experience.

Mentoring some young college folks into a project foreign to them is very
rewarding both for them as well as
for the mentor. As a mentor, i learnt a lot during this process due to the
beginner doubts and issues the mentees
ran into which we often ignore as solving such issues seems trivial to us.
This led to a lot of improvements
in breeze as well during that time.

Similar to Dennis, I won't be able to commit the time required (which isn't
a lot btw) and I also want to give this
chance to others as well :)

Thanks & Regards,
Amogh Desai


On Sat, Apr 5, 2025 at 2:32 AM Ferruzzi, Dennis 
wrote:

> Hey.  As Alex mentioned, I've worked with them for a couple of cohorts.
> The only reason I'm not doing the next one is because I have a lot going on
> in Real Life this summer and won't be able to commit the time.  If anyone
> is interested but wants to chat about what it's like, feel free to hit me
> up on the Slack channel.
>
>
>  - ferruzzi
>
>
> 
> From: karan alang 
> Sent: Thursday, April 3, 2025 4:38 PM
> To: dev@airflow.apache.org; al...@majorleaguehacking.com
> Subject: RE: [EXT] Airflow Mentors for Summer 2025 MLH Fellowship
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe.
> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez
> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que
> le contenu ne présente aucun risque.
>
>
>
> Hi Alex,
>
> I'm happy to help out with this.
>
> regds,
> Karan Alang
>
>
> On Thu, Apr 3, 2025 at 4:13 PM Alex Gornet
>  wrote:
>
> > Hi Airflow team,
> >
> > I'm Alex Gornet, a Partner Success Manager at Major League Hacking. Here
> at
> > MLH we run an Open Source Fellowship program
> >  that functions like an internship
> for
> > student developers to contribute to open source projects. *Fellows from
> our
> > program have successfully contributed to Airflow since Fall 2024. *
> >
> > This year Royal Bank of Canada is sponsoring some seats in the program,
> and
> > they'd like their Fellows to contribute to Airflow as a way to give back
> to
> > the community. *With the recent release of Airflow 3.0, these Fellows
> would
> > be great candidates for tackling the backlog of bugs / issues following
> the
> > transition. *
> >
> > *Right now, we're looking for maintainers / core contributors to provide
> > technical mentorship to the students*, so we'd love your help to
> identify 2
> > maintainers / core contributors (or more!) who could take lead here and
> > support us bringing on some great new contributors.
> >
> >- *The program runs from May 19th to August 8th*, and we'd ask for ~2
> >hours of your time each week.
> >
> > *If this is something you'd be interested in helping out with, you can
> > simply reply to this email or reach out to me directly at
> > al...@majorleaguehacking.com * — since
> we're
> > not too far out from program launch, the sooner you can let us know the
> > better!
> >
> > If you have any questions about the program or want to chat more about
> it,
> > let me know and I'm happy to find time. You can also reach out to Dennis
> > Ferruzzi who has mentored the last couple of cohorts to hear more about
> his
> > mentoring experience.
> >
> > Best,
> > Alex
> >
>

Re: [DISCUSS] New Provider for Gremlin

[Stephen Mallette]

Hi everyone, I just wanted to check in on the status of the TinkerPop
Provider PR:

https://github.com/apache/airflow/pull/47446

I don't know if there is any outstanding work left here but I was curious
if there was any sense from committers as to when this body of work will be
merged and made available in a release?

Thanks,

Stephen


On Sat, Mar 8, 2025 at 2:57 PM Paul King  wrote:

>
> Nice to see the Apache TinkerPop/Gremlin support. As well as using Gremlin
> with TinkerPop's TinkerGraph, I have also used it with OrientDB, ArcadeDB,
> and Apache HugeGraph. If anyone is interested, I did a blog post here:
> https://groovy.apache.org/blog/groovy-graph-databases
> (Sorry but it's Groovy not Python)
>
> Cheers, Paul.
>
> On 2025/03/08 11:34:11 Ahmad Farhan wrote:
> > Hi!
> > I managed to do the last cleanup on the PR, and is ready for thorough
> > review. I will create a  `[LAZY CONSENSUS]` thread on Monday/Tuesday.
> Feel
> > free to review and comment on the PR in the meantime.
> >
> > Farhan
> >
> > On Thu, Mar 6, 2025 at 1:40 PM Stephen Mallette 
> > wrote:
> >
> > > Hi Ahmad, thanks for the updates! I've spent some time looking at the
> > > changes and have added some comments and questions. I've pointed out a
> > > couple things for future work that I think will be important, but
> nothing
> > > that needs to be changed for this PR in my mind. Looking forward to
> seeing
> > > your responses, other community feedback, and ultimately a merge of the
> > > provider - take care!
> > >
> > > On Thu, Mar 6, 2025 at 7:28 AM Ahmad Farhan <
> ahmad.farhan9...@gmail.com>
> > > wrote:
> > >
> > > > Hello again!
> > > > After several days of debugging and branch restructuring I managed to
> > > > create another PR https://github.com/apache/airflow/pull/47446,
> this is
> > > > due
> > > > to major merge conflicts after the www clean up.
> > > > The renaming of the provider to apache-tinkerpop is done but I kept
> the
> > > > integration testing as 'gremlin' to avoid the conflict/confusion
> with the
> > > > server name within the CI.
> > > >
> > > > Please review the PR thoroughly and I am happy to work on the
> following
> > > > steps.
> > > >
> > > > Thanks,
> > > > Farhan
> > > >
> > > > On Wed, Feb 26, 2025 at 7:43 PM Ahmad Farhan <
> ahmad.farhan9...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I did look into the naming and I thought that it would need to be
> > > > > discussed at some point before the lazy consensus stage after the
> dev
> > > > work
> > > > > is done but I guess I was wrong :)
> > > > >
> > > > > I read through some docs regarding the naming and I kept thinking
> that
> > > > > Apache Gremlin might not be right, so I decided to remove 'Apache'
> from
> > > > all
> > > > > doc strings. One thing that popped in one of the documentations
> from
> > > > > Microsoft that says Apache Gremlin (here
> > > > >
> https://learn.microsoft.com/en-us/azure/cosmos-db/gremlin/introduction
> > > )
> > > > > which confused me.
> > > > >
> > > > > I will change the folder to apache/tinkerpop. Thanks Stephen for
> the
> > > > > in-depth explanation.
> > > > >
> > > > > On Wed, Feb 26, 2025 at 6:56 PM Jarek Potiuk 
> wrote:
> > > > >
> > > > >> Cool. I will let Ahmad comment, but I think we found the
> **someone**
> > > who
> > > > >> will help in case there are some future issues with the
> > > > Tinkerpop/Gremlin
> > > > >> provider.
> > > > >> While I like Gremlin better (it's just a cool name and I like the
> > > logo,
> > > > >> tinkerpop has a cool logo as well
> > > > https://tinkerpop.apache.org/index.html
> > > > >> ).
> > > > >>
> > > > >> So as long as we decide not to use common.graph -> I am fine with
> both
> > > > :)
> > > > >>
> > > > >> j.
> > > > >>
> > > > >>
> > > > >> On Wed, Feb 26, 2025 at 7:42 PM Stephen Mallette <
> > > spmalle...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > On Wed, Feb 26, 2025 at 12:57 PM Jarek Potiuk  >
> > > > wrote:
> > > > >> >
> > > > >> > > > In the interest of ASF trademarks, I would suggest it be
> called
> > > > >> > > "apache/tinkerpop" with "Gremlin" naming reserved for
> operators
> > > and
> > > > >> the
> > > > >> > > like, as it is now with GremlinOperator. I think this makes
> sense
> > > > >> because
> > > > >> > > it is connecting to TinkerPop-enabled systems via Gremlin. I
> would
> > > > >> > > similarly suggest that references to "Apache Gremlin" and the
> like
> > > > >> become
> > > > >> > > "Apache TinkerPop".
> > > > >> > >
> > > > >> > > That's an interesting one - indeed TinkerPop is the PMC/
> > > Framework -
> > > > >> > > Gremlin is the language.
> > > > >> > >
> > > > >> > > I am not sure we are actually using TinkerPop here - because
> > > > >> TinkerPop is
> > > > >> > > the whole framework - Ahmad, can you explain the relation
> there -
> > > > are
> > > > >> > those
> > > > >> > > other systems simply implement Gremlin as language or do they
> use
> > > > >> > TinkerPop
> > > > >> > > for something / as a backend?

Re: [VOTE] Release Airflow 3.0.0 from 3.0.0rc1 & Task SDK 1.0.0 from 1.0.0rc1

[Amogh Desai]

Thanks Kaxil, Ash and others for the work and sleepless nights!!


Thanks & Regards,
Amogh Desai


On Fri, Apr 4, 2025 at 5:18 AM Kaxil Naik  wrote:

> Couldn’t sleep so there you go!
>
> Following image is now available
>
> docker pull apache/airflow:3.0.0.rc1.post4
> docker pull apache/airflow:3.0.0.rc1.post4-python3.12
> docker pull apache/airflow:3.0.0.rc1.post4-python3.11
> docker pull apache/airflow:3.0.0.rc1.post4-python3.10
>
>
> On Fri, 4 Apr 2025 at 03:37, Vikram Koka 
> wrote:
>
> > Thanks for the update, Ash!
> >
> > I am running RC1 based on the Python package above.
> >
> > Best regards,
> > Vikram
> >
> >
> > On Thu, Apr 3, 2025 at 2:30 PM Ash Berlin-Taylor  wrote:
> >
> > > Anyone waiting for the docker images is going to have to wait until
> > > tomorrow, (or perhaps even Monday) as the build isn’t currently
> behaving
> > > itself after the split of airflow-core and the new meta package airflow
> > >
> > >   #95 5.136 The conflict is caused by:
> > >   #95 5.136 The user requested apache-airflow-core==3.0.0rc1.post1
> > >   #95 5.136 apache-airflow 3.0.0rc1.post1 depends on
> > > apache-airflow-core==3.0.0.rc1
> > >
> > > It’s a quirk of the RC naming, we’ll fix it and get the docker images
> > > build.
> > >
> > > -ash
> > >
> > > > On 3 Apr 2025, at 22:12, Vikram Koka 
> > > wrote:
> > > >
> > > > Awesome!
> > > > Thank you Kaxil for all your work and also thank you to all the
> > > > contributors whose hard work and dedication made this release a
> > reality.
> > > >
> > > > Best regards,
> > > > Vikram
> > > >
> > > >
> > > > On Thu, Apr 3, 2025 at 2:08 PM Kaxil Naik 
> wrote:
> > > >
> > > >> Docker images will be out soon too.
> > > >>
> > > >> On Fri, 4 Apr 2025 at 02:35, Kaxil Naik 
> wrote:
> > > >>
> > > >>> Hey fellow Airflowers,
> > > >>>
> > > >>> I am thrilled to announce the availability of Apache Airflow
> > 3.0.0rc1 &
> > > >> *Task
> > > >>> SDK 1.0.0rc1* for testing! Airflow 3.0 marks a significant
> milestone
> > as
> > > >>> the first major release in over four years, introducing
> improvements
> > > that
> > > >>> enhance user experience, task execution, and system scalability.
> > > >>>
> > > >>> This email is calling for a vote on the release,
> > > >>> which will last at least 7 days until 10th April.
> > > >>> and until 3 binding +1 votes have been received.
> > > >>>
> > > >>> Consider this my (non-binding) +1.
> > > >>>
> > > >>> Airflow 3.0.0rc1 is available at:
> > > >>> https://dist.apache.org/repos/dist/dev/airflow/3.0.0rc1/
> > > >>>
> > > >>>
> > > >>> "apache-airflow" Meta package:
> > > >>>
> > > >>>
> > > >>>   - *apache-airflow-3.0.0-source.tar.gz* is a source release that
> > comes
> > > >>>   with INSTALL instructions.
> > > >>>   - *apache-airflow-3.0.0.tar.gz* is the binary Python "sdist"
> > release.
> > > >>>   - *apache_airflow-3.0.0-py3-none-any.whl* is the binary Python
> > > >>>   wheel "binary" release.
> > > >>>
> > > >>> "apache-airflow-core" package
> > > >>>
> > > >>>
> > > >>>   - *apache_airflow_core-3.0.0.tar.gz* is the binary Python "sdist"
> > > >>>   release.
> > > >>>   - *apache_airflow_3.0.0-py3-none-any.whl* is the binary Python
> > > >>>   wheel "binary" release.
> > > >>>
> > > >>>
> > > >>> Task SDK 1.0.0rc1 is available at:
> > > >>> https://dist.apache.org/repos/dist/dev/airflow/task-sdk/1.0.0rc1/
> > > >>>
> > > >>>
> > > >>> "apache-airflow-task-sdk" package
> > > >>>
> > > >>>   - *apache-airflow-task-sdk-1.0.0-source.tar.gz* is a source
> release
> > > >>>   - *apache_airflow_task_sdk-1.0.0.tar.gz* is the binary Python
> > "sdist"
> > > >>>   release.
> > > >>>   - *apache_airflow_task_sdk-1.0.0-py3-none-any.whl* is the binary
> > > >>>   Python wheel "binary" release.
> > > >>>
> > > >>>
> > > >>>
> > > >>> Public keys are available at:
> > > >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> > > >>>
> > > >>> Please vote accordingly:
> > > >>>
> > > >>> [ ] +1 approve
> > > >>> [ ] +0 no opinion
> > > >>> [ ] -1 disapprove with the reason
> > > >>>
> > > >>> Only votes from PMC members are binding, but all members of the
> > > community
> > > >>> are encouraged to test the release and vote with "(non-binding)".
> > > >>>
> > > >>> The test procedure for PMC members is described in:
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members
> > > >>>
> > > >>> The test procedure for contributors and members of the community
> who
> > > >> would
> > > >>> like to test this RC is described in:
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-contributors
> > > >>>
> > > >>> Please note that the version number excludes the 'rcX' string, so
> > it's
> > > >> now
> > > >>> simply 3.0.0 for Airflow package and 1.0.0 for Task SDK. This will
> > > allow
> > > >>> us to rename the artifact without modifying
> >

Re: [VOTE] Release Airflow 3.0.0 from 3.0.0rc1 & Task SDK 1.0.0 from 1.0.0rc1

[Jarek Potiuk]

Cool.. Fantastic job :)



On Fri, Apr 4, 2025 at 10:27 AM Amogh Desai 
wrote:

> Thanks Kaxil, Ash and others for the work and sleepless nights!!
>
>
> Thanks & Regards,
> Amogh Desai
>
>
> On Fri, Apr 4, 2025 at 5:18 AM Kaxil Naik  wrote:
>
> > Couldn’t sleep so there you go!
> >
> > Following image is now available
> >
> > docker pull apache/airflow:3.0.0.rc1.post4
> > docker pull apache/airflow:3.0.0.rc1.post4-python3.12
> > docker pull apache/airflow:3.0.0.rc1.post4-python3.11
> > docker pull apache/airflow:3.0.0.rc1.post4-python3.10
> >
> >
> > On Fri, 4 Apr 2025 at 03:37, Vikram Koka 
> > wrote:
> >
> > > Thanks for the update, Ash!
> > >
> > > I am running RC1 based on the Python package above.
> > >
> > > Best regards,
> > > Vikram
> > >
> > >
> > > On Thu, Apr 3, 2025 at 2:30 PM Ash Berlin-Taylor 
> wrote:
> > >
> > > > Anyone waiting for the docker images is going to have to wait until
> > > > tomorrow, (or perhaps even Monday) as the build isn’t currently
> > behaving
> > > > itself after the split of airflow-core and the new meta package
> airflow
> > > >
> > > >   #95 5.136 The conflict is caused by:
> > > >   #95 5.136 The user requested
> apache-airflow-core==3.0.0rc1.post1
> > > >   #95 5.136 apache-airflow 3.0.0rc1.post1 depends on
> > > > apache-airflow-core==3.0.0.rc1
> > > >
> > > > It’s a quirk of the RC naming, we’ll fix it and get the docker images
> > > > build.
> > > >
> > > > -ash
> > > >
> > > > > On 3 Apr 2025, at 22:12, Vikram Koka  >
> > > > wrote:
> > > > >
> > > > > Awesome!
> > > > > Thank you Kaxil for all your work and also thank you to all the
> > > > > contributors whose hard work and dedication made this release a
> > > reality.
> > > > >
> > > > > Best regards,
> > > > > Vikram
> > > > >
> > > > >
> > > > > On Thu, Apr 3, 2025 at 2:08 PM Kaxil Naik 
> > wrote:
> > > > >
> > > > >> Docker images will be out soon too.
> > > > >>
> > > > >> On Fri, 4 Apr 2025 at 02:35, Kaxil Naik 
> > wrote:
> > > > >>
> > > > >>> Hey fellow Airflowers,
> > > > >>>
> > > > >>> I am thrilled to announce the availability of Apache Airflow
> > > 3.0.0rc1 &
> > > > >> *Task
> > > > >>> SDK 1.0.0rc1* for testing! Airflow 3.0 marks a significant
> > milestone
> > > as
> > > > >>> the first major release in over four years, introducing
> > improvements
> > > > that
> > > > >>> enhance user experience, task execution, and system scalability.
> > > > >>>
> > > > >>> This email is calling for a vote on the release,
> > > > >>> which will last at least 7 days until 10th April.
> > > > >>> and until 3 binding +1 votes have been received.
> > > > >>>
> > > > >>> Consider this my (non-binding) +1.
> > > > >>>
> > > > >>> Airflow 3.0.0rc1 is available at:
> > > > >>> https://dist.apache.org/repos/dist/dev/airflow/3.0.0rc1/
> > > > >>>
> > > > >>>
> > > > >>> "apache-airflow" Meta package:
> > > > >>>
> > > > >>>
> > > > >>>   - *apache-airflow-3.0.0-source.tar.gz* is a source release that
> > > comes
> > > > >>>   with INSTALL instructions.
> > > > >>>   - *apache-airflow-3.0.0.tar.gz* is the binary Python "sdist"
> > > release.
> > > > >>>   - *apache_airflow-3.0.0-py3-none-any.whl* is the binary Python
> > > > >>>   wheel "binary" release.
> > > > >>>
> > > > >>> "apache-airflow-core" package
> > > > >>>
> > > > >>>
> > > > >>>   - *apache_airflow_core-3.0.0.tar.gz* is the binary Python
> "sdist"
> > > > >>>   release.
> > > > >>>   - *apache_airflow_3.0.0-py3-none-any.whl* is the binary Python
> > > > >>>   wheel "binary" release.
> > > > >>>
> > > > >>>
> > > > >>> Task SDK 1.0.0rc1 is available at:
> > > > >>>
> https://dist.apache.org/repos/dist/dev/airflow/task-sdk/1.0.0rc1/
> > > > >>>
> > > > >>>
> > > > >>> "apache-airflow-task-sdk" package
> > > > >>>
> > > > >>>   - *apache-airflow-task-sdk-1.0.0-source.tar.gz* is a source
> > release
> > > > >>>   - *apache_airflow_task_sdk-1.0.0.tar.gz* is the binary Python
> > > "sdist"
> > > > >>>   release.
> > > > >>>   - *apache_airflow_task_sdk-1.0.0-py3-none-any.whl* is the
> binary
> > > > >>>   Python wheel "binary" release.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Public keys are available at:
> > > > >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> > > > >>>
> > > > >>> Please vote accordingly:
> > > > >>>
> > > > >>> [ ] +1 approve
> > > > >>> [ ] +0 no opinion
> > > > >>> [ ] -1 disapprove with the reason
> > > > >>>
> > > > >>> Only votes from PMC members are binding, but all members of the
> > > > community
> > > > >>> are encouraged to test the release and vote with "(non-binding)".
> > > > >>>
> > > > >>> The test procedure for PMC members is described in:
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members
> > > > >>>
> > > > >>> The test procedure for contributors and members of the community
> > who
> > > > >> would
> > > > >>> like to test this RC is described in:
> > > > >>>
> > > > >>

Re: [DISCUSS] New Provider for Gremlin

[Ahmad Farhan]

Hey Stephen!

I am still waiting for review from code maintainers and they are currently
busy with Airflow 3. Also, recently there had been major code changes
(after uv PR) that are raising a lot of errors so some work is needed on my
side which is WIP.

Thanks,
Farhan

On Fri, 4 Apr 2025 at 1:47 pm, Stephen Mallette 
wrote:

> Hi everyone, I just wanted to check in on the status of the TinkerPop
> Provider PR:
>
> https://github.com/apache/airflow/pull/47446
>
> I don't know if there is any outstanding work left here but I was curious
> if there was any sense from committers as to when this body of work will be
> merged and made available in a release?
>
> Thanks,
>
> Stephen
>
>
> On Sat, Mar 8, 2025 at 2:57 PM Paul King  wrote:
>
> >
> > Nice to see the Apache TinkerPop/Gremlin support. As well as using
> Gremlin
> > with TinkerPop's TinkerGraph, I have also used it with OrientDB,
> ArcadeDB,
> > and Apache HugeGraph. If anyone is interested, I did a blog post here:
> > https://groovy.apache.org/blog/groovy-graph-databases
> > (Sorry but it's Groovy not Python)
> >
> > Cheers, Paul.
> >
> > On 2025/03/08 11:34:11 Ahmad Farhan wrote:
> > > Hi!
> > > I managed to do the last cleanup on the PR, and is ready for thorough
> > > review. I will create a  `[LAZY CONSENSUS]` thread on Monday/Tuesday.
> > Feel
> > > free to review and comment on the PR in the meantime.
> > >
> > > Farhan
> > >
> > > On Thu, Mar 6, 2025 at 1:40 PM Stephen Mallette 
> > > wrote:
> > >
> > > > Hi Ahmad, thanks for the updates! I've spent some time looking at the
> > > > changes and have added some comments and questions. I've pointed out
> a
> > > > couple things for future work that I think will be important, but
> > nothing
> > > > that needs to be changed for this PR in my mind. Looking forward to
> > seeing
> > > > your responses, other community feedback, and ultimately a merge of
> the
> > > > provider - take care!
> > > >
> > > > On Thu, Mar 6, 2025 at 7:28 AM Ahmad Farhan <
> > ahmad.farhan9...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello again!
> > > > > After several days of debugging and branch restructuring I managed
> to
> > > > > create another PR https://github.com/apache/airflow/pull/47446,
> > this is
> > > > > due
> > > > > to major merge conflicts after the www clean up.
> > > > > The renaming of the provider to apache-tinkerpop is done but I kept
> > the
> > > > > integration testing as 'gremlin' to avoid the conflict/confusion
> > with the
> > > > > server name within the CI.
> > > > >
> > > > > Please review the PR thoroughly and I am happy to work on the
> > following
> > > > > steps.
> > > > >
> > > > > Thanks,
> > > > > Farhan
> > > > >
> > > > > On Wed, Feb 26, 2025 at 7:43 PM Ahmad Farhan <
> > ahmad.farhan9...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > I did look into the naming and I thought that it would need to be
> > > > > > discussed at some point before the lazy consensus stage after the
> > dev
> > > > > work
> > > > > > is done but I guess I was wrong :)
> > > > > >
> > > > > > I read through some docs regarding the naming and I kept thinking
> > that
> > > > > > Apache Gremlin might not be right, so I decided to remove
> 'Apache'
> > from
> > > > > all
> > > > > > doc strings. One thing that popped in one of the documentations
> > from
> > > > > > Microsoft that says Apache Gremlin (here
> > > > > >
> > https://learn.microsoft.com/en-us/azure/cosmos-db/gremlin/introduction
> > > > )
> > > > > > which confused me.
> > > > > >
> > > > > > I will change the folder to apache/tinkerpop. Thanks Stephen for
> > the
> > > > > > in-depth explanation.
> > > > > >
> > > > > > On Wed, Feb 26, 2025 at 6:56 PM Jarek Potiuk 
> > wrote:
> > > > > >
> > > > > >> Cool. I will let Ahmad comment, but I think we found the
> > **someone**
> > > > who
> > > > > >> will help in case there are some future issues with the
> > > > > Tinkerpop/Gremlin
> > > > > >> provider.
> > > > > >> While I like Gremlin better (it's just a cool name and I like
> the
> > > > logo,
> > > > > >> tinkerpop has a cool logo as well
> > > > > https://tinkerpop.apache.org/index.html
> > > > > >> ).
> > > > > >>
> > > > > >> So as long as we decide not to use common.graph -> I am fine
> with
> > both
> > > > > :)
> > > > > >>
> > > > > >> j.
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Feb 26, 2025 at 7:42 PM Stephen Mallette <
> > > > spmalle...@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > On Wed, Feb 26, 2025 at 12:57 PM Jarek Potiuk <
> ja...@potiuk.com
> > >
> > > > > wrote:
> > > > > >> >
> > > > > >> > > > In the interest of ASF trademarks, I would suggest it be
> > called
> > > > > >> > > "apache/tinkerpop" with "Gremlin" naming reserved for
> > operators
> > > > and
> > > > > >> the
> > > > > >> > > like, as it is now with GremlinOperator. I think this makes
> > sense
> > > > > >> because
> > > > > >> > > it is connecting to TinkerPop-enabled systems via Gremlin. I
> > would
> > > > > >> > > similarly

[ANNOUNCE] Apache Airflow Providers prepared on March 26, 2025 are released

[Elad Kalif]

Dear Airflow community,

I'm happy to announce that new versions of Airflow Providers packages
prepared on March 26, 2025 were just released. Full list of PyPI packages
released is added at the end of the message.

The source release, as well as the binary releases, are available here:

https://airflow.apache.org/docs/apache-airflow-providers/installing-from-sources

You can install the providers via PyPI:
https://airflow.apache.org/docs/apache-airflow-providers/installing-from-pypi

The documentation is available at https://airflow.apache.org/docs/ and
linked from the PyPI packages.



Full list of released PyPI packages:

https://pypi.org/project/apache-airflow-providers-apache-hive/9.0.4/
https://pypi.org/project/apache-airflow-providers-apache-kafka/1.8.0/
https://pypi.org/project/apache-airflow-providers-apache-kylin/3.8.2/
https://pypi.org/project/apache-airflow-providers-apache-spark/5.1.1/
https://pypi.org/project/apache-airflow-providers-apprise/2.0.2/
https://pypi.org/project/apache-airflow-providers-atlassian-jira/3.0.2/
https://pypi.org/project/apache-airflow-providers-celery/3.10.4/
https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.4.0/
https://pypi.org/project/apache-airflow-providers-common-compat/1.6.0/
https://pypi.org/project/apache-airflow-providers-common-io/1.5.2/
https://pypi.org/project/apache-airflow-providers-common-sql/1.24.1/
https://pypi.org/project/apache-airflow-providers-databricks/7.2.2/
https://pypi.org/project/apache-airflow-providers-dbt-cloud/4.3.1/
https://pypi.org/project/apache-airflow-providers-discord/3.9.4/
https://pypi.org/project/apache-airflow-providers-docker/4.3.0/
https://pypi.org/project/apache-airflow-providers-google/14.1.0/
https://pypi.org/project/apache-airflow-providers-microsoft-azure/12.2.2/
https://pypi.org/project/apache-airflow-providers-microsoft-winrm/3.9.2/
https://pypi.org/project/apache-airflow-providers-openai/1.5.3/
https://pypi.org/project/apache-airflow-providers-opsgenie/5.8.3/
https://pypi.org/project/apache-airflow-providers-pagerduty/4.0.3/
https://pypi.org/project/apache-airflow-providers-presto/5.8.3/
https://pypi.org/project/apache-airflow-providers-slack/9.0.3/
https://pypi.org/project/apache-airflow-providers-smtp/2.0.2/
https://pypi.org/project/apache-airflow-providers-standard/0.2.0/
https://pypi.org/project/apache-airflow-providers-trino/6.1.1/
https://pypi.org/project/apache-airflow-providers-yandex/4.0.3/

Cheers,
Elad Kalif

Re: Airflow Mentors for Summer 2025 MLH Fellowship

[karan alang]

Hi Alex,

I'm happy to help out with this.

regds,
Karan Alang


On Thu, Apr 3, 2025 at 4:13 PM Alex Gornet
 wrote:

> Hi Airflow team,
>
> I'm Alex Gornet, a Partner Success Manager at Major League Hacking. Here at
> MLH we run an Open Source Fellowship program
>  that functions like an internship for
> student developers to contribute to open source projects. *Fellows from our
> program have successfully contributed to Airflow since Fall 2024. *
>
> This year Royal Bank of Canada is sponsoring some seats in the program, and
> they'd like their Fellows to contribute to Airflow as a way to give back to
> the community. *With the recent release of Airflow 3.0, these Fellows would
> be great candidates for tackling the backlog of bugs / issues following the
> transition. *
>
> *Right now, we're looking for maintainers / core contributors to provide
> technical mentorship to the students*, so we'd love your help to identify 2
> maintainers / core contributors (or more!) who could take lead here and
> support us bringing on some great new contributors.
>
>- *The program runs from May 19th to August 8th*, and we'd ask for ~2
>hours of your time each week.
>
> *If this is something you'd be interested in helping out with, you can
> simply reply to this email or reach out to me directly at
> al...@majorleaguehacking.com * — since we're
> not too far out from program launch, the sooner you can let us know the
> better!
>
> If you have any questions about the program or want to chat more about it,
> let me know and I'm happy to find time. You can also reach out to Dennis
> Ferruzzi who has mentored the last couple of cohorts to hear more about his
> mentoring experience.
>
> Best,
> Alex
>

Re: Should we drop support for pre_execute & post_execute for AF 3.0

[Jarek Potiuk]

Yeah. I am convinced by Bolke's and Constance posts. Changing my vote to
-0.9.

On Sun, Mar 30, 2025 at 8:25 PM Constance Martineau
 wrote:

> Just to add to the discussion:
>
> Removing these overrides will disproportionately impact teams at companies
> with more than a handful of Airflow users. Unless there’s a clear benefit
> to removing them *and* a well-defined migration path, I don’t think it’s
> worth the disruption. +1 to Bolke’s points.
>
> Like Tamara mentioned, we frequently see teams build internal providers
> where they subclass standard operators and use pre_execute/post_execute to
> inject things like service-specific auth, tracking identifiers, or other
> cross-cutting concerns. These hooks act as a central entry point for
> operational consistency, especially in environments where DAG authors
> aren’t expected to manage those behaviors themselves.
>
> Pushing that logic down to every DAG or task definition would be a pretty
> big shift for those teams, and, from what I’ve seen, a regression in terms
> of maintainability and user experience.
>
> On Sun, Mar 30, 2025 at 4:33 AM Michał Modras
>  wrote:
>
> > +1 to Bolke's points
> >
> > niedz., 30 mar 2025, 10:30 użytkownik Bolke de Bruin 
> > napisał:
> >
> > > I would be in favor of removing the experimental feature of the
> > constructor
> > > arguments, but I don't understand the reasoning for removing the
> > override.
> > > It is a breaking change from something that was there since 1.X so not
> > > really experimental anymore. I think you might underestimate how much
> it
> > is
> > > used. What would be the migration path? It does not execute the same as
> > > TearDown / Setup and Moving it to a constructor argument requires it to
> > be
> > > a part of a DAG, limiting deployment options.
> > >
> > > B.
> > >
> > >
> > >
> > > On Sat, 29 Mar 2025 at 13:56, Kaxil Naik  wrote:
> > >
> > > > @tamara : Correct, we are now proposing to remove overriding the
> > pre/post
> > > > execute
> > > >
> > > > >Quick question if I am understanding the proposed change correctly.
> > > >
> > > >
> > > > What you want to remove is overriding the pre/post execute when
> > creating
> > > > > custom operators:
> > > > >
> > > > > class MyOperator(BaseOperator):
> > > > >
> > > > > ...
> > > > >
> > > > > def pre_execute(self, context):   # This would break?
> > > > > 
> > > > > def post_execute(self, context):   # And this as well?
> > > > > But keep the (currently experimental) use of the pre_execute and
> > > > > post_execute parameters (I've only used post_execute before for
> > similar
> > > > > reasons as TP posted, interacting with outlets assets)
> > > >
> > > >
> > > > On Sat, 29 Mar 2025 at 18:23, Kaxil Naik 
> wrote:
> > > >
> > > > > No, @run_if / @skip_if uses pre_execute from task argument [1] not
> > the
> > > > the
> > > > > method and is just a syntactic sugar. You can also do the following
> > as
> > > an
> > > > > example:
> > > > >
> > > > > ```
> > > > > def skip_at_random(context):
> > > > > if randint(0, 1) == 0:
> > > > > raise AirflowSkipException()
> > > > >
> > > > > t2 = BashOperator(task_id='conditional2',
> pre_execute=skip_at_random,
> > > > > dag=dag, bash_command="airflow version")
> > > > > ```
> > > > >
> > > > > [1]:
> > > > >
> > > >
> > >
> >
> https://github.com/apache/airflow/blob/8c3a30e3ffc3f114c1d2cc3e6e109f4d9e29ca8b/airflow-core/src/airflow/decorators/condition.py#L57-L59
> > > > >
> > > > > On Sat, 29 Mar 2025 at 05:09, Matthew Block <
> > matthew.l.bl...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Would this also break @run_if/@skip_if decorators?
> > > > >>
> > > > >> Best,
> > > > >> Matt Block
> > > > >>
> > > > >> > On Mar 28, 2025, at 3:44 PM, Tamara Fingerlin
> > > > >>  wrote:
> > > > >> >
> > > > >> > Hey :)
> > > > >> >
> > > > >> > Quick question if I am understanding the proposed change
> > correctly.
> > > > >> >
> > > > >> > What you want to remove is overriding the pre/post execute when
> > > > creating
> > > > >> > custom operators:
> > > > >> >
> > > > >> > class MyOperator(BaseOperator):
> > > > >> >...
> > > > >> >def pre_execute(self, context):   # This would break?
> > > > >> >
> > > > >> >
> > > > >> >def post_execute(self, context):   # And this as well?
> > > > >> >
> > > > >> >
> > > > >> > But keep the (currently experimental) use of the pre_execute and
> > > > >> > post_execute parameters (I've only used post_execute before for
> > > > similar
> > > > >> > reasons as TP posted, interacting with outlets assets)
> > > > >> >
> > > > >> >BashOperator(
> > > > >> >task_id='hello_world',
> > > > >> >bash_command='sleep 5',
> > > > >> >pre_execute=lambda context: print("Pre-execute function
> > > > >> called!"),
> > > > >> > # this would still work?
> > > > >> >post_execute=lambda context: print("Post-execute function
> > > > >> > called!"),   # this would be supposed to still work 

Re: Clean up the task context dict

[Jarek Potiuk]

> There was an issue a while ago proposing to add more values into the
context dict.[1] I responded at the time that the proposed values are
already available by accessing 'ti' although it’s unclear whether the
internals of the variable (which was a concrete SQLAlchemy model
TaskInstance at the time) should be considered public.
> Fast forward 11 months, with Task SDK implemented, 'ti' (and other things
that used to be SQLAlchemy models, e.g. dag_run) are now only simple data
constructs that do not provide any server-side functionalities.

This is really cool and a great achievement. So much better than AIP-44
version of it when we had Pydantic "variants" of those

> We should first expand the template documentation to outline what
attributes are available on the task instance, and encourage users to use
them.[2]

Absolutely. that should also be explicitly called out in the
https://airflow.apache.org/docs/apache-airflow/stable/public-airflow-interface.html
to mention that the whole context is public interface and link to the exact
specification of it that is documented, maintained, versioned, and we
should make sure that any change to it is reflected in "changelog" -
because that will now become linked to particular task-sdk version. Also I
think what is worth defining is how this relates to the versions of
distributions used.

Since task-sdk will be "freely" upgradeable and independent of the Airflow
version, we should have some rules describing what to expect there. Are
"new fields" we are going to add there going to be connected to "airflow"
version or "task_sdk" version. I think we need to agree and document
expected behaviour here - not only current snapshot, so that our users know
what to expect. Including deprecation patterns.

> Furthermore, I am also wondering if we should remove try_number and
run_id from the context dict. This is the exact same value as on the task
instance, and they stand out a bit as not really necessary.

I think that might be a bit early, but I would be very much (as long as we
agree on deprecation patterns) to deprecate them now rather than remove (to
ease airflow 3 migration) - but then agree on some deprecation rules that
we could follow and remove them in not-so-distant future.

J.



On Tue, Mar 25, 2025 at 10:04 AM Tzu-ping Chung 
wrote:

> Hi all,
>
> There was an issue a while ago proposing to add more values into the
> context dict.[1] I responded at the time that the proposed values are
> already available by accessing 'ti' although it’s unclear whether the
> internals of the variable (which was a concrete SQLAlchemy model
> TaskInstance at the time) should be considered public.
>
> Fast forward 11 months, with Task SDK implemented, 'ti' (and other things
> that used to be SQLAlchemy models, e.g. dag_run) are now only simple data
> constructs that do not provide any server-side functionalities.
>
> We should first expand the template documentation to outline what
> attributes are available on the task instance, and encourage users to use
> them.[2]
>
> - dag_id
> - run_id
> - task_id
> - try_number
> - map_index
> - hostname (not totally sure about this)
>
> Furthermore, I am also wondering if we should remove try_number and run_id
> from the context dict. This is the exact same value as on the task
> instance, and they stand out a bit as not really necessary.
>
> TP
>
>
> [1]: https://github.com/apache/airflow/issues/40958
> [2]:
> https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html
> -
> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> For additional commands, e-mail: dev-h...@airflow.apache.org
>
>

Re: [DISCUSS] Next steps of repo restructuring - airflow-core (finally!)

[Shahar Epstein]

That's hardcore (pun intended) :D
Great work and good luck merging it!

On Fri, Mar 14, 2025 at 9:28 PM Jarek Potiuk  wrote:

> Hey here,
>
> I have a first (very draft and still requires a number of changes) PR for
> the final step of big refactoring of our projects and using workspace. This
> is to let you know about the changes coming (so please take a look at the
> consequences to not be surprised).
>
> This is the most *scary* one -> moving all airflow code to
> "airflow-core". And I have  draft version of it in
> https://github.com/apache/airflow/pull/47798
>
> And it's not for the faint of heart :)
>
> [image: image.png]
>
> Note! It's not yet complete and unless you have some general comments,
> it's likely not worth pointing to individual changes (yet) - it's more to
> take a look at how things will look like eventually. I will work in the
> next two days to get it to  reviewable state, and will keep it rebased and
> running till mid next-week. I would like to have it ready (including the
> release process) for the fourth (and final?) beta).
>
> Some resulting packaging changes:
>
> *FOR DEVELOPMENT:*
>
> * the pyproject.toml in the "root" of Airflow is still "apache-airflow"
> package - but this will be an empty "meta" package that will install
> together "apache-airflow-core", "apache-airflow-task-sdk" and optionally
> providers (via extras)
>
> * the airflow-core is a new "apache-airflow-core" distribution, where only
> airflow dependencies and airflow "core" extras are configured (smtp/ otel,
> pandas,rabbitmq etc) - I will likely cleanup some of those as well, some of
> them are not needed. the nice thing is that this package has all
> dependencies static (no hatch_build.py - everything is in pyproject.toml) -
> which is pretty cool and allow us to better use dependabot for security
> upgrades and notifications
>
> The airflow-core structure is pretty standard:
>
> airflow-core  # <- this is folder where airflow-core distribution is
> \- src
> | \ airflow # <- This is airflow package
> | \- api
> | |- api_fastapi
> | |- assets
> | ...
> |- tests
> |   \- always
> |   |- api
> |   ...
> |- docs
> |
> |- pyproject.toml
> |- README.md
>
>
> * for development - i will describe later the `pypi` way, but with `uv`
> things get simpler and we have a few new options (Dennis - this is
> continuation of discussion on the uv sync commands, so it's worth to
> look closely:
>
> There are a number of ways you will be (eventually able to interact with
> venv. After you checkout Airflow. You can change working directory and work
> on different packages and depending on which directory you run `uv sync` -
> uv (using workspace feature) will sync the **expected** dependencies.
>
> It's best to get used to the fact that instead of one airflow project we
> will have ~100 pretty independent projects, and while you can continue
> working with all of them as a single huge "workspace", it is generally way
> more convenient to change directory to the "distribution" you are working
> on currently and do everything there - with isolated set of dependencies
> required only for that "distribution" - "airflow-core", "task-sdk",
> "providers/amazon", "providers/mongo" - those are all separate
> distributions, and more and more we will be able to treat them as
> independent projects (but we will conveniently keep the option to develop
> and run tests in a joined "workspace" environment at the top of the project
> where we can install and test everything together - that's a bit of `uv
> workspace` magic in play.
>
> Here are typical patterns:
>
> 1) Installing all development dependencies for everything (I.e complete
> environment like in breeze)  -- allows to run all tests for all airflow and
> all providers
>
> cd .
> uv sync --all-packages
>
> 2) installing just airflow core with required dependencies (ready for most
> core tests)
>
> cd airflow-core
> uv sync
>
> 3) installing airflow core with optional dependencies (should allow to run
> all core tests - including for the optional core features such as otel etc).
>
> cd airflow-core
> uv sync --all-extras
>
> 4) installing individual provider dependencies (say amazon) - this allows
> to run all tests of the provider you are working on - including installing
> all dependencies from cross-provider dependencies (i.e. if you have google
> tests in amazon provider, it will also install necessary google
> dependencies).
>
> cd providers/amazon
> uv sync
>
> Generally speaking - "airflow-core" will become (eventually) a truly
> airflow-only distribution. It will have a few dependencies to "standard"
> and "fab" providers - but I hope we will be able to get rid of those during
> the resulting cleanup.
>
> The IDE (IntelliJ) setting will just require 

Re: New committer: Rahul Vats

[Karthikeyan]

Congratulations Rahul. Thanks for all your efforts towards the Airflow 3
release including detailed bug reports that improved release stability.

On Tue, Apr 1, 2025 at 12:53 PM Kunal Bhattacharya 
wrote:

> Congratulations Rahul, very well deserved :)
>
> Regards,
> Kunal Bhattacharya
>
> On Tue, Apr 1, 2025 at 11:04 AM Amogh Desai 
> wrote:
>
> > Congratulations Rahul!
> >
> > I have first hand seen the impact you create in Airflow and the way you
> > drive
> > testing efforts. Kudos to you.
> >
> > Thanks & Regards,
> > Amogh Desai
> >
> >
> > On Tue, Apr 1, 2025 at 9:24 AM Phani Kumar
> >  wrote:
> >
> > > Congratulations Rahul 🎊
> > >
> > > On Tue, 1 Apr, 2025, 08:56 Aritra Basu, 
> > wrote:
> > >
> > > > Congrats Rahul, great job!
> > > > --
> > > > Regards,
> > > > Aritra Basu
> > > >
> > > > On Tue, 1 Apr 2025, 8:42 am kalyan reddy, 
> > wrote:
> > > >
> > > > > Congratulations Rahul! Thanks for your amazing work. Well deserved
> !!
> > > > > 
> > > > > From: Hemkumar Chheda 
> > > > > Sent: 01 April 2025 07:12
> > > > > To: dev@airflow.apache.org 
> > > > > Subject: Re: New committer: Rahul Vats
> > > > >
> > > > > Congratulations Rahul! Well Deserved 🥳🎉
> > > > >
> > > > > Best Regards,
> > > > > Hemkumar
> > > > >
> > > > > > On 1 Apr 2025, at 6:37 AM, Wei Lee  wrote:
> > > > > >
> > > > > > Congratulations!
> > > > > >
> > > > > > Best,
> > > > > > Wei
> > > > > >
> > > > > >> On Apr 1, 2025, at 8:36 AM, Zhe You Liu 
> > > wrote:
> > > > > >>
> > > > > >> Congrats Rahul, well deserved!
> > > > > >>
> > > > > >> On Tue, Apr 1, 2025 at 9:28 AM Ephraim Anierobi <
> > > > > ephraimanier...@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Congratulations Rahul! Well deserved
> > > > > >>>
> > > > > >>> On Tue, 1 Apr 2025 at 01:27, Pierre Jeambrun <
> > > pierrejb...@gmail.com>
> > > > > >>> wrote:
> > > > > >>>
> > > > >  Amazing work Rahul, congratulations.
> > > > > 
> > > > >  On Mon 31 Mar 2025 at 23:14, Pavankumar Gopidesu <
> > > > > >>> gopidesupa...@gmail.com>
> > > > >  wrote:
> > > > > 
> > > > > > Woohoo, Congratulations Rahul 🎉🎉
> > > > > >
> > > > > > Pavan.
> > > > > >
> > > > > > On Mon, Mar 31, 2025 at 9:17 PM Pankaj Koti
> > > > > >  wrote:
> > > > > >
> > > > > >> Many congratulations Rahul! Very happy for you and well
> > deserved
> > > > > 🎉🎉
> > > > > >>
> > > > > >> On Tue, 1 Apr, 2025, 01:17 Buğra Öztürk, <
> > > ozturkbugr...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >>
> > > > > >>> Amazing news! Congratulations Rahul! Well deserved!
> > > > > >>>
> > > > > >>> On Mon, Mar 31, 2025 at 9:14 PM Vincent Beck <
> > > > vincb...@apache.org>
> > > > > >> wrote:
> > > > > >>>
> > > > >  Congrats Rahul! Very well deserved!
> > > > > 
> > > > >  On 2025/03/31 19:08:09 Shubham Raj wrote:
> > > > > > Congrats Rahul, well deserved!!
> > > > > >
> > > > > > On Tue, 1 Apr 2025 at 00:33, Vishnu Chilukoori <
> > > > >  vish.chiluko...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Congrats Rahul...well deserved!!
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Regards,
> > > > > >> Vishnu Chilukoori
> > > > > >>
> > > > > >> On Mon, Mar 31, 2025 at 11:53 AM Idris Adebisi
> > > > > >>  wrote:
> > > > > >>
> > > > > >>> Congratulations, Rahul!
> > > > > >>>
> > > > > >>>   On Monday, March 31, 2025 at 07:45:27 PM GMT+1,
> Ambika
> > > > > > Garg <
> > > > > >>> ambikagarg1...@gmail.com> wrote:
> > > > > >>>
> > > > > >>> Congratulations Rahul!!
> > > > > >>> On Mon, Mar 31, 2025 at 2:43 PM Ankit Chaurasia <
> > > > > >>> sunank...@gmail.com
> > > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > >  Congratulations, Rahul! Well deserved.
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > >  On Tue, Apr 1, 2025 at 12:16 AM Constance Martineau
> > > > >   wrote:
> > > > > 
> > > > > > Congrats Rahul! Well deserved :)
> > > > > >
> > > > > > On Mon, Mar 31, 2025 at 2:28 PM Oliveira, Niko
> > > > >   > > > > >>
> > > > > > wrote:
> > > > > >
> > > > > >> Congrats Rahul, great work!
> > > > > >>
> > > > > >> 
> > > > > >> From: Jed Cunningham 
> > > > > >> Sent: Monday, March 31, 2025 11:19:53 AM
> > > > > >> To: dev@airflow.apache.org
> > > > > >> Subject: [EXT] New committer: Rahul Vats
> > > > > >>
> > > > > >> CAUTION: This email

Re: [ANNOUNCE] UV mandatory tooling and (much nicer) doc building coming

[Pavankumar Gopidesu]

That's awesome Jarek, thank you for this :)

Regards,
Pavan

On Wed, Apr 2, 2025 at 1:09 PM Jarek Potiuk  wrote:

> You might also want to do *docker system prune*  or even *docker system
> prune --all*  or run *breeze doctor* to clean-up some stale cache, images,
> docker volumes
>
> Also IntelliJ/PyCharm users *uv run setup_idea.py* will add some missing
> directories and regenerate your IntelliJ project configuration.
>
>
>
>
>
> On Wed, Apr 2, 2025 at 1:11 PM Jarek Potiuk  wrote:
>
> > Hello here,
> >
> > As part of the packaging work - I merged the
> > https://github.com/apache/airflow/pull/48223
> >
> > *TL;DR; Rebase all PRS, rebuild images and uv sync - and
> > hopefully everything should work as before even if a lot of things moved.
> > Hopefully the move will be largely transparent (except changing *include:
> > in open PRs in docs). *
> >
> > *NOTE! UV is now mandatory and a lot of code is gone thanks to that.
> > Breeze will also refuse to work if uv is not installed.*
> >
> > It took a bit of time, but we are in a much more standard and better
> shape
> > now - and as a side effect (which was intended but I had to implement it
> as
> > part of that monster PR to fix documentation) we now have a much simpler
> > (more guidelines are coming) way to iterate on doc building.
> >
> > *Few important things first: *
> >
> > 1) Make sure to rebase your PRs, Run `breeze image build`, Rnu `uv sync`.
> > Due to the way git handles things - you ** might **  have some dangling
> > generated directories in your repo and they might cause some problems.
> Run
> > "git status" after rebase and see you have some files you need to delete
> > (manually)
> >
> > 2) If you are brave enough - you might want to run `breeze doctor` and
> > cleanup git repo - it should clean all files that should be removed, but
> > also it might remove some of your custom configurations and files you
> > created,
> >
> > 3) Generally everything should work as it worked before with breeze (for
> > example `breeze build-docs` command works as before. But a number of
> > folders/distributions/code (not airflow nore providers directly) were
> > moved/updated. For now you can just continue to build docs as before -
> with
> > breeze. But simpler/faster ways are coming as follow up.
> >
> > 4) If you have some new examples or documentation included in your PRs
> the
> > doc build might start falling for you - but this is because `include::`
> or
> > `exampleinclude:`  might need to be updated - look at other examples - I
> > fixed the includes in all providers. More explanation in follow-up doc
> > build improvement PR - in the meantime, feel free to ask on slack or PR
> for
> > help.
> >
> > *Generated provider_dependencies.json do not need to be updated*
> >
> > The "generated/provider_dependencies.json" is no longer committed to the
> > repo - it is .gitignored. We are generating it as-needed on the flight.
> It
> > should be automatically regenerated when you run pre-commits locally and
> > when you build the breeze image.
> >
> > There might be some cases when we add dependencies and you will need to
> > regenerate it but that should happen automatically as needed.
> >
> > *New, updated folders*
> >
> > The change are mostly with these:
> >
> > ./dev/pyproject.toml
> > ./devel-common/pyproject.toml
> > ./doc
> > ./docker-stack-docs
> > ./providers-summary-docs
> >
> > *More explanation for distributions/folder changes*
> >
> > The dev is now a separate distribution with its own pyproject.toml
> > dependencies that are used for all the release management and general dev
> > housekeeping. This is different from "devel-common" which is a common
> > package with a lot of common code reused for tests and builds (including
> > doc builds scripts are using).
> >
> > We might want to change names later, as they are confusing but I will
> > leave that discussion for later when we complete all the isolation work -
> > with those changes I implemented it will be very easy (as opposed to how
> it
> > was) to move those distributions around - and if we will want to
> > restructure it again, that will be a very simple move.
> >
> > *Improved doc structure*
> >
> > This is because we finally made the last step - where the "doc" code is
> > moved to "devel-common" and we are importing it from there, also each
> > distribution has it's own "conf.py" and that makes modifying docs
> building
> > and our sphinx building scripts MUCH more readable ana mangeable. The
> "doc"
> > code now contains just spelling wordlist and README.md explaining where
> to
> > look for the documentation.
> >
> > Those new top-level folders ("docker-stack-docs" and
> > "providers-summary-docs") - contain the two independent pieces of our
> > documentation ("docker-stack" and "apache-airflow-providers" package
> names
> > from "breeze build-docs" command.
> >
> > The REALLY nice thing now is that the doc files are not copied between
> > places and each of the docs folder (inclu

Re: New committer: Rahul Vats

[Freddy Demiane]

Congratulations Rahul!

On Wed, Apr 2, 2025 at 12:27 PM Rahul Vats  wrote:

> Thank you, everyone. It is great to be part of the team. I truly appreciate
> all the guidance and support from the community.
>
> Regards,
> Rahul Vats
>
> On Wed, 2 Apr 2025 at 13:43, Shahar Epstein  wrote:
>
> > Congratulation Rahul! :)
> >
> > On Mon, Mar 31, 2025, 21:20 Jed Cunningham 
> > wrote:
> >
> > > The Project Management Committee (PMC) for Apache Airflow
> > > has invited Rahul Vats to become a committer and we are pleased
> > > to announce that they have accepted.
> > >
> > > Please join us in welcoming Rahul to their new role and
> > > responsibility in our project community.
> > >
> > > Rahul has had a tremendous impact on Airflow 3, primarily around
> testing.
> > > Thanks Rahul, welcome, it's well deserved!
> > >
> > > Jed
> > > On behalf of the Apache Airflow PMC
> > >
> >
>

Re: [LAZY CONSENSIS] Make git provider pre-installed for Airflow 3

[Jarek Potiuk]

Cancelling it. I did not expect objections because it ... seemed obvious to
me - but obviously it is not ... :)

 I think we can leave it as is - not preinstalled and possibly revisit it
in the future, no need to lose time now. I already have a nice way to deal
with it for the tests - where - with the new setup / uv only config we can
simply add git provider as "development" dependency, no blocker from that
part.

On Mon, Mar 31, 2025 at 8:58 PM Jarek Potiuk  wrote:

> ok. Then no consensus... Needs discussion then :)
>
> On Mon, Mar 31, 2025 at 8:05 PM Vikram Koka 
> wrote:
>
>> A little stronger than Jed actually and for very similar reasons.
>>
>> My view is that Airflow 3 is a new foundation and I very much believe that
>> there will be multiple, different bundling mechanisms.
>> I don't think we should include the "git-provider" as a required
>> dependency.
>>
>> Vikram
>>
>>
>> On Mon, Mar 31, 2025 at 10:26 AM Jed Cunningham > >
>> wrote:
>>
>> > I'm slightly against installing by default, actually. Weird from the guy
>> > who added bundle versioning huh? But, if I ignore the short term where
>> we
>> > only have git with bundle versioning, I fully expect there will be more
>> > bundle options that support versioning soon, and forcing git into the
>> mix
>> > just doesn't seem right long term.
>> >
>> > This is also just 1 aspect of "versioning" - bundle versioning gives you
>> > "dag code" consistency across a run, but you get dag versioning
>> regardless
>> > of your bundle type. It's helpful, yes, and I'd want it for production.
>> But
>> > not sure it hits the "required" bar.
>> >
>>
>

Re: New committer: Rahul Vats

[Ambika Garg]

Congratulations Rahul!!
On Mon, Mar 31, 2025 at 2:43 PM Ankit Chaurasia  wrote:

> Congratulations, Rahul! Well deserved.
>
>
>
>
>
>
>
> On Tue, Apr 1, 2025 at 12:16 AM Constance Martineau
>  wrote:
>
> > Congrats Rahul! Well deserved :)
> >
> > On Mon, Mar 31, 2025 at 2:28 PM Oliveira, Niko
>  > >
> > wrote:
> >
> > > Congrats Rahul, great work!
> > >
> > > 
> > > From: Jed Cunningham 
> > > Sent: Monday, March 31, 2025 11:19:53 AM
> > > To: dev@airflow.apache.org
> > > Subject: [EXT] New committer: Rahul Vats
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> externe.
> > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne
> > pouvez
> > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain
> > que
> > > le contenu ne présente aucun risque.
> > >
> > >
> > >
> > > The Project Management Committee (PMC) for Apache Airflow
> > > has invited Rahul Vats to become a committer and we are pleased
> > > to announce that they have accepted.
> > >
> > > Please join us in welcoming Rahul to their new role and
> > > responsibility in our project community.
> > >
> > > Rahul has had a tremendous impact on Airflow 3, primarily around
> testing.
> > > Thanks Rahul, welcome, it's well deserved!
> > >
> > > Jed
> > > On behalf of the Apache Airflow PMC
> > >
> >
>

Re: Airflow Mentors for Summer 2025 MLH Fellowship

[Ferruzzi, Dennis]

Hey.  As Alex mentioned, I've worked with them for a couple of cohorts.  The 
only reason I'm not doing the next one is because I have a lot going on in Real 
Life this summer and won't be able to commit the time.  If anyone is interested 
but wants to chat about what it's like, feel free to hit me up on the Slack 
channel.


 - ferruzzi



From: karan alang 
Sent: Thursday, April 3, 2025 4:38 PM
To: dev@airflow.apache.org; al...@majorleaguehacking.com
Subject: RE: [EXT] Airflow Mentors for Summer 2025 MLH Fellowship

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.



AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne 
cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas 
confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le 
contenu ne présente aucun risque.



Hi Alex,

I'm happy to help out with this.

regds,
Karan Alang


On Thu, Apr 3, 2025 at 4:13 PM Alex Gornet
 wrote:

> Hi Airflow team,
>
> I'm Alex Gornet, a Partner Success Manager at Major League Hacking. Here at
> MLH we run an Open Source Fellowship program
>  that functions like an internship for
> student developers to contribute to open source projects. *Fellows from our
> program have successfully contributed to Airflow since Fall 2024. *
>
> This year Royal Bank of Canada is sponsoring some seats in the program, and
> they'd like their Fellows to contribute to Airflow as a way to give back to
> the community. *With the recent release of Airflow 3.0, these Fellows would
> be great candidates for tackling the backlog of bugs / issues following the
> transition. *
>
> *Right now, we're looking for maintainers / core contributors to provide
> technical mentorship to the students*, so we'd love your help to identify 2
> maintainers / core contributors (or more!) who could take lead here and
> support us bringing on some great new contributors.
>
>- *The program runs from May 19th to August 8th*, and we'd ask for ~2
>hours of your time each week.
>
> *If this is something you'd be interested in helping out with, you can
> simply reply to this email or reach out to me directly at
> al...@majorleaguehacking.com * — since we're
> not too far out from program launch, the sooner you can let us know the
> better!
>
> If you have any questions about the program or want to chat more about it,
> let me know and I'm happy to find time. You can also reach out to Dennis
> Ferruzzi who has mentored the last couple of cohorts to hear more about his
> mentoring experience.
>
> Best,
> Alex
>

Re: [DISCUSS] confusing alert re SimpleAuthManager

[Amogh Desai]

Hmmm, I wonder if it can instead be made clearer. Something like this?

*Simple Auth Manager Enabled.*
*The Simple Auth Manager is intended for development and testing. If you're
using it in production, ensure that access is controlled through other
means. *
**

Thanks & Regards,
Amogh Desai


On Thu, Mar 20, 2025 at 11:58 PM Daniel Standish
 wrote:

> I'm saying, sounds confusing!
>
> On Thu, Mar 20, 2025 at 11:27 AM  wrote:
>
> > Sounds great! Do we have something in the config linter to highlight this
> > change?
> >
> > > On Mar 20, 2025, at 11:19 PM, Daniel Standish
> >  wrote:
> > >
> > > It says this:
> > >
> > > Development-only auth manager configured
> > > The auth manager configured in your environment is the Simple Auth
> > Manager,
> > > which is intended for development use only. It is not suitable for
> > > production and should not be used in a production environment.
> > >
> > >> On Thu, Mar 20, 2025 at 10:48 AM Jarek Potiuk 
> wrote:
> > >>
> > >> What's the alert - at least for me it did not get through
> > >>
> > >> On Thu, Mar 20, 2025 at 6:33 PM Daniel Standish
> > >>  wrote:
> > >>
> > >>> I should add, the import here is, many users who never customized
> auth
> > >>> before will now see this message and not really have a clue what they
> > are
> > >>> supposed to do, and I think it will probably create a good amount of
> > >>> confusion.
> > >>>
> > >>> On Thu, Mar 20, 2025 at 10:27 AM Daniel Standish <
> > >>> daniel.stand...@astronomer.io> wrote:
> > >>>
> >  I just saw this when spinning up airflow
> > 
> >  [image: image.png]
> > 
> >  I think the message is confusing / misleading / not very helpful.
> > 
> >  There's nothing necessarily wrong with having simple auth or no auth
> > if
> >  you control access some other way.  Moreover we don't tell users
> what
> > >> they
> >  should do instead!
> > 
> >  So I think we should either remove this bubble or add more nuance
> and
> >  point them in a direction that will lead them to what we *do*
> > recommend.
> > 
> > 
> > >>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> > For additional commands, e-mail: dev-h...@airflow.apache.org
> >
> >
>