Re: Airflow using NFS(PVC) - jobs running in namespace different from airflow install namespace

Agreed with Jarek, but briefly your problem suggests that you want to mount a PVC onto running workers but the PVC is in a different namespace. This is generally not possible in K8s, some blogs like this

Re: [DISCUSS] Publishing articles in Airflow Medium publication and AI generated content

A general filtering for "clear" AI generated content would be really nice. Validating content for correctness isn't an easy task (doing this will need in depth knowledge of Airflow which is an evolving project). Thanks & Regards, Amogh Desai On Thu, Nov 28, 2024 at 2:26 AM Ferruzzi, Dennis wr

Re: [VOTE] November 2024 PR of the Month

+1 for https://github.com/apache/airflow/pull/43902 Best, Wei > On Nov 28, 2024, at 7:28 AM, Pavankumar Gopidesu > wrote: > > +1 for 41916, Great work Illumaria. > > Regards, > Pavan Kumar > > On Wed, Nov 27, 2024 at 4:36 PM kalyan reddy wrote: >> >> +1 for https://github.com/apache/airflo

Re: [VOTE] November 2024 PR of the Month

+1 for 41916, Great work Illumaria. Regards, Pavan Kumar On Wed, Nov 27, 2024 at 4:36 PM kalyan reddy wrote: > > +1 for https://github.com/apache/airflow/pull/43902 > > From: Kaxil Naik > Sent: 27 November 2024 20:14 > To: dev@airflow.apache.org > Subject: Re:

Re: Airflow using NFS(PVC) - jobs running in namespace different from airflow install namespace

While we appreciate you reaching out - [Lease open a discussion in Github Discussions. https://github.com/apache/airflow/discussions - this is where you can ask for troubleshooting help. Also you can ask your questions in Airflow Slack (#user-troubleshooting channel). Devlist is used mostly for dev

Airflow using NFS(PVC) - jobs running in namespace different from airflow install namespace

I've Airflow installed in namespace - airflow, and i'm using NFS to store the dags & pyspark code. I want to run the airflow jobs in different namespace - airflow-spark-apps .. however, i'm unable to do this since the PVC associated with the ns - airflow-spark-apps is not accessible to airflow depl

Re: [DISCUSS] Publishing articles in Airflow Medium publication and AI generated content

I agree with all points here. - ferruzzi From: Maciej Obuchowski Sent: Wednesday, November 27, 2024 5:45 AM To: dev@airflow.apache.org Subject: RE: [EXT] [DISCUSS] Publishing articles in Airflow Medium publication and AI generated content CAUTION: This email

Re: [VOTE] November 2024 PR of the Month

+1 for https://github.com/apache/airflow/pull/43902 From: Kaxil Naik Sent: 27 November 2024 20:14 To: dev@airflow.apache.org Subject: Re: [VOTE] November 2024 PR of the Month +1 for 41916 given that it was their first time contribution and I would like to encoura

Re: Adding a significant newsfragment template to ease breaking change tracking

Very nice indeed :) On 2024/11/27 14:46:28 Kaxil Naik wrote: > Nice, a good step for categorization for breaking changes. > > On Wed, 27 Nov 2024 at 13:38, Amogh Desai wrote: > > > Nice! > > > > It was really hard to do back and forth with the precommit while adding a > > significant > > one. T

Re: Adding a significant newsfragment template to ease breaking change tracking

Nice, a good step for categorization for breaking changes. On Wed, 27 Nov 2024 at 13:38, Amogh Desai wrote: > Nice! > > It was really hard to do back and forth with the precommit while adding a > significant > one. This solves that! > > Thanks & Regards, > Amogh Desai > > > On Wed, Nov 27, 2024

Re: [VOTE] November 2024 PR of the Month

+1 for 41916 given that it was their first time contribution and I would like to encourage them to continue contributing. On Wed, 27 Nov 2024 at 13:52, Phani Kumar wrote: > +1 for https://github.com/apache/airflow/pull/43902 > > I'd like to nominate PR #43502 as it was a monumental effort *(329

Re: [VOTE] November 2024 PR of the Month

+1 for https://github.com/apache/airflow/pull/43902 I'd like to nominate PR #43502 as it was a monumental effort *(329 files)*, both in terms of its scope and the challenges it addressed. Resolving the CI conflicts alone required significant persistence and multiple days of effort. I believe it de

Re: [DISCUSS] Publishing articles in Airflow Medium publication and AI generated content

Blogs that duplicate content of the documentation, or provide surface level best practices can be a net negative even if they are providing information that's currently valid, since that can change later. In OpenLineage we've seen people coming to Slack or GitHub with problems due to following old,

Re: Adding a significant newsfragment template to ease breaking change tracking

Nice! It was really hard to do back and forth with the precommit while adding a significant one. This solves that! Thanks & Regards, Amogh Desai On Wed, Nov 27, 2024 at 4:31 PM Jarek Potiuk wrote: > Cool :) > > On Wed, Nov 27, 2024 at 10:55 AM Wei Lee wrote: > > > Hi all, > > > > We just mer

Re: [VOTE] November 2024 PR of the Month

+1 on #41916 as well. On Wed, Nov 27, 2024 at 2:16 PM Amogh Desai wrote: > +1 for https://github.com/apache/airflow/pull/41916 too. > > It is nice how a "first" time contributor has collaborated with the > maintainers > and managed to solve many issues and merge it. > > Kudos to https://github.

[ANNOUNCE] Apache Airflow Providers prepared on November 24, 2024 are released

Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages prepared on November 24, 2024 were just released. Full list of PyPI packages released is added at the end of the message. The source release, as well as the binary releases, are available here: https://

Re: [VOTE] November 2024 PR of the Month

+1 for https://github.com/apache/airflow/pull/41916 too. It is nice how a "first" time contributor has collaborated with the maintainers and managed to solve many issues and merge it. Kudos to https://github.com/Illumaria irrespective of the result! Thanks & Regards, Amogh Desai On Wed, Nov 27

[RESULT][VOTE] Airflow Providers - release of November 24, 2024

Hello, Apache Airflow Providers prepared on November 24, 2024 have been accepted. 3 "+1" binding votes received: - Elad Kalif - Jarek Potiuk - Ephraim Anierobi 2 "+1" non binding votes received: - Amogh Desai - Utkarsh Sharma Vote thread: https://lists.apache.org/thread/v67owkgdtnflbw1g4bkkoyk

Re: [VOTE] Airflow Providers prepared on November 24, 2024

+1 binding - checked reproducibility, checksums, signatures and licences On Wed, 27 Nov 2024 at 07:50, utkarsh sharma wrote: > +1 non-binding. > > Thanks, > Utkarsh Sharma > > On Mon, Nov 25, 2024 at 3:28 PM Jarek Potiuk wrote: > > > +1 (binding) -> all looks good: reproducibility, checksums,

[DISCUSS] Publishing articles in Airflow Medium publication and AI generated content

Hello here, TL;DR; We have this very nice Airflow Publication https://medium.com/apache-airflow - and recently we had some submissions that were clearly Gen AI generated and provided no value and I think we need to agree on some general acceptance criteria. We had quite a few submissions submitte

Re: [LAZY CONSENSUS] Use Ruff as the official linter to detect Airflow 3 incompatibilities in user code

Very cool :) On Wed, Nov 27, 2024 at 10:46 AM Wei Lee wrote: > Hi folks, > > If anyone plans to work on any of the rules in this list > https://github.com/apache/airflow/issues/41641, please create an issue and > leave a comment in that issue 41641 so that we can avoid duplicate efforts. > I wil

Re: Adding a significant newsfragment template to ease breaking change tracking

Cool :) On Wed, Nov 27, 2024 at 10:55 AM Wei Lee wrote: > Hi all, > > We just merged a significant newsfragment template to the main branch. > https://github.com/apache/airflow/pull/44378 It would make updating #41641 > and > > https://cwiki.apach

Adding a significant newsfragment template to ease breaking change tracking

Hi all, We just merged a significant newsfragment template to the main branch. https://github.com/apache/airflow/pull/44378 It would make updating #41641 and https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+breaking+changes much easier

Re: [LAZY CONSENSUS] Use Ruff as the official linter to detect Airflow 3 incompatibilities in user code

Hi folks, If anyone plans to work on any of the rules in this list https://github.com/apache/airflow/issues/41641, please create an issue and leave a comment in that issue 41641 so that we can avoid duplicate efforts. I will update the issue description whenever I notice that someone has commented