Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-02 Thread Vikram Koka
t; > >>> ruff check --preview --select AIR3 > >>> is the command right now if you want to test, but I'd wait until > Airflow > >> 3 > >>> release so it is all final. There will also be upgrading guides. :) > >>> > >>> >

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-02 Thread Wei Lee
M Ferruzzi, Dennis >> >>> wrote: >>> >>>> IMHO, if they were deprecated that long ago then adding a ruff rule is >>>> just enabling the users to ignore the deprecation. They should >>>> /eventually/ have to clear some of their tech debt and actually update >>>>

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-02 Thread Kaxil Naik
, no? They've had four and half YEARS of > >> warning at this point. But to the original topic, an AF2-AF3 ruff rule > >> was "always" the plan AFAIK. I've lost track of who is working on what > at > >> this point, but I know I heard talk about

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-02 Thread Eugen Kosteev
entually/ have to clear some of their tech debt and actually update > >> their code to modern standards, no? They've had four and half YEARS of > >> warning at this point. But to the original topic, an AF2-AF3 ruff rule > >> was "always" the plan AFAIK. I've

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Wei Lee
gt;> was "always" the plan AFAIK. I've lost track of who is working on what at >> this point, but I know I heard talk about that almost since the beginning, >> no? That seems like as good of a place as any to handle this redirect. >> >> >> - ferruzzi >> &g

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Tamara Fingerlin
. > > > - ferruzzi > > > > From: Jarek Potiuk > Sent: Tuesday, April 1, 2025 9:32 AM > To: dev@airflow.apache.org > Subject: RE: [EXT] [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3 > > CAUTION: This email originated from out

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Ferruzzi, Dennis
irect. - ferruzzi From: Jarek Potiuk Sent: Tuesday, April 1, 2025 9:32 AM To: dev@airflow.apache.org Subject: RE: [EXT] [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3 CAUTION: This email originated from outside of the organization. Do not click links or ope

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Jarek Potiuk
Ah.. I missed that we have those _operators ... Those indeed should have been fixed a long time ago and automated conversion rules from ruff should be fixing them. On Tue, Apr 1, 2025 at 6:17 PM Kaxil Naik wrote: > Those were deprecated 4.5+ years ago: > > https://github.com/apache/airflow/blob/

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Kaxil Naik
Those were deprecated 4.5+ years ago: https://github.com/apache/airflow/blob/2.0.0/airflow/operators/python_operator.py On Tue, 1 Apr 2025 at 21:45, Kaxil Naik wrote: > Instead handle it via ruff rules AIR2 something > > On Tue, 1 Apr 2025 at 21:44, Kaxil Naik wrote: > >> ` - ModuleNotFoundErr

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Kaxil Naik
Instead handle it via ruff rules AIR2 something On Tue, 1 Apr 2025 at 21:44, Kaxil Naik wrote: > ` - ModuleNotFoundError: No module named > 'airflow.operators.python_operator'` <-- those paths are Airflow 1.x old > > We had already stripped `_operator` from the module names in Airflow 2.0.0 >

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Kaxil Naik
` - ModuleNotFoundError: No module named 'airflow.operators.python_operator'` <-- those paths are Airflow 1.x old We had already stripped `_operator` from the module names in Airflow 2.0.0 -- so IMO there is no need to keep back-compatibility for something that was working 2 major versions ago

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Jarek Potiuk
An example where deprection_tools are still used https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/utils/log/__init__.py It's rather straightforward = needs a package with __init__.py - only where you list all the classes and provide redirections. It will automatically raise dep

Re: [DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Jarek Potiuk
We were going to have compatibility shims to redirect the imports - with - there are few ways to do it - Ash had a little POC with module loader, but I think it has some potential side effect and I think Ash abandoned the idea and I would personally prefer to use our old PEP-563 mechanism using air

[DISCUSSION] Airflow 2 DAGs compatible with Airflow 3

2025-04-01 Thread Eugen Kosteev
Hi everyone. I am testing compatibility of Airflow 2 DAGs with Airflow 3, and would like to discuss this topic. I took bunch of DAGs from existing Airflow 2 instances and deployed them to instance with Airflow 3 (3.0.0b4) and have bunch of import errors: - ModuleNotFoundError: No module named '