Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-10 Thread Jarek Potiuk
Yeah. when we get the final PR I will also want to to test more scenarios - with IDE/mypy integration switching branches, uv syncing etc. and will be happy to help and document the contributor's doc to explain what and how to work with it. This would be a super cool thing if we get it to work seaml

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-10 Thread Ash Berlin-Taylor
Not quite final PR, but good enough that I want to see how it behaves on CI, and other IDEs etc https://github.com/apache/airflow/pull/53149 (We updated the `setup_idea.py`, so either re-run that or add the new src root manually) Let the naming discussions start! -ash > On 10 Jul 2025, at 12:5

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-10 Thread Kevin Yang
Hi, I joined airflow community around two months ago and I would also like to share some thoughts, probably more from a user perspective. I agree that LLM can streamline DAG creation, especially for users/teams that are new to Airflow. However, in production system, static DAG files may be more

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Vincent Beck
+1 on TP's proposal, it reads well and stands out more than the `.assert_called_...`. I'll try to use it in the future On 2025/07/10 14:47:30 Tzu-ping Chung wrote: > Does pytest-mock have an equivalent for call()? I agree for mocking in > general we should consider replacing plain decorators and

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Kyungjun Lee
Thank you all — I really appreciate how many people have joined the discussion. I also like the approach that Tzu-ping Chung suggested. This really isn’t an easy topic. At first, I thought it would be best to use only plain assert statements, but after reading through the different perspectives he

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Wei Lee
I like what TP proposed. Even the original pytest style fits my taste better. The unittest style bugs me when I first contributed to Airflow. It’s already way better than it used to be. But, I would love to see it move more towards a pytest style. Best, Wei > On Jul 10, 2025, at 2:30 PM, Tzu-p

RE: Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Dev-iL
One tiny comment regarding TP's suggestion - IMHO it's better to avoid `unittest.mock` in favor of the equivalent `mocker` fixture provided by `pytest-mock`. On 2025/07/10 06:30:22 Tzu-ping Chung wrote: > Personally I dislike things like assert_called_once_with etc. since they are easy to miss

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-10 Thread Ash Berlin-Taylor
Oh one thing to note: In order to get both mypy and IntelliJ working wit this, we needed to commit the typestubs, and in order to make Python not get confused by the src/airflow/_vendor/airflow_shared directory existing and treating it as a namespace package, so the loader installer code now lo

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc5 & Task SDK 1.0.3 from 1.0.3rc5

2025-07-10 Thread Amogh Desai
+1 binding. airflow-core and task-sdk: - Checked reproducible package builds - Performed SVN checks - Checked Licenses - Checked Signatures - Checked SHA512 checksums Installed the RC bits with breeze and performed targeted testing for all my changes as reported in https://github.com/apache/airfl

Re: Seeking Apache Airflow Expert for Data Sourcing Discussion

2025-07-10 Thread Jarek Potiuk
If you want to offer a job - I think you can ask on Slack in the #jobs channel (you will find the link to slack in the "community" page of our website). Generally speaking if you want to ask a concrete question - with a problem to solve - you can ask there and if someone decides to spend their tim

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Tzu-ping Chung
Does pytest-mock have an equivalent for call()? I agree for mocking in general we should consider replacing plain decorators and context managers with the mocker fixture. This probably deserves its own discussion thread. -- Sent from my iPhone > On 10 Jul 2025, at 14:37, Dev-iL wrote: > > O

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Jarek Potiuk
Agreed - the proposal from TP looks much better. I did not know you could do that. And it feels more consistent. On Thu, Jul 10, 2025 at 2:22 PM Wei Lee wrote: > I like what TP proposed. Even the original pytest style fits my taste > better. The unittest style bugs me when I first contributed to

Re: [DISCUSS] Consistent test assertion style: pytest-native vs unittest-style

2025-07-10 Thread Amogh Desai
+1 to TP's proposal too. It's easy to read and also stands out better. We have a few places in the task-sdk tests where we also have done patterns like: assert not any( x == mock.call( msg=GetXCom( key="key", dag_id="test_dag", run_id="test_run

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-10 Thread Amogh Desai
Cool! Thanks for sharing your thoughts here Kevin, user perspective is always important. Thanks & Regards, Amogh Desai On Thu, Jul 10, 2025 at 6:59 PM Kevin Yang wrote: > Hi, I joined airflow community around two months ago and I would also like > to share some thoughts, probably more from a u

Re: S3 Dag Bundle Versions and DB Manager

2025-07-10 Thread Oliveira, Niko
Thanks for the reply Jarek :) Indeed we have different philosophies about this so we will certainly keep going in circles about where to draw the line on making things easy and enjoyable to use, whether to intentionally add friction or not, etc, etc. I think if we have optional paths to take an

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc5 & Task SDK 1.0.3 from 1.0.3rc5

2025-07-10 Thread Jens Scheffler
Airflow-core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task-SDK: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Briefly tested Edge Executor in the release and ran the integration test Dag, looks all good. On 10.07.25 22:10,

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc5 & Task SDK 1.0.3 from 1.0.3rc5

2025-07-10 Thread Jarek Potiuk
+1 (binding). Checked * reproducibility (All good now!!!) * SVN * licences * signatures * checksums Run it in breeze, did some random checking on UI/ run some dags. all looks good! Looks like the 5th time's a charm ! J. On Thu, Jul 10, 2025 at 7:37 PM Amogh Desai wrote: > +1 binding. > > ai