Hello everyone.

I created the PR for that https://github.com/apache/airflow/pull/47281.

It's even nicer than I anticipated. I love the new super-simple workflows
this restructuring finally enabled.

With `uv` and workspace, and the new structure of tests, developing and
running  tests for providers, task-sdk or any of the other future
sub-projects becomes very, very straightforward, we avoid duplication of
pytest options and switching between airflow, task-sdk, providers tests
will be simple and straightforward.

1) First of all with this change, we remove `devel-tests` extra. It was
always needed in local venv to install all test dependencies, But this is
completely gone right now. Test dependencies are automatically installed
now when you run `uv sync`  - and you do not need to specify `--extra
devel-tests`. If you are still using `pip` (I strongly recommend switching
to uv) - you just install `pip install -e ./tests-common`.

2) the previous way of syncing and running tests in "everything installed"
mode works as it worked before:

uv sync --all-extras

This installs all possible extras of airflow, allows you to run tests for
all providers, activate the venv in `.venv` and run `pytests tests/always`
or `pytest providers/mongo/tests`  or `pytest task_sdk/tests and it should
all work fine as it did before

3) you can also install dependencies of a selected provider (or a few of
those) in your .venv

uv sync --extra mongo

3) All the `breeze testing provider-tests --test-type "Providers[mongo]""
and generally all the ways to replicate what we run in CI inside breeze
containers with selective checks will also continue to work - no changes
here.

But then, a little bit of standardization, a bit of pre-commits that
automatically update `pyproject.toml` files of ours with cross-provider
dependencies and  a little of `uv` workspace magic sprinkled over the repo
of ours and now we can do something much more straightforward:

*You can change directory to task_sdk, or provider folder of your choice
and simply run:*

*cd providers/mongo*
*uv run pytest*

or

*cd task_sdk*
*uv run pytest *

And it will do what you would expect to do - even if this is the first
thing you do after checking out Airflow's repo without any earlier syncing
or preparation. Yes. It's that simple.

The `uv run pytest` command will now automatically: install python, create
a venv, sync your venv to ONLY have dependencies needed for your provider
or any of the providers imported by your provider, and all test
dependencies needed, it will find only non-system, non-integration tests
and run them. That's it. If you want to only develop one provider - you
will not have to worry any more about breeze image or setting up all other
dependencies.

Go ahead and try it. It's unbelievably simple.

J.



On Tue, Feb 25, 2025 at 5:45 PM Ash Berlin-Taylor <a...@apache.org> wrote:

> I like this approach, lets do it!
>
> > On 25 Feb 2025, at 16:02, Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > airflow-core
> > task-sdk
> > tests-common
>
>

Reply via email to