Dear Airflow Community, I am thrilled to announce the availability of *Apache Airflow 3.0.0.alpha3* for testing! Airflow 3.0 marks a significant milestone as the first major release in over four years, introducing improvements that enhance user experience, task execution, and system scalability. First, a few caveats:
- This is an *alpha release*, so do not run it in production. It may contain significant issues, and you will likely need to reset your database between this and subsequent alpha, beta, or release candidate versions. *(Consider yourself warned!)* - This release is intended for *Airflow developers and early adopters* to test the build and start preparing for Airflow 3.0.0. This is *not an official release*βthat will happen when we create a release candidate and hold a vote. - The *expected timeline* for the first release candidate is *the week of 2025-03-31*, but we encourage early feedback to help stabilize the release. What's New in 3.0.0.alpha3? Apache Airflow 3.0.0 introduces significant enhancements and breaking changes. *Notable Features* *DAG Versioning & Bundles* - Airflow now tracks *DAG versions*, offering better visibility into historical DAG changes and execution states. - *DAG Bundles* ensure tasks run with the correct code version, even as DAGs evolve. *Modern Web Application* - The UI has been *rebuilt using React* and a complete *API-driven structure*, improving maintainability and extensibility. - A new *React-based plugin system* supports custom widgets, improved workflow visibility, and integration with external tools. *Task Execution Interface* - Airflow 3.0 adopts a *client/server architecture*, decoupling task execution from the internal meta-database via API-based interaction. - This allows for *remote execution*, *multi-language support*, enhanced security, and better dependency management. - The *Edge Executor* enables seamless remote task execution without direct database connections. *Data Assets & Asset-Centric Syntax* - Airflow 3.0 enhances dataset management by introducing *Data Assets*, expanding beyond tables and files to include ML models and more. - Assets can be explicitly defined using the @asset decorator, simplifying tracking and dependencies. *External Event-Driven Scheduling* - Airflow now supports *event-driven DAG triggers* from external sources like message queues. - This builds upon dataset scheduling and enhances integration with the external data ecosystem. For a more comprehensive list of new features, see the 3.0.0.alpha3 release notes <https://github.com/apache/airflow/blob/3.0.0a2/RELEASE_NOTES.rst#airflow-300alpha2-2025-01-31> . For a list of the breaking changes, visit: Apache Airflow Breaking Changes <https://cwiki.apache.org/confluence/x/9pCMEw> *Known Issues in 3.0.0.alpha3* *AIP-72 - Task Execution Interface* - *DB isolation in tasks* is not enforced yet. - *KubernetesExecutor* does not yet use the Task Execution Interface. - The following do not work on *LocalExecutor* or *CeleryExecutor*: - Skip-based tasks (e.g. branch or skip operators). - Task callbacks. *AIP-38 - Modern Web Application* - The new UI has *limited functionality* at this time and is still being enhanced until GA. - *FastAPI API server* (including the new UI and public API) does not yet have authentication. - UI performance may degrade with a *large number of DAGs* or *very big DAGs*. - Notable areas of functionality *not yet implemented* in the UI: - DAG Versioning, Data Assets, Backfills, DAG Gantt View. *AIP-82 - Event-Based Scheduling* - Some triggers (e.g., *file-based triggers*) may lead to *duplicate DAG runs*. *Where to Get It?* The *alpha snapshot* is available at: Apache Airflow 3.0.0.alpha3 Distribution <https://dist.apache.org/repos/dist/dev/airflow/3.0.0a3/> - *Binary Python "sdist" snapshot:* apache-airflow-3.0.0a3-bin.tar.gz - *Binary Python wheel snapshot:* apache_airflow-3.0.0a3-py3-none-any.whl This snapshot has *not been published to PyPI*. Also present are *alpha releases* for: - apache-airflow-task-sdk - apache-airflow-providers-standard - apache-airflow-providers-fab - apache-airflow-providers-celery These *are also not published to PyPI*. *Public Keys & Verification* Public keys for verification are available at: Apache Airflow KEYS <https://www.apache.org/dist/airflow/KEYS> *Installation Instructions* To build a virtualenv that installs this alpha release and required packages: uv venv uv pip install --find-links https://dist.apache.org/repos/dist/dev/airflow/3.0.0a3/ \ apache-airflow==3.0.0a3 \ apache-airflow-task-sdk==1.0.0a3 \ apache-airflow-providers-standard==0.1.0a3 \ apache-airflow-providers-fab==2.0.0a3 \ apache-airflow-providers-celery==3.11.0a3 *Changes Youβll Need to Consider* - The *standalone DAG processor* is now required, start it with: airflow dag-processor - The new UI and public API start with: airflow fastapi-api UI is available at http://localhost:9091/webapp. - Depending on your deployment setup, you may need to set the [workers] execution_api_server_url config option. Default: http://localhost:9091/execution/ *Get Involved* We encourage the community to *test this release* and report any issues or feedback. Your contributions help us ensure a *stable and reliable Airflow 3.0.0 release*. A huge *thank you* to all the contributors who have worked on this milestone release! *Best,* Utkarsh and Jed
