Hi everyone, I'm a new contributor to Airflow. I'd like to propose a new feature for Airflow: DAG Version Pinning. Building on the foundation introduced by AIP-63: DAG Versioning (https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-63%3A+DAG+Versioning), this proposal aims to extend Airflow's capabilities to support true continuous deployment (CD) gating and safer release cycles. The Problem & Use Cases Currently, the scheduler always creates DagRuns using the latest parsed DagVersion. This means that the updated DAG code is deployed (takes effect) right after the dag-processor processes it. While this is great for rapid development, teams running business-critical pipelines often need stricter deployment mechanisms. Specifically:
* Safe Deployment Gating: The ability to pin a DAG to its last known stable version while new code is parsed in the background. This allows the new version to be held back until it passes automated regression tests or receives explicit manual approval. * Instant Rollbacks: If an issue is detected in a newly promoted DAG version, users need the capability to instantly roll back to a previous version via the UI/API, without having to revert the underlying code and wait for the repository sync and DAG processing cycle. High-Level Proposed Solution Introduce an optional active_dag_version_id to the DagModel. This field can be used to pin a DAG version for scheduling and execution, while the dag-processor can continue to parse and register newer DAG versions. * When this pin is set, the scheduler and API will respect the pinned version for creating runs and executing tasks, separating the parsing of new code from the execution of new code. * If the pin is NULL, the system defaults to the current behavior (always executing the latest parsed version). This way, we can maintain complete backwards compatibility. I have put together some detailed notes covering the data model changes, database migrations, and edge cases with this approach. If there is general alignment that this fits the vision for Airflow, I would like to take this proposal through the formal AIP review process. But I would love to get the community's feedback on the feature and the high-level approach. I'll also need someone to grant me access to create content on the Airflow Confluence wiki. Thanks for your time! Regards, Piyush
