The GitHub Actions job "Tests" on airflow.git/feat/edge-cli-concurrency has failed. Run started by GitHub user dheerajturaga (triggered by dheerajturaga).
Head commit for run: 52760b21453d370f3c4176781c1ac4ce965b2c5d / Dheeraj Turaga <[email protected]> Add runtime concurrency control for remote Edge Workers via CLI In distributed Airflow deployments using EdgeExecutor, edge workers run at remote sites that are often unreachable from the central control plane — behind firewalls, in air-gapped networks, or on edge computing nodes. Prior to this change, the worker's concurrency (the number of tasks it runs in parallel) could only be set at startup via the -c CLI flag. Adjusting it required direct shell access to the remote machine and a full worker restart, causing job execution downtime and defeating the operational value of the edge architecture. This contribution solves the problem entirely by introducing a server-driven concurrency control mechanism that works within the existing security and connectivity constraints of edge deployments. Rather than requiring a new communication channel, it repurposes the worker's existing heartbeat protocol — the only bidirectional link between the central Airflow API server and the remote worker — to deliver concurrency updates. An administrator runs: airflow edge set-worker-concurrency --edge-hostname <worker> --concurrency <n> This writes the desired concurrency to the central database. On the worker's next heartbeat (typically within seconds), the API server returns the new value in its response, and the worker adopts it immediately — no restart, no remote access, no downtime. The design follows the established pattern for queue management introduced in this provider, but extends it to a runtime-mutable execution parameter for the first time. It required coordinated changes across five architectural layers: the database schema (new migration), the SQLAlchemy model, the FastAPI worker API response contract, the async worker heartbeat loop, and the CLI command surface. The solution preserves the core architectural guarantee of the edge executor — that workers never hold a direct database connection — while making a previously static configuration parameter dynamically controllable from the central site This capability was made possible by DB schema versioning introduced in #61155 Report URL: https://github.com/apache/airflow/actions/runs/22692221221 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
