This has been merged, thank you all. On Tue, 22 Sept 2026 at 15:48, Vikram Koka via dev <[email protected]> wrote:
> +1 > > This makes sense to me and I am looking forward to seeing this become a > reality. > > Vikram > > > On Mon, Sep 21, 2026 at 9:02 PM Amogh Desai <[email protected]> wrote: > > > Makes sense, thanks for the clarification. > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Mon, Sep 21, 2026 at 4:28 PM Kaxil Naik <[email protected]> wrote: > > > > > Amogh: > > > > > > 1. It becomes an optional dep once Modal Sandbox uses the provider. > > > 2. Similar to Anthropic & Pydantic IMO. The hooks are pretty > light-weight > > > by design ensuring we can get best of the Airflow integration without > > > having to test entirety of Modal infra. That said I do use Modal so > > happily > > > test and can integrate any dags as needed. > > > > > > On Mon, 21 Sept 2026 at 11:33, Jarek Potiuk <[email protected]> wrote: > > > > > > > +1. Looks great as a standalone provider > > > > > > > > On Mon, Sep 21, 2026, 11:55 Amogh Desai <[email protected]> > wrote: > > > > > > > > > Hi Kaxil, > > > > > > > > > > +1 on the standalone provider. > > > > > The infrastructure vs model vendor split makes sense to me. > > > > > > > > > > I have two questions regarding this: > > > > > > > > > > 1. When the common.ai sandbox backend starts taking modal_conn_id, > > > does > > > > > that make the Modal provider a > > > > > dependency of common.ai, or is it an optional import that only > kicks > > > in > > > > > when someone actually uses Modal? > > > > > I just want to ensure that common.ai stays light for people who > are > > > not > > > > > using Modal. > > > > > > > > > > 2. How do you plan to test this, particularly in CI? AFAICT, Modal > > is a > > > > > paid service. > > > > > > > > > > Thanks & Regards, > > > > > Amogh Desai > > > > > > > > > > On Mon, Sep 21, 2026 at 6:38 AM Kaxil Naik <[email protected]> > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'd like to propose a new community provider, > > > > > > apache-airflow-providers-modal, for Modal (https://modal.com/), > > the > > > > > > serverless compute platform for running functions, GPU jobs and > > > > sandboxed > > > > > > containers. Targeting Airflow 3+ and shipping as incubation, same > > as > > > > the > > > > > > Anthropic and OpenAI providers. > > > > > > > > > > > > Scope of the first version > > > > > > > > > > > > - A modal connection type (token id and secret, optional Modal > > > > > > environment), so Modal credentials live in an Airflow connection > > and > > > > > > secrets backend rather than in MODAL_TOKEN_ID / > MODAL_TOKEN_SECRET > > on > > > > > every > > > > > > worker. > > > > > > - ModalHook, which turns that connection into an authenticated > > > > > modal.Client > > > > > > and hands out handles to deployed functions and classes, secrets, > > > > > volumes, > > > > > > apps and sandboxes, each scoped to the connection's environment. > > > > > Execution > > > > > > stays on the Modal SDK objects (.remote(), .spawn(), > > Sandbox.wait()), > > > > so > > > > > > the hook does not re-wrap the SDK. > > > > > > > > > > > > No operators or executor in this first PR. The intent is to land > > the > > > > > > connection type and hook first, because several things want them: > > the > > > > > Modal > > > > > > backend for the common.ai sandbox toolset (#72910) currently has > > to > > > > read > > > > > > ambient credentials, and a ModalExecutor that runs each task in > its > > > own > > > > > > Modal sandbox has a working prototype that I'd bring as a > follow-up > > > PR > > > > > once > > > > > > the provider exists to host it. > > > > > > > > > > > > Why a standalone provider rather than a piece of common.ai > > > > > > > > > > > > common.ai is provider-agnostic. Modal is infrastructure, not a > > model > > > > > > vendor: a connection type, an executor and compute-oriented > > operators > > > > > have > > > > > > no home in an AI-neutral interface, and tying non-AI Modal users > to > > > > > > common.ai would be the wrong dependency direction. The sandbox > > > backend > > > > > > stays in common.ai and will take an optional modal_conn_id > > resolved > > > > > > through > > > > > > this hook, which is the same "vendor provider alongside > common.ai" > > > > split > > > > > > we > > > > > > settled on for Anthropic and OpenAI. > > > > > > > > > > > > Working implementation > > > > > > > > > > > > https://github.com/apache/airflow/pull/73418 > > > > > > > > > > > > I'm happy to steward and sponsor the provider, and would welcome > a > > > > > > co-steward per the community provider policy. If there are no > > > > objections > > > > > > within seven days I'll proceed with merging. > > > > > > > > > > > > Thanks, > > > > > > Kaxil > > > > > > > > > > > > > > > > > > > > >
