Yeah - for sure we need some details, the direction seems sound (and is indeed pretty standard - https://auth0.com/docs/secure/tokens/token-best-practices for example describes a number of ways API security is handled with JWT. Possibly - again - we should look at some ready to use solution in fast_api and have it as the standard blueprint to start AIP (or depending on how "standard" answers to those questions / best practices are ) just a vote on a more detailed proposal.
I would be quite surprised if we had to reinvent the wheel and produce something "ours", I am quite certain we can use something that is already proven and established there. That would also help us likely to get something that has multiple implementations for a number of enterprise-ready protocols. Maybe eventually also another thing to use as FAB replacement also for UI. The general rule of thumb is that if you think you can invent something new in security and you are not a security expert, you are likely doing it wrong. So I would rather look at what we can "use" rather than what new we can come up with. J. On Mon, Mar 3, 2025 at 4:17 PM Ash Berlin-Taylor <a...@apache.org> wrote: > Hi Vincent, > > Can you elaborate more on Option 1? > > Can you give a concrete example of the request flow between browser/CLI, > the API server and any backends? > > How often is this API to generate a JWT called? > > What is the request flow of user credentials? > > What validation/verification is don eon the returned JWT? Does it have any > required claims or structure? > > -ash > > > On 28 Feb 2025, at 17:50, Beck, Vincent <vincb...@amazon.com.INVALID> > wrote: > > > > Hi everyone, > > > > I would like to talk about auth backends. > > > > In Airflow 2, there are multiple options for authenticating REST API > calls. These options are called auth backends ( > https://airflow.apache.org/docs/apache-airflow-providers-fab/stable/auth-manager/api-authentication.html). > The deployment manager configures which authentication mechanism is used > for REST API calls. There are several options: > > - session > > - basic_auth > > - Kerberos > > - Google OpenID > > > > For example, if the deployment manager sets `auth_backends = > airflow.providers.fab.auth_manager.api.auth.backend.basic_auth`, then users > must authenticate their Rest API calls using basic authentication > (username/password). Note: Multiple auth backends can be configured (e.g. > `auth_backends = > airflow.providers.fab.auth_manager.api.auth.backend.basic_auth,airflow.providers.fab.auth_manager.api.auth.backend.session`). > > > > Most (if not all) of these auth backends are provided by providers. They > follow the same interface and must define two functions: > > - `init_app`. To initialize resources if needed > > - `requires_authentication`. Checks whether the authentication provided > in the request is valid. For example, `basic_auth` extracts authentication > information from the Flask request and checks whether the username/password > provided are valid. If the authentication succeeds then the user is saved > in session so that it can be used subsequently by the API itself. > > > > In Airflow 3, all APIs (but here we will focus only on the public API) > use JWT tokens for authentication. Every API request must include a valid > JWT token to be authenticated. This is not an issue when using the UI since > the UI manages JWT tokens automatically. However, what happens when users > call the public API directly? > > > > I see three possible options: > > > > Option 1. Deprecate auth backends and introduce an API to generate JWT > tokens. This approach aligns with how modern web applications handle > authentication. It would simplify authentication in Airflow by enforcing a > single strategy: JWT-based authentication. User flow: > > - Call an API to generate a JWT token, providing authentication details > such as a username and password > > - If authentication succeeds, the API returns a JWT token > > - Use this JWT token to authenticate public API calls > > > > The API for generating JWT tokens would be provided by the auth manager. > The Simple auth manager already supports this, and Bugra is working on > adding it to the FAB auth manager (PR: > https://github.com/apache/airflow/pull/47043). > > > > This is personally my preferred solution but there are some caveats: > > - Users will need to update their authentication methods. However, since > Airflow 3 already introduces breaking changes, users will need to adjust > their integrations regardless. > > - Some authentication strategy would no longer be possible such as > Google OpenID. Since authentication shifts from the auth backend to the > auth manager, providers without an auth manager (like Google) would lose > their auth backends. More explanations on this: because the API to create a > JWT token is provided by the auth manager, a provider could support all the > different authentication mechanisms provided by its auth backends through > this API. Example: The FAB provider currently supports basic_auth and > Kerberos as auth backends. In Airflow 3, the FAB auth manager could support > both authentication mechanisms for generating JWT tokens. > > > > Option 2. Update auth backends to be compatible with Airflow 3. To > support Airflow 3, we would need to: > > - Modify auth backends to use Fastapi instead of Flask > > - Update the `requires_authentication` function, since it currently > validates authentication and stores the user in a session—an approach > incompatible with Airflow 3. > > - Ensure compatibility with both Airflow 2 and Airflow 3, meaning we > would have two implementations for each auth backend (one for AF2, one for > AF3). > > > > Pros: > > - Backward compatibility: Users can continue using existing > authentication methods across Airflow 2 and Airflow 3. > > > > Cons: > > - Increased maintenance complexity due to dual implementations. > > > > Option 3. Support both Option 1 and Option 2. This would give users the > flexibility to choose their preferred authentication method. > > > > What do you think? > > > > Vincent > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
