>
> I still think different people have different expectations of the Airflow
> URI format. My expectation is that this is only a SerDe mechanism around
> Airflow Connection. If we have a look at the codebase of Airflow we found
> that this is not always true and we try to use it as SA URI.


Yes, this is in my view exactly the problem.  I believe we run into trouble
only when we falsely think that we should be able to define an Airflow
connection using, say, a jdbc connection string. It's not true in general.

Observe that DbApiHook has a get_uri
<https://github.com/apache/airflow/blob/064fc2b7751a44e37ccce97609cff7c496098e56/airflow/providers/common/sql/hooks/sql.py#L188-L196>
method.

This method is used to produce a connection string valid for the service
you're connecting to, and importantly it may be different from the
Connection.get_uri.

The primary responsibility of Airflow connection URI is to serialize the
airflow connection object which has type, login, pass, host, schema port,
extra and that's it.  For simple cases it can serve both purposes but it's
not true in general and folks need to understand that.

I think part of the problem may simply be that `uri` is an argument of the
Connection object. It means not the connection string but the airflow conn
URI serialization.  Would probably be a good candidate for deprecation
on the basis that it is confusing.

Fundamentally, as a hook designer, if you need a conn string that is a URI,
you must implement get_uri on the hook in the right way to be able to
use the attributes available to you in Connection.  Connection.get_uri will
handle creating the Airflow Conn URI serialization and that's a separate
thing.

The connection object has no clue how to generate a valid connection string
for your service, but the hook can have this understanding.

There are many examples in the codebase where users implement get_uri on
the hook and it works just fine.  A hook author could write it so you can
optionally dump the full connection string in the extra object: {"extra":
{"uri": "blah"}}.

Reply via email to