-1 — for the simple case the URI format is perfect: AIRFLOW_CONN_MY_DB=postgresql://user:pass@host
We can push JSON format for more complex cases, and/or limit URI to only the simple cases as long as we don’t remove it entirely. -ash > On 17 Nov 2023, at 22:44, Andrey Anshin <andrey.ans...@taragol.is> wrote: > > Greetings! > > I want to propose to deprecate Airflow Connection URI representation and > remove it in Airflow 3 in favor of the already existing replacement - JSON > representation. > > In the past URI representation helped to add one of the awesome features - > Alternative Secrets Backends: Environment Variables, Files or Provider > specific Backends. > > However I have a feeling that nowadays it have some sort of limitations due > to the parser, see: > - https://github.com/apache/airflow/issues/33442 > > Or miss interpretation that Airflow connection it is something more than > just representation of Airflow Connection and it is a the replacement to > any other type of URIs such as: HTTP, Postgres URIs, SQLAlchemy URI, Spark > URI and etc > > And attempt to fix it > - https://github.com/apache/airflow/issues/10913 > - https://github.com/apache/airflow/pull/27796 > - https://github.com/apache/airflow/pull/35712 > > And attempt to use it in DbApiHooks, for detail see: > - https://lists.apache.org/thread/8rhmz3qh30hvkondct4sfmgk4vd07mn5 > > In addition, complicated Connection in the URI format is not human > readable, requires additional decoding, and it is hard to create it > manually. > > The only one beneficial part of URI representation is ability to create it > programatically > https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#generating-a-connection-uri > , there is no such of method for generate JSON, but I don't see the problem > to create such of method in Connection object which return string > representation of JSON connection. > > So my suggestion would be pretty simple: > 1. Deprecate accept URI as parameter to the Connection object > 2. Deprecate uri property > 3. Replace get_uri by get_json > 4. Replace all URI examples in Providers Connections by JSON alternatives > > > WDYT? Maybe I miss something and we should keep Airflow Connection as URI > in the future Airflow's major versions and support both ways to provide > connections as JSON and URI. > > ---- > Best Wishes > *Andrey Anshin*
