Thanks Mayank for the proposal. I think it's a great addition to Flink to define secure connectivity in general for table, model and other resources later on. +1.
Hao On Fri, May 2, 2025 at 5:32 AM Gustavo de Morais <gustavopg...@gmail.com> wrote: > Hi Mayank, > > Thanks for the initiative. Looking at the FLIP, this looks like a > well-thought-out proposal that addresses a clear need for more secure and > reusable external connections in Flink SQL and Table API. Separating > connection details would a valuable improvement. > > Best Regards, > Gustavo > > Am Fr., 2. Mai 2025 um 07:12 Uhr schrieb Ferenc Csaky > <ferenc.cs...@pm.me.invalid>: > > > Hi Mayank, > > > > Thank you for starting the discussion! In general, I think such > > functionality > > would be a really great addition to Flink. > > > > Could you pls. elaborate a bit more one what is the reason of defining a > > `connection` resource on the database level instead of the catalog level? > > If I think about `JdbcCatalog`, or `HiveCatalog`, the catalog is in > 1-to-1 > > mapping with an RDBMS, or a HiveMetastore, so my initial thinking is > that a > > `connection` seems more like a catalog level resource. > > > > WDYT? > > > > Thanks, > > Ferenc > > > > > > > > On Tuesday, April 29th, 2025 at 17:08, Mayank Juneja < > > mayankjunej...@gmail.com> wrote: > > > > > > > > > > > Hi all, > > > > > > I would like to open up for discussion a new FLIP-529 [1]. > > > > > > Motivation: > > > Currently, Flink SQL handles external connectivity by defining > endpoints > > > and credentials in table configuration. This approach prevents > > reusability > > > of these connections and makes table definition less secure by exposing > > > sensitive information. > > > We propose the introduction of a new "connection" resource in Flink. > This > > > will be a pluggable resource configured with a remote endpoint and > > > associated access key. Once defined, connections can be reused across > > table > > > definitions, and eventually for model definition (as discussed in > > FLIP-437) > > > for inference, enabling seamless and secure integration with external > > > systems. > > > The connection resource will provide a new, optional way to manage > > external > > > connectivity in Flink. Existing methods for table definitions will > remain > > > unchanged. > > > > > > [1] https://cwiki.apache.org/confluence/x/cYroF > > > > > > Best Regards, > > > Mayank Juneja > > >
