+1 (non-binding). I think this would be a great addition as well! On Thu, May 29, 2025 at 9:28 AM Tom Cooper <c...@tomcooper.dev> wrote:
> +1 (non-binding) from me! This seems like it would be a really useful > functionality to have within Flink. > > Can we get some official notice (email on this thread, Github > discussion/issue) from the GetInData folks and other maintainers that they > are happy to donate the project to Flink and would be willing to support it > once it moves? > > Also does this require a new FLIP or can we resurrect the original > FLIP-233? > > Tom Cooper > @tomcooper.dev | https://tomcooper.dev > > > On Friday, 23 May 2025 at 12:13, David Radley <david_rad...@uk.ibm.com> > wrote: > > > Hi, > > Flip-233 [2] has been closed due to lack of capacity. I and others have > been working on enhancing the Flink getIndata HTTP connector[1]. This > connector is widely used. In the last year this connector has been > substantially improved. I want to discuss with the community whether it has > enough capabilities to donate to Flink. > > > > The current HTTP Connector: > > > > * works with SQL and datastream APIs > > * manages a thread pool to help with scaling under load > > * provides lookup join and sink capabilities > > * ability to customize > > * 90% junit coverage > > > > More recent enhancements include: > > > > * a retry mechanism ( that was asked for in Flips previous discussions) > [3] > > * a new query creator, that allows json rest calls to map query param, > body, path request information from table columns; without needing to write > your own java, [4] > > * standard Flink caching using the same mechanism as JDBC [5] > > * supports OIDC [6] > > > > > > > > The above capabilities are useful for many scenarios. > > One omission is that it is not currently a scan source. Though this is > being discussed [7]. > > > > Is there an appetite for the connector to be donated pretty much as-is > (with changes to make it fit with the existing Flink conventions – package > name, licenses etc). > > > > I am happy to drive and do this work, if the community is supportive of > it. > > > > Kind regards, David > > > > > > [1] > > https://github.com/getindata/flink-http-connector > > [2] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-233%3A+Introduce+HTTP+Connector > > [3] > > https://github.com/getindata/flink-http-connector/pull/148 > > [4] > > https://github.com/getindata/flink-http-connector/pull/149 > > [5] > > https://github.com/getindata/flink-http-connector/pull/94 > > [6] > > > https://github.com/getindata/flink-http-connector?tab=readme-ov-file#oidc-bearer-authentication > > [7] > > https://github.com/getindata/flink-http-connector/issues/41 > > > > Unless otherwise stated above: > > > > IBM United Kingdom Limited > > Registered in England and Wales with number 741598 > > Registered office: Building C, IBM Hursley Office, Hursley Park Road, > Winchester, Hampshire SO21 2JN >
