Hi, There has been support from community to proceed on this. Krzysztof Zarzycki, CTO of Getindata (soon to be Xebia) has confirmed on the dev list that they support this donation. I will update the Flip and start an official Flip discussion thread.
We will need 3 binding votes, please could the committers review this proposal and feedback. We in IBM use this connector for lookup joins, so think it is robust enough for production. I suggest we lift and shift the existing connector as-is – amending package names etc and then use this as a base. We then assess new functions like gRPC in follow on Flips; gRPC sounds like a valuable addition. Kind regards ,David. From: Tom Cooper <c...@tomcooper.dev> Date: Thursday, 29 May 2025 at 17:27 To: dev@flink.apache.org <dev@flink.apache.org> Subject: [EXTERNAL] Re: Proposal Flip-233: Donate GetInData HTTP Connector to Flink +1 (non-binding) from me! This seems like it would be a really useful functionality to have within Flink. Can we get some official notice (email on this thread, Github discussion/issue) from the GetInData folks and other maintainers that they are happy to donate the project to Flink and would be willing to support it once it moves? Also does this require a new FLIP or can we resurrect the original FLIP-233? Tom Cooper @tomcooper.dev | https://tomcooper.dev On Friday, 23 May 2025 at 12:13, David Radley <david_rad...@uk.ibm.com> wrote: > Hi, > Flip-233 [2] has been closed due to lack of capacity. I and others have been > working on enhancing the Flink getIndata HTTP connector[1]. This connector is > widely used. In the last year this connector has been substantially improved. > I want to discuss with the community whether it has enough capabilities to > donate to Flink. > > The current HTTP Connector: > > * works with SQL and datastream APIs > * manages a thread pool to help with scaling under load > * provides lookup join and sink capabilities > * ability to customize > * 90% junit coverage > > More recent enhancements include: > > * a retry mechanism ( that was asked for in Flips previous discussions) [3] > * a new query creator, that allows json rest calls to map query param, body, > path request information from table columns; without needing to write your > own java, [4] > * standard Flink caching using the same mechanism as JDBC [5] > * supports OIDC [6] > > > > The above capabilities are useful for many scenarios. > One omission is that it is not currently a scan source. Though this is being > discussed [7]. > > Is there an appetite for the connector to be donated pretty much as-is (with > changes to make it fit with the existing Flink conventions – package name, > licenses etc). > > I am happy to drive and do this work, if the community is supportive of it. > > Kind regards, David > > > [1] > https://github.com/getindata/flink-http-connector > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-233%3A+Introduce+HTTP+Connector > [3] > https://github.com/getindata/flink-http-connector/pull/148 > [4] > https://github.com/getindata/flink-http-connector/pull/149 > [5] > https://github.com/getindata/flink-http-connector/pull/94 > [6] > https://github.com/getindata/flink-http-connector?tab=readme-ov-file#oidc-bearer-authentication > [7] > https://github.com/getindata/flink-http-connector/issues/41 > > Unless otherwise stated above: > > IBM United Kingdom Limited > Registered in England and Wales with number 741598 > Registered office: Building C, IBM Hursley Office, Hursley Park Road, > Winchester, Hampshire SO21 2JN Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: Building C, IBM Hursley Office, Hursley Park Road, Winchester, Hampshire SO21 2JN
