Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-07 Thread lorenzo . affetti
Thanks joao for your replies! I also saw the latest PR that allows properties to be specified. Thank for adding the pain points as well, that clarifies a lot. On May 7, 2024 at 09:50 +0200, Muhammet Orazov , wrote: > Thanks João for pointing it out. I didn't know about the PR, I am going > to ch

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-07 Thread Muhammet Orazov
Thanks João for pointing it out. I didn't know about the PR, I am going to check it. Best, Muhammet On 2024-05-06 14:45, João Boto wrote: Hi Muhammet, Have you had a chance to review the recently merged pull request [1]? We've introduced a new feature allowing users to include ad hoc config

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-06 Thread João Boto
Hi Jeyhun, > > I would also ask to include a sample usage and changes for end-users in the > > FLIP. We want to ensure a seamless transition for end-users, minimizing any disruptions in their current usage of the connector. To achieve this, we will uphold consistency by maintaining the same int

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-06 Thread João Boto
Hi Muhammet, Have you had a chance to review the recently merged pull request [1]? We've introduced a new feature allowing users to include ad hoc configurations in the 'JdbcConnectionOptions' class. ``` new JdbcConnectionOptions.JdbcConnectionOptionsBuilder() .withUrl(F

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-06 Thread Muhammet Orazov
Morning João, Recently we had a case where the JDBC drivers authentication was different than username&password authentication. For it to work, certain hacks required, there interface would have been helpful. But I agree maybe the interface module separation is not required at the moment.

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-03 Thread Jeyhun Karimov
Hi Boto, Thanks for driving this FLIP. +1 for it. I would also ask to include a sample usage and changes for end-users in the FLIP. flink-connector-jdbc: The current module, which will be transformed to > shade all other modules and maintain backward compatibility. Also, in order to ensure the

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-03 Thread João Boto
Hi, > > You can now update the derby implementation and the core independently and > > decide at your own will when to include the new derby in the core? Not really, we are talking about creating modules in the same repository, not about externalizing the database modules. That is, whenever ther

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-03 Thread João Boto
Hi Muhammet, While I generally agree, given our current usage, I'm struggling to discern any clear advantage. We already have abstract implementations that cover all necessary interfaces and offer essential functionality, complemented by a robust set of reusable tests to streamline implementati

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-26 Thread lorenzo . affetti
Hello Joao, thank your for your proposal, modularity is always welcome :) > To maintain clarity and minimize conflicts, we're currently leaning towards > maintaining the existing structure, where flink-connector-jdbc-${version}.jar > remains shaded for simplicity, encompassing the core functiona

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-25 Thread Muhammet Orazov
Hey João, Thanks for FLIP proposal! Since proposal is to introduce modules, would it make sense to have another module for APIs (flink-jdbc-connector-api)? For this I would suggest to move all public interfaces (e.g, JdbcRowConverter, JdbcConnectionProvider). And even convert some classes into

Re: Re:[DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-25 Thread Yuepeng Pan
Hi, Boto. It's already clear enough to me. Thanks for your reply. Best, Yuepeng Pan On 2024/04/25 15:41:01 João Boto wrote: > Hi Pan, > > Users who wish to utilize only one database and prefer not to use > flink-connector-jdbc-${version}.jar + ${database-connector-driver}.jar should > opt for

Re: Re:[DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-25 Thread João Boto
Hi Pan, Users who wish to utilize only one database and prefer not to use flink-connector-jdbc-${version}.jar + ${database-connector-driver}.jar should opt for option 1: flink-connector-jdbc-core-${version}.jar + flink-connector-jdbc-mysql-${version}.jar + ${database-connector-driver}.jar. We

Re:[DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-25 Thread Yuepeng Pan
Hi, Boto. Thanks for your driving it ! +1 from me on the proposal. Maybe we need to ensure that a simple usage method is provided to users after the refactoring. In the current situation, which supported database does the user intend to use, Users only need to add the flink-connector-jdbc-$

[DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

2024-04-25 Thread Joao Boto
Hi all, I'd like to start a discussion on FLIP-449: Reorganization of flink-connector-jdbc [1]. As Flink continues to evolve, we've noticed an increasing level of complexity within the JDBC connector. The proposed solution is to address this complexity by separating the core functionality from ind