Hi, On source we could improve the JdbcParameterValuesProvider.. to be defined as a query(s) or something more dynamic. The most time if your job is dynamic or have some condition to be met (based on data on table) you have to create a connection an get that info from database
If we are going to create/allow a "streaming" jdbc source, we should be able to define watermark and get new data from table using that watermark.. For the sink (but it could apply on source) will be great to be able to set your implementation of the connection type.. For example if you are connecting to clickhouse, be able to set a implementation based on "BalancedClickhouseDataSource" for example (in this[1] implementation we have a example) or set a extension version of a implementation for debug purpose Regards [1] https://github.com/apache/flink/pull/20097/files#diff-8b36e3403381dc14c748aeb5de0b4ceb7d7daec39594b1eacff1694b5266419d On 2022/06/27 13:09:51 Roc Marshal wrote: > Hi, all, > > > > > I would like to open a discussion on porting JDBC Source to new Source API > (FLIP-27[1]). > > Martijn Visser, Jing Ge and I had a preliminary discussion on the JIRA > FLINK-25420[2] and planed to start the discussion about the source part first. > > > > Please let me know: > > - The issues about old Jdbc source you encountered; > - The new feature or design you want; > - More suggestions from other dimensions... > > > > You could find more details in FLIP-239[3]. > > Looking forward to your feedback. > > > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface > > [2] https://issues.apache.org/jira/browse/FLINK-25420 > > [3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217386271 > > > > > Best regards, > > Roc Marshal
