Re: Integrating new connector with Flink SQL.

2021-06-03 Thread Jingsong Li
Hi santhosh, 1.Yes. 2.I'm very glad if you can contribute. Best, Jingsong On Fri, Jun 4, 2021 at 1:13 AM santhosh venkat wrote: > Hi, Jingsong, > > Thanks for taking time to respond to my questions. Really appreciate it. > > 1. Just to ensure we are on the same page, are you recommending us t

Re: Integrating new connector with Flink SQL.

2021-06-03 Thread santhosh venkat
Hi, Jingsong, Thanks for taking time to respond to my questions. Really appreciate it. 1. Just to ensure we are on the same page, are you recommending us to implement Source, SourceReader and SplitEnumberator abstractions for the new source connector. And use either the DataStreamScanProvider or

Re: Integrating new connector with Flink SQL.

2021-06-02 Thread Jingsong Li
Hi santhosh, 1.I recommend you use the new source with ScanTablesource. 2.You can use `org.apache.flink.table.connector.source.SourceProvider` to integrate to ScanTablesource. (Introduced in 1.12) 3.You can just implement a new source, this one can be used by both Flink DataStream and Flink SQL.

Integrating new connector with Flink SQL.

2021-06-02 Thread santhosh venkat
Hi, Please correct me if I'm wrong anywhere. I'm just new to Flink and trying to navigate the landscape. Within my company, currently we're trying to develop a connector for our internal change data capture system(brooklin) for flink. We are planning to use Flink SQL as a primary API to build str