Re: Flink CDC to Paimon

2025-03-11 Thread Taher Koitawala
Hi All, I am facing issues finding a class Org.apache.flink.cdc.connectors.shaded.org.apache.kafka.connect.json.JsonConverter I have added 1. Flink-connector-base 1.18.1 Flink-connector-debezium 3.1.0 Flink-cdc-pipeline-connectors-values 3.1.0 Flink-cdc-base 3.1.0 Flink-cdc-pipeline-conn

Re: Flink CDC to Paimon

2025-03-07 Thread Taher Koitawala
Hi Leonard, Yes i did see Xianqian’s reply however i thought my email did not go through as the community is often very active but I did not receive a response until Xianqian’s reply. Thank you Xianqian we are currently trying out the suggestions for altering schema via the catalog. Thank yo

Re: Flink CDC to Paimon

2025-03-04 Thread Leonard Xu
Hey Taher, Xianqian has replied your email, did you subscribe Flink dev mailing-list? Best, Leonard > 2025年2月11日 16:52,Xiqian YU 写道: > > Hi Taher, > > Since we’re creating a DataStream-based pipeline job with SQL Server CDC, > schema change events must be handled manually. A possible approac

Re: Flink CDC to Paimon

2025-03-04 Thread Taher Koitawala
Hi Devs, Any response here? On Tue, 11 Feb 2025 at 11:59 AM, Taher Koitawala wrote: > Hi Devs, >As a POC we are trying to create a steaming pipeline from MSSQL cdc > to Paimon: > > To do this we are doing > 1. msSql server cdc operator > 2. Transform operator > 3. paimon si

Re: Flink CDC to Paimon

2025-02-11 Thread Xiqian YU
Hi Taher, Since we’re creating a DataStream-based pipeline job with SQL Server CDC, schema change events must be handled manually. A possible approach would be: 1) Enable schema change events with `.includeSchemaChanges(true)` option, so DDL events will be parsed and encoded in `SourceRecord`s.