Re: Flink CDC to Paimon

2025-03-11 Thread Taher Koitawala
Hi All, I am facing issues finding a class Org.apache.flink.cdc.connectors.shaded.org.apache.kafka.connect.json.JsonConverter I have added 1. Flink-connector-base 1.18.1 Flink-connector-debezium 3.1.0 Flink-cdc-pipeline-connectors-values 3.1.0 Flink-cdc-base 3.1.0 Flink-cdc-pipeline-conn

Re: Flink CDC to Paimon

2025-03-07 Thread Taher Koitawala
Hi Leonard, Yes i did see Xianqian’s reply however i thought my email did not go through as the community is often very active but I did not receive a response until Xianqian’s reply. Thank you Xianqian we are currently trying out the suggestions for altering schema via the catalog. Thank yo

Re: Flink CDC to Paimon

2025-03-04 Thread Leonard Xu
Hey Taher, Xianqian has replied your email, did you subscribe Flink dev mailing-list? Best, Leonard > 2025年2月11日 16:52,Xiqian YU 写道: > > Hi Taher, > > Since we’re creating a DataStream-based pipeline job with SQL Server CDC, > schema change events must be handled manually. A possible approac

Re: Flink CDC to Paimon

2025-03-04 Thread Taher Koitawala
Hi Devs, Any response here? On Tue, 11 Feb 2025 at 11:59 AM, Taher Koitawala wrote: > Hi Devs, >As a POC we are trying to create a steaming pipeline from MSSQL cdc > to Paimon: > > To do this we are doing > 1. msSql server cdc operator > 2. Transform operator > 3. paimon si

Re: Flink CDC to Paimon

2025-02-11 Thread Xiqian YU
Hi Taher, Since we’re creating a DataStream-based pipeline job with SQL Server CDC, schema change events must be handled manually. A possible approach would be: 1) Enable schema change events with `.includeSchemaChanges(true)` option, so DDL events will be parsed and encoded in `SourceRecord`s.

Flink CDC to Paimon

2025-02-11 Thread Taher Koitawala
Hi Devs, As a POC we are trying to create a steaming pipeline from MSSQL cdc to Paimon: To do this we are doing 1. msSql server cdc operator 2. Transform operator 3. paimon sink We have written the cdc connector with is a JsonDebeziumDeserialisedSchema String I wish to write this paimon i

[jira] [Created] (FLINK-36790) Flink CDC to Paimon: DeletionVectorsMaintainer CME problem

2024-11-25 Thread Rascal Wu (Jira)
Rascal Wu created FLINK-36790: - Summary: Flink CDC to Paimon: DeletionVectorsMaintainer CME problem Key: FLINK-36790 URL: https://issues.apache.org/jira/browse/FLINK-36790 Project: Flink Issue