Re: [EXTERNAL] Re: flink s3[parquet] -> s3[iceberg]

Hi Dan I think using Flink SQL should be able to meet your needs. You can write a Flink Jar program. Accept different directories, schemas, mappings, and sink tables to generate DDL and DML. Assuming you have two directories: directory1 -> f1, f2, f3, f4 -> iceberg1 directory2 -> f1, f2, f3 ->

Re: dependency error with latest Kafka connector

Hi all, I've cherry-picked FLINK-30400 onto v3.0 branch of flink-connector-kafka. Treating this thread as justification to start a vote for 3.0.2 RC #1 immediately so we can get out a new release ASAP. Please see the vote thread here [1]. @guenterh.lists Would you be able to test this RC and se

Doubts about state and table API

Hi, We are having a job in production where we use table API to join multiple topics. The query looks like this: SELECT * FROM topic1 AS t1 JOIN topic2 AS t2 ON t1.userId = t2.userId JOIN topic3 AS t3 ON t1.userId = t3.accountUserId This works and produces an EnrichedActivity any time any of t

Elasticsearch connector support?

Hi From the documentation I can see there isn’t any ES support in Flink 1.18 right now and Flink-26088 (ES 8 support) is still open. Does anyone has an idea when ES connector support will be available in 1.18 Please let me know. Med venlig hilsen / Best regards Lasse Nedergaard

Re: dependency error with latest Kafka connector

> built a fat uber jar from quickstart with Flink 1.18.0 for > flink-streaming-java and flink-clients, and flink-connector-kafka version > 3.0.1-1.18 > then submitted to local Flink cluster 1.18.0. Things worked as expected and > the job ran fine. Hey，@Gordan I guess things may work as expecte

Re: Confluent Kafka conection error

Hi Hang, Few more points regarding this issue 1. The issue does not replicate in my local installation and occurs only on kubernetes server installation. 2. We have used kubernetes operator to install flink on the server Please let me know if any other info is required here. Thanks, Tauseef On

Re: Confluent Kafka conection error

Hi Hang, I cross checked this issue multiple times. I also upgraded to flink 1.18 but the issue persists. Can you please let me know a few guidelines on how to investigate this and fix it positively. Thanks, Tauseef On Thu, 23 Nov 2023 at 18:08, Tauseef Janvekar wrote: > Thanks Hang. > > I go

Re: dependency error with latest Kafka connector

Hi Gordon, I'm wondering if this might be a difference between how Maven and Gradle build their projects, since you've done your validations with Maven, but Günter uses Gradle. In the end, the quickest fix would be to backport FLINK-30400 to the Flink Kafka 3.0 release branch. Best regards, Mart