Hi all,
I'm attempting to create a POC in flink to create a pipeline to stream parquet
to a data warehouse in iceberg format.
Ideally - I'd like to watch a directory in s3 (minio locally) and stream those
to iceberg, doing the appropriate schema mapping/translation.
I guess first; does this so
Dan
From: Feng Jin
Sent: Thursday, November 23, 2023 6:49:11 PM
To: Oxlade, Dan
Cc: user@flink.apache.org
Subject: [EXTERNAL] Re: flink s3[parquet] -> s3[iceberg]
Hi Oxlade
I think using Flink SQL can conveniently fulfill your requirements.
For S3 Parquet files, you can create a t
Hi,
If I use StreamExecutionEnvironment.createRemoteEnvironment and then
var tEnv = StreamTableEnvironment.create(env) from the resulting remote
StreamExecutionEvironment will any sql executed using tEnv.executeSql be
executed remotely inside the flink cluster?
I'm seeing unexpected behavior wh
`b` DOUBLE,
`c` DOUBLE,
`d` STRING
);
""".formatted(catalogName, dbName));
tEnv.executeSql("""
INSERT INTO `%s`.`%s`.`out`
