Re: [DISCUSS] Sink V2 API missing features

Hi Martijn, Thanks for the prompt answer, good to hear that there is interest in the community to improve things around the APIs! *First and foremost, we still plan to use the PostCommitTopology* to do the compaction for the Iceberg Sink. That is on our roadmap, but the first step is to do the Si

Support AWS SDK V2 for Flink's S3 FileSystem

Hi Flink Dev, I’m Maomao, a developer from AWS EMR. Recently, our team is working on adding AWS SDK V2 support for Flink’s S3 Filesystem. During development, we found out that our work was blocked by Presto. This is because that Presto still uses AWS SDK V1 and won’t add support for AWS SDK V2

Re: [DISCUSS] FLIP 333 - Redesign Apache Flink website

There's now a PR (https://github.com/apache/flink-web/pull/676) and a preview available for this FLIP ( https://website-refresh.d193kg429zpv7e.amplifyapp.com/). On Fri, Jul 28, 2023 at 8:45 AM Mohan, Deepthi wrote: > Matthias, Markos, Martijn, thank you for your feedback. > > Markos, I've addres

Re: [ANNOUNCE] Release 1.18.0, release candidate #1

Hi all, Thanks to the team for this RC. I did a quick check of this RC against user pipelines (1) coded with DataSet (even if deprecated and soon removed), DataStream and SQL APIs based on the small scope of this test, LGTM +1 (non-binding) [1] https://github.com/echauchot/tpcds-benchmark-f

[jira] [Created] (FLINK-33169) Window TVFs don't consider column expansion

Timo Walther created FLINK-33169: Summary: Window TVFs don't consider column expansion Key: FLINK-33169 URL: https://issues.apache.org/jira/browse/FLINK-33169 Project: Flink Issue Type: Sub-t

Re: [ANNOUNCE] Release 1.18.0, release candidate #1

Thanks for creating RC1. I did the following checks: * Downloaded artifacts * Built Flink from sources * Verified SHA512 checksums GPG signatures * Compared checkout with provided sources * Verified pom file versions * Went over NOTICE file/pom files changes without finding anything suspicious * D

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

Hi Jark, A minor suggestion. Would it be ok if we changed the config name to ` table.optimizer.query-options.enabled`? This is inline with other existing configurations such as: table.dynamic-table-options.enabled table.optimizer.distinct-agg.split.enabled table.optimizer.dynamic-filtering.enable

Re: [ANNOUNCE] Release 1.18.0, release candidate #1

Thanks for the efforts! +1 (non-binding) * Verified versions in the poms * Built from source * Verified checksums and signatures * Started basic workloads with kubernetes operator * Verified NOTICE and LICENSE files G On Fri, Sep 29, 2023, 18:16 Matthias Pohl wrote: > Thanks for creating RC1.

Re: [DISCUSS] FLIP-360: Merging ExecutionGraphInfoStore and JobResultStore into a single component

Thanks for sharing your thoughts, Shammon FY. I kind of see your point. Initially, I didn't put much thought into splitting up the interface into two because the dispatcher would have been the only component dealing with the two interfaces. Having two interfaces wouldn't have added much value (in

[jira] [Created] (FLINK-33170) HybridSourceSplitEnumerator causes dropped records in Hybrid Sources with 3+ sources

Robert Hoyt created FLINK-33170: --- Summary: HybridSourceSplitEnumerator causes dropped records in Hybrid Sources with 3+ sources Key: FLINK-33170 URL: https://issues.apache.org/jira/browse/FLINK-33170 Pr