Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hi Martijn, Thank you for sharing your thoughts on the matter. I understand that you don't have a strong opinion on whether to support exactly-once processing from the beginning or at a later stage. For initial implementation I will go ahead with at-least-once semantics. > The only consideration

Re: [DISCUSS] FLIP-324: Introduce Runtime Filter for Flink Batch Jobs

Hi Ron, Thanks for sharing your thoughts! It makes sense. It would be helpful if these references of Hive, Polardb, etc. could be added into the FLIP. Best regards, Jing On Tue, Jun 20, 2023 at 5:41 PM liu ron wrote: > Hi, Jing > > The default value for this ratio is a reference to other syste

[jira] [Created] (FLINK-32402) FLIP-294: Support Customized Catalog Modification Listener

Fang Yong created FLINK-32402: - Summary: FLIP-294: Support Customized Catalog Modification Listener Key: FLINK-32402 URL: https://issues.apache.org/jira/browse/FLINK-32402 Project: Flink Issue Ty

[jira] [Created] (FLINK-32403) Add database related operations in catalog manager

Fang Yong created FLINK-32403: - Summary: Add database related operations in catalog manager Key: FLINK-32403 URL: https://issues.apache.org/jira/browse/FLINK-32403 Project: Flink Issue Type: Sub-

[jira] [Created] (FLINK-32404) Introduce catalog modification listener and factory interfaces

Fang Yong created FLINK-32404: - Summary: Introduce catalog modification listener and factory interfaces Key: FLINK-32404 URL: https://issues.apache.org/jira/browse/FLINK-32404 Project: Flink Iss

[jira] [Created] (FLINK-32405) Initialize catalog listener for CatalogManager

Fang Yong created FLINK-32405: - Summary: Initialize catalog listener for CatalogManager Key: FLINK-32405 URL: https://issues.apache.org/jira/browse/FLINK-32405 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32406) Notify catalog listener for database events

Fang Yong created FLINK-32406: - Summary: Notify catalog listener for database events Key: FLINK-32406 URL: https://issues.apache.org/jira/browse/FLINK-32406 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32407) Notify catalog listener for table events

Fang Yong created FLINK-32407: - Summary: Notify catalog listener for table events Key: FLINK-32407 URL: https://issues.apache.org/jira/browse/FLINK-32407 Project: Flink Issue Type: Sub-task

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

+1 (binding) On Mon, Jun 19, 2023 at 8:09 AM Ryan van Huuksloot wrote: > +1 (non-binding) > > +1 for DynamicKafkaSource > > Ryan van Huuksloot > Sr. Production Engineer | Streaming Platform > [image: Shopify] > > > > On M

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Thanks Samrat for driving this FLIP. Since the community has already built a set of basic components for the connector, I only have three comments. 1 Reusing the capabilities of JDBC and Filesystem in the Redshift connector generally makes sense to me. However, since all of them are managed in

[DISCUSS] Releasing Flink ML 2.3.0

Hi devs, Dong and I would like to start a discussion regarding the release of Flink ML 2.3.0. The release mainly provides the ability to run on multiple Flink versions. With 2.3.0, users can build and run Flink ML libraries with Flink 1.15, 1.16, and 1.17. Besides, some algorithm performence

[RESULT][VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

Hi all, I am happy to announce that FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID[1] has been accepted. There are 8 approving votes, 6 of which are binding: - Lijie Wang (binding) - Jing Ge (binding) - Tzu-Li (Gordon) Tai (binding) - Zhu Zhu (binding) - Yuepeng P

[jira] [Created] (FLINK-32408) JobManager HA configuration update needed in Flink k8s Operator

dongwoo.kim created FLINK-32408: --- Summary: JobManager HA configuration update needed in Flink k8s Operator Key: FLINK-32408 URL: https://issues.apache.org/jira/browse/FLINK-32408 Project: Flink

[jira] [Created] (FLINK-32409) Remove MultipleComponentLeaderElectionDriverAdapter

Matthias Pohl created FLINK-32409: - Summary: Remove MultipleComponentLeaderElectionDriverAdapter Key: FLINK-32409 URL: https://issues.apache.org/jira/browse/FLINK-32409 Project: Flink Issue T

[jira] [Created] (FLINK-32410) Allocate hash-based collections with sufficient capacity for expected size

Stefan Richter created FLINK-32410: -- Summary: Allocate hash-based collections with sufficient capacity for expected size Key: FLINK-32410 URL: https://issues.apache.org/jira/browse/FLINK-32410 Projec

Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

Hi Aitozi, Thanks for your updates! By the design of hints, the hints after select clause belong to the query hints category, and this new hint is also a kind of join hints[1]. Join table function is one of the join type defined by flink sql joins[2], all existing join hints[1] omit the 'join' ke

Re: [DISCUSS] Deprecate SourceFunction APIs

I'd like to revive the efforts to deprecate the SourceFunction API. It would be great to get a review for this PR: https://github.com/apache/flink/pull/21774 It immediately unblocks marking the actual SourceFunction as deprecated. https://github.com/apache/flink/pull/20049 There is also this wor

[jira] [Created] (FLINK-32411) SourceCoordinator thread leaks when job recovers from checkpoint

Rui Fan created FLINK-32411: --- Summary: SourceCoordinator thread leaks when job recovers from checkpoint Key: FLINK-32411 URL: https://issues.apache.org/jira/browse/FLINK-32411 Project: Flink Issue

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

Hi Piotr, Thanks again for proposing the isProcessingBacklog concept. After discussing with Becket Qin and thinking about this more, I agree it is a better idea to add a top-level concept to all source operators to address the target use-case. The main reason that changed my mind is that isProce