[jira] [Created] (FLINK-35537) Error parsing list of enum in legacy yaml configuration

Zakelly Lan created FLINK-35537: --- Summary: Error parsing list of enum in legacy yaml configuration Key: FLINK-35537 URL: https://issues.apache.org/jira/browse/FLINK-35537 Project: Flink Issue

Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn

Thanks Zhu for the suggestion. I have updated the description of the option. Best, Yuxin Zhu Zhu 于2024年6月6日周四 14:59写道： > +1 > > Maybe explain in the description of > `taskmanager.network.hybrid-shuffle.external-remote-tier-factory.class` > that it only accepts Celeborn as the remote shuffle ti

[jira] [Created] (FLINK-35538) pipeline sink support caseSensitive

melin created FLINK-35538: - Summary: pipeline sink support caseSensitive Key: FLINK-35538 URL: https://issues.apache.org/jira/browse/FLINK-35538 Project: Flink Issue Type: Improvement Compo

[jira] [Created] (FLINK-35539) The artifactId of flink-migration-test-utils module has a typo

Zhen Wang created FLINK-35539: - Summary: The artifactId of flink-migration-test-utils module has a typo Key: FLINK-35539 URL: https://issues.apache.org/jira/browse/FLINK-35539 Project: Flink Is

[jira] [Created] (FLINK-35540) flink-cdc-pipeline-connector-mysql lost table which database and table with the same name

Qigeng Lin created FLINK-35540: -- Summary: flink-cdc-pipeline-connector-mysql lost table which database and table with the same name Key: FLINK-35540 URL: https://issues.apache.org/jira/browse/FLINK-35540

[jira] [Created] (FLINK-35541) Introduce retry limiting for AWS connector sinks

Aleksandr Pilipenko created FLINK-35541: --- Summary: Introduce retry limiting for AWS connector sinks Key: FLINK-35541 URL: https://issues.apache.org/jira/browse/FLINK-35541 Project: Flink

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #2

+1 (binding) - Release notes look good - Source archive checksum and signature is correct - Binary checksum and signature is correct - Contents of Maven repo looks good - Verified there are no binaries in the source archive - Builds from source, tests pass using Java 8 - CI run passed - Tag exists

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #2

This vote is now closed, I will announce the results in a new threaad On Thu, Jun 6, 2024 at 10:43 AM Danny Cranmer wrote: > +1 (binding) > > - Release notes look good > - Source archive checksum and signature is correct > - Binary checksum and signature is correct > - Contents of Maven repo loo

[RESULT][VOTE] flink-connector-mongodb 1.2.0, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Ahmed Hamdy * Hang Ruan * Jiabao Sun * Qingsheng Ren (binding) * Leonard Xu (binding) * Danny Cranmer (binding) There are no disapproving votes. Thanks everyone! Danny

[ANNOUNCE] Apache flink-connector-mongodb 1.2.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-mongodb 1.2.0 for Flink 1.18 and 1.19. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release i

[jira] [Created] (FLINK-35542) ClassNotFoundException when deserializing CheckpointedOffset

Jan Gurda created FLINK-35542: - Summary: ClassNotFoundException when deserializing CheckpointedOffset Key: FLINK-35542 URL: https://issues.apache.org/jira/browse/FLINK-35542 Project: Flink Issue

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Tue, Jun 4, 2024 at 11:08 AM Fabian Paul wrote: > +1 (non-binding) > > - Verified signature > - Verified checksum

Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn

Hi all, Thanks for all the feedback and suggestions so far. If there is no further comment, we will open the voting thread tomorrow. Best, Yuxin Yuxin Tan 于2024年6月6日周四 15:40写道： > Thanks Zhu for the suggestion. > I have updated the description of the option. > > Best, > Yuxin > > > Zhu Zhu 于

[jira] [Created] (FLINK-35543) Upgrade Hive 2.3 connector to version 2.3.10

Cheng Pan created FLINK-35543: - Summary: Upgrade Hive 2.3 connector to version 2.3.10 Key: FLINK-35543 URL: https://issues.apache.org/jira/browse/FLINK-35543 Project: Flink Issue Type: Improvemen

Re: Poor Load Balancing across TaskManagers for Multiple Kafka Sources

Thanks for the response Zhanghao. Since FLIP-370 won't help, any ideas on how this can be improved? Can we round-robin assign partitions from all KafkaTopics to TaskManagers as suggested in https://issues.apache.org/jira/browse/FLINK-31762? On Wed, Jun 5, 2024 at 10:52 PM Zhanghao Chen wrote: >

Re: [DISCUSS] FLIP-461: FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing

Thanks for the FLIP Matthias, I think it looks pretty solid! I also don't see a relation to unaligned checkpoints. From the AS perspective, the checkpoint time doesn't matter. Is it possible a change event observed right after a complete checkpoint > (or within a specific short time after a check

[VOTE] Release 1.19.1, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the flink v1.19.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn

Hi Yuxin, +1 for this proposal. This change will greatly alleviate the pressure on local storage resources (especially when there is limited local storage) particularly in the context of cloud-native environments. Regards, Jeyhun On Thu, Jun 6, 2024 at 1:20 PM Yuxin Tan wrote: > Hi all, > > Th

Question about v2 committer guarantees

Hi there, I'm working on developing a new version of the fully open-source Delta-Flink Sink using the Unified Flink Sink V2 APIs and the fully open-source Delta-Kernel library (link for more details about Delta-Kernel). I have a few questions about the API gu

[DISCUSS] FLIP-462: Support Custom Data Distribution for Input Stream of Lookup Join

Hi devs, I'd like to start a discussion about FLIP-462[1]: Support Custom Data Distribution for Input Stream of Lookup Join. Lookup Join is an important feature in Flink, It is typically used to enrich a table with data that is queried from an external system. If we interact with the external s

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #2

Thanks Danny for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Fri, May

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

Thanks Danny for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Wed, May

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

Thanks Danny for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Tue, Jun

Re: [VOTE] Release flink-connector-opensearch v2.0.0, release candidate #1

Thanks Sergey for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Mon, Ma

Re: [VOTE] Release flink-connector-opensearch v1.2.0, release candidate #1

Thanks Sergey for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Thu, Ma

[jira] [Created] (FLINK-35544) No description of Flink Operator in the Deployment of  Kubernetes document

linqigeng created FLINK-35544: - Summary: No description of Flink Operator in the Deployment of  Kubernetes document Key: FLINK-35544 URL: https://issues.apache.org/jira/browse/FLINK-35544 Project: Flink

Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

Congratulations, Rui! Best, Xia Paul Lam 于2024年6月6日周四 11:59写道： > Congrats, Rui! > > Best, > Paul Lam > > > 2024年6月6日 11:02，Junrui Lee 写道： > > > > Congratulations, Rui. > > > > Best, > > Junrui > > > > Hang Ruan 于2024年6月6日周四 10:35写道： > > > >> Congratulations, Rui! > >> > >> Best, > >> Hang > >

Re: [VOTE] Release flink-connector-gcp-pubsub v3.1.0, release candidate #1

Thanks Danny for the hard work! +1(binding) - Verified signatures - Verified sha512 (hashsums) - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Checked Github release tag - Reviewed the flink-web PR Best, Rui On Wed, May

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #2

Thanks Danny! +1(binding) - Verified signatures and hashsums - Build from source - Checked release tag - Reviewed the flink-web PR - Checked the CI build. Best regards, Weijie Rui Fan <1996fan...@gmail.com> 于2024年6月7日周五 11:00写道： > Thanks Danny for the hard work! > > +1(binding) > > - Verifie

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

Thanks Danny! +1(binding) - Verified signatures and hash sums - Checked the CI build - Checked the release note - Reviewed the flink-web PR - Build from source. Best regards, Weijie Rui Fan <1996fan...@gmail.com> 于2024年6月7日周五 11:08写道： > Thanks Danny for the hard work! > > +1(binding) > > - Ve

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

Thanks Danny! +1(binding) - Verified signatures and hash sum - Checked the CI build from tag - Build from source - Reviewed flink-web PR Best regards, Weijie Rui Fan <1996fan...@gmail.com> 于2024年6月7日周五 11:01写道： > Thanks Danny for the hard work! > > +1(binding) > > - Verified signatures > - V

[jira] [Created] (FLINK-35545) Miss 3.0.1 version in snapshot flink-cdc doc version list

Zhongqiang Gong created FLINK-35545: --- Summary: Miss 3.0.1 version in snapshot flink-cdc doc version list Key: FLINK-35545 URL: https://issues.apache.org/jira/browse/FLINK-35545 Project: Flink

Re: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

Thank you Gyula for the feedback. >From the above proposed conditions, so will be having two conditions as below status: conditions: - type: JobReady message: The Job is running reason: running status: 'True' lastTransitionTime: '' - type: ReconciliationSucceed message: The resource deployment i

Re: [DISCUSS] FLIP-462: Support Custom Data Distribution for Input Stream of Lookup Join

+1 for this proposal. This FLIP will make it possible for each lookup join parallel task to only access and cache a subset of the data. This will significantly improve the performance and reduce the overhead when using Paimon for the dimension table. And it's general enough to also be leveraged by

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

+1 (non-binding) - verified gpg signatures - verified sha512 hash - built from source code with java 8/11/17 - checked Github release tag - checked the CI result - checked release notes Danny Cranmer 于2024年4月22日周一 21:56写道： > Hi everyone, > > Please review and vote on release candidate #1 for fl

[jira] [Created] (FLINK-35546) Elasticsearch 8 connector fails fast for non-retryable bulk request items

Mingliang Liu created FLINK-35546: - Summary: Elasticsearch 8 connector fails fast for non-retryable bulk request items Key: FLINK-35546 URL: https://issues.apache.org/jira/browse/FLINK-35546 Project:

Savepoints not considered during failover

Hey Devs! What is the reason / rationale for savepoints being ignored during failover scenarios? I see they are not even recorded as the last valid checkpoint in the HA metadata (only the checkpoint id counter is bumped) so if the JM fails after a manually triggered savepoint the job will still f

Re: Savepoints not considered during failover

One reason could be that the savepoints are self-contained, owned by the user rather than Flink and, therefore, could be moved. Flink wouldn't have a proper reference in that case anymore. I don't have a link to a discussion, though. Best, Matthias On Fri, Jun 7, 2024 at 8:47 AM Gyula Fóra wrot