[jira] [Created] (FLINK-33424) Resolve the problem that yarnClient cannot load yarn configurations

2023-10-31 Thread zhengzhili (Jira)
zhengzhili created FLINK-33424: -- Summary: Resolve the problem that yarnClient cannot load yarn configurations Key: FLINK-33424 URL: https://issues.apache.org/jira/browse/FLINK-33424 Project: Flink

Re: [DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-10-31 Thread Lijie Wang
Hi Xia, Thanks for driving this FLIP, +1 for the proposal. I have 2 questions about the relationship between static inference and dynamic inference: 1. AFAIK, currently the hive table source enable static inference by default. In this case, which one (static vs dynamic) will take effect ? I thin

[jira] [Created] (FLINK-33423) Resolve the problem that yarnClient cannot load yarn configurations

2023-10-31 Thread zhengzhili (Jira)
zhengzhili created FLINK-33423: -- Summary: Resolve the problem that yarnClient cannot load yarn configurations Key: FLINK-33423 URL: https://issues.apache.org/jira/browse/FLINK-33423 Project: Flink

Re: [VOTE] Release flink-connector-aws, v4.2.0 release candidate #1

2023-10-31 Thread mystic lama
+1 (non-binding) - validated shasum - verified build - Java 8 - build good and all test cases pass - Java 11 - build good and all test cases pass Observations: got test failures with Java 17, something to look for in future On Tue, 31 Oct 2023 at 08:42, Danny Cranmer wrote: > Hi everyo

Re: Pointers to computational models of Flink CEP

2023-10-31 Thread David Anderson
The implementation of Flink CEP was largely based on Efficient Pattern Matching over Event Streams by Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman from UMass Amherst [1]. [1] https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf Cheers, David On Tue, Oct 31, 202

[jira] [Created] (FLINK-33422) Add restore tests for Calc node

2023-10-31 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-33422: --- Summary: Add restore tests for Calc node Key: FLINK-33422 URL: https://issues.apache.org/jira/browse/FLINK-33422 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33421) Implement ExecNode Restore Tests

2023-10-31 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-33421: --- Summary: Implement ExecNode Restore Tests Key: FLINK-33421 URL: https://issues.apache.org/jira/browse/FLINK-33421 Project: Flink Issue Type: Improvemen

[ANNOUNCE] Apache Flink Kafka Connectors 3.0.1 released

2023-10-31 Thread Tzu-Li (Gordon) Tai
The Apache Flink community is very happy to announce the release of Apache Flink Kafka Connectors 3.0.1. This release is compatible with the Apache Flink 1.17.x and 1.18.x release series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available

[jira] [Created] (FLINK-33420) Run e2e test fails Intermetiently with ClientCoordinationHandler : Unhandled exception

2023-10-31 Thread Samrat Deb (Jira)
Samrat Deb created FLINK-33420: -- Summary: Run e2e test fails Intermetiently with ClientCoordinationHandler : Unhandled exception Key: FLINK-33420 URL: https://issues.apache.org/jira/browse/FLINK-33420 Pr

Pointers to computational models of Flink CEP

2023-10-31 Thread Vishwas Kalani
Hey, I am a computer science student working on a project related to flink cep. I want to understand the theoretical foundations of flink cep. Could I get some pointers to the computational models used by Flink CEP and how do various elements of flink CEP function. Thanking you in advance

Re: [DISCUSS] AWS Connectors v4.2.0 release + 1.18 support

2023-10-31 Thread Danny Cranmer
There is no plan for the other connectors yet. I can pickup a few, will start a separate thread tomorrow if someone doesn't beat me to it. Danny. On Tue, 31 Oct 2023, 04:00 Leonard Xu, wrote: > +1, thanks Dany for driving this. > > One related question, do we have plan to find some volunteers t

[VOTE] Release flink-connector-aws, v4.2.0 release candidate #1

2023-10-31 Thread Danny Cranmer
Hi everyone, Please review and vote on release candidate #1 for the version 4.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the

[jira] [Created] (FLINK-33419) Port PROCTIME/ROWTIME functions to the new inference stack

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33419: Summary: Port PROCTIME/ROWTIME functions to the new inference stack Key: FLINK-33419 URL: https://issues.apache.org/jira/browse/FLINK-33419 Project: Flink

[jira] [Created] (FLINK-33418) SqlGatewayE2ECase failed due to ConnectException

2023-10-31 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33418: - Summary: SqlGatewayE2ECase failed due to ConnectException Key: FLINK-33418 URL: https://issues.apache.org/jira/browse/FLINK-33418 Project: Flink Issue Type

Re: Request to release flink 1.6.3

2023-10-31 Thread Rui Fan
Thanks Vikas for the ask! Hi devs, Is anyone willing to pick up the release of 1.16.3 and 1.17.2 with me? If so, I can volunteer to release one of the versions. If no one picks it up for more than three days, I volunteer to release two versions. After it’s determined, the official discussion can

Re: Request to release flink 1.6.3

2023-10-31 Thread vikas patil
Hello Rui, Do we need more votes for this or are we good to go with the release of 1.6.3 ? Please let me know. Thanks. -Vikas On Tue, Oct 24, 2023 at 9:27 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi Vikas, > > Thanks for your feedback! > > Do you mean flink 1.16.3 instead of 1.6.3? > > The 1.

Re: [DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-10-31 Thread Zhu Zhu
Thanks for opening the FLIP and kicking off this discussion, Xia! The proposed changes make up an important missing part of the dynamic parallelism inference of adaptive batch scheduler. Besides that, it is also one good step towards supporting dynamic parallelism inference for streaming sources,

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-31 Thread Timo Walther
Hi Jark, here are the checks I had in mind so far. But we can also discuss this during the implementation in the PRs. Most of the tasks are very similar to PARTITIONED BY which is also a characteristic of a sink. 1) Check that DISTRIBUTED BY columns reference physical columns and at least 1.

[jira] [Created] (FLINK-33417) Update netty version to 4.1.83 for flink-shaded

2023-10-31 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33417: - Summary: Update netty version to 4.1.83 for flink-shaded Key: FLINK-33417 URL: https://issues.apache.org/jira/browse/FLINK-33417 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-33416) FineGrainedSlotManagerTest failed with fatal error

2023-10-31 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33416: - Summary: FineGrainedSlotManagerTest failed with fatal error Key: FLINK-33416 URL: https://issues.apache.org/jira/browse/FLINK-33416 Project: Flink Issue Ty

[jira] [Created] (FLINK-33415) HAJobRunOnHadoopS3FileSystemITCase failed due to NoSuchMethodError

2023-10-31 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33415: - Summary: HAJobRunOnHadoopS3FileSystemITCase failed due to NoSuchMethodError Key: FLINK-33415 URL: https://issues.apache.org/jira/browse/FLINK-33415 Project: Flink

[jira] [Created] (FLINK-33414) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException

2023-10-31 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33414: - Summary: MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException Key: FLINK-33414 URL: https://issues.apache.org/jira/browse/FLINK-3

[jira] [Created] (FLINK-33413) Bump Avro in AWS Connectors to address CVE-2023-39410

2023-10-31 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-33413: - Summary: Bump Avro in AWS Connectors to address CVE-2023-39410 Key: FLINK-33413 URL: https://issues.apache.org/jira/browse/FLINK-33413 Project: Flink Issue

Re: [DISCUSS] Kubernetes Operator 1.7.0 release planning

2023-10-31 Thread Rui Fan
Thanks Gyula for driving this release! I'd like to check with you and community, could we postpone the code freeze by a week? I'm developing the FLINK-33099[1], and the prod code is done. I need some time to develop the tests. I hope this feature is included in 1.7.0 for two main reasons: 1. We

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

2023-10-31 Thread Alexander Fedulov
Hi Zhanghao, Thanks for the proposition. In general +1, this sounds like a good idea as long it is clear that the usage of these settings is discouraged. Just one minor concern - the configuration page is already very long, do you have a rough estimate of how many more options would be added with

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-10-31 Thread Jing Ge
Since we didn't get any new feedback from the reporters, +1 for closing FLINK-30238 according to the analysis done by Gordon, Martijn, and Alex. Happy to see we can move a little bit forward. Thanks for your effort! Best regards, Jing On Tue, Oct 31, 2023 at 3:03 AM Yun Gao wrote: > Hi Martijn

[jira] [Created] (FLINK-33412) Implement type inference for reinterpret_cast function

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33412: Summary: Implement type inference for reinterpret_cast function Key: FLINK-33412 URL: https://issues.apache.org/jira/browse/FLINK-33412 Project: Flink

[jira] [Created] (FLINK-33411) Implement type inference for window properties functions

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33411: Summary: Implement type inference for window properties functions Key: FLINK-33411 URL: https://issues.apache.org/jira/browse/FLINK-33411 Project: Flink

[jira] [Created] (FLINK-33410) Implement type inference for Over offsets functions

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33410: Summary: Implement type inference for Over offsets functions Key: FLINK-33410 URL: https://issues.apache.org/jira/browse/FLINK-33410 Project: Flink I

[jira] [Created] (FLINK-33409) Bump Guava to address CVE-2020-8908/CVE-2023-2976

2023-10-31 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-33409: - Summary: Bump Guava to address CVE-2020-8908/CVE-2023-2976 Key: FLINK-33409 URL: https://issues.apache.org/jira/browse/FLINK-33409 Project: Flink Issue Typ

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-10-31 Thread Jiabao Sun
Hi Becket, Actually, for FileSystemSource, it is not always desired, only OCR file formats support filter pushdown. We can disable predicate pushdown for FileSystemSource by setting 'table.optimizer.source.predicate-pushdown-enabled' to false. I think we can also disable filter pushdown at a m

[jira] [Created] (FLINK-33408) Fixing the container vulnerability by upgrade the SnakeYaml Maven dependency in flink-kubernetes module.

2023-10-31 Thread Zhou Shijie (Jira)
Zhou Shijie created FLINK-33408: --- Summary: Fixing the container vulnerability by upgrade the SnakeYaml Maven dependency in flink-kubernetes module. Key: FLINK-33408 URL: https://issues.apache.org/jira/browse/FLINK-3

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-10-31 Thread Becket Qin
Hi Jiabao, Thanks for the explanation. Maybe it's easier to explain with an example. Let's take FileSystemTableSource as an example. Currently it implements SupportsFilterPushDown interface. With your proposal, does it have to support `source.filter-push-down.enabled` as well? But this configurat

[jira] [Created] (FLINK-33407) Port time functions to the new type inference stack

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33407: Summary: Port time functions to the new type inference stack Key: FLINK-33407 URL: https://issues.apache.org/jira/browse/FLINK-33407 Project: Flink I

Re: [DISCUSS] Kubernetes Operator 1.7.0 release planning

2023-10-31 Thread Samrat Deb
Thank you Gyula (+1 non-binding) in support of you taking on the role of release manager. > I think this is reasonable as I am not aware of any big features / bug fixes being worked on right now. Given the size of the changes related to the autoscaler module refactor we should try to focus the re

[DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-10-31 Thread Xia Sun
Hi everyone, I would like to start a discussion on FLIP-379: Dynamic source parallelism inference for batch jobs[1]. In general, there are three main ways to set source parallelism for batch jobs: (1) User-defined source parallelism. (2) Connector static parallelism inference. (3) Dynamic parallel

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-10-31 Thread Jiabao Sun
Thanks Becket for the further explanation. Perhaps I didn't explain it clearly. 1. If a source does not implement the SupportsFilterPushDown interface, the newly added configurations do not need to be added to either the requiredOptions or optionalOptions. Similar to LookupOptions, if a sourc