Re: [DISCUSS] Confluent Avro support without Schema Registry access

2023-11-24 Thread Ryan Skraba
Hello Dale and Martijn, I've been looking into some schema registry issues, and I thought I'd bring this back up. I can *kind of* see the value in configuring the Flink job with sufficient information that you can run and/or test without a schema registry, but it really seems like the best way to

Re: [VOTE] Release 1.16.3, release candidate #1

2023-11-24 Thread Martijn Visser
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Fri, Nov 24, 2023 at 7:29 AM Jing Ge wrote: > > +1(not-binding) > > - verified signatures hash > - verified hash >

Re: [VOTE] Release flink-connector-aws, v4.2.0 release candidate #1

2023-11-24 Thread Martijn Visser
Hi Danny, Thanks for driving this. +1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Mon, Nov 20, 2023 at 12:29 PM Jiabao Sun wrote: > > Thanks Danny for driving t

[jira] [Created] (FLINK-33637) [JUnit5 Migration] Introduce ArchTest rules to ban Junit 4 for table-planner

2023-11-24 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-33637: -- Summary: [JUnit5 Migration] Introduce ArchTest rules to ban Junit 4 for table-planner Key: FLINK-33637 URL: https://issues.apache.org/jira/browse/FLINK-33637 Project: Fli

[jira] [Created] (FLINK-33638) Support variable-length data generation for variable-length data types

2023-11-24 Thread Yubin Li (Jira)
Yubin Li created FLINK-33638: Summary: Support variable-length data generation for variable-length data types Key: FLINK-33638 URL: https://issues.apache.org/jira/browse/FLINK-33638 Project: Flink

Re: [VOTE] Release 1.17.2, release candidate #1

2023-11-24 Thread Martijn Visser
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Fri, Nov 24, 2023 at 7:25 AM Jing Ge wrote: > > +1(not-binding) > > - verified signatures and hash > - reviewed the

Re: [DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-11-24 Thread Xia Sun
Hi all, Offline discussed with Zhu Zhu and Leonard Xu and we have reached the following three points of consensus: 1. Rename the interface method Context#getMaxSourceParallelism proposed by the FLIP to Context#getParallelismInferenceUpperBound, to make the meaning of the method clearer. See [1] fo

[jira] [Created] (FLINK-33639) Support zookeeper 3.9.0+

2023-11-24 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33639: --- Summary: Support zookeeper 3.9.0+ Key: FLINK-33639 URL: https://issues.apache.org/jira/browse/FLINK-33639 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-33640) Support configuring FLINK_CONF_FILE through environment variables

2023-11-24 Thread Matt Wang (Jira)
Matt Wang created FLINK-33640: - Summary: Support configuring FLINK_CONF_FILE through environment variables Key: FLINK-33640 URL: https://issues.apache.org/jira/browse/FLINK-33640 Project: Flink

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2023-11-24 Thread Gyula Fóra
Hi Peter! Thank you for the analysis of the options. I don't really have a strong opinion, but in general I am in favor of the mix in style interfaces. We follow the same approach for table sources / sinks as well. Some other benefits I see in addition to what you mentioned: - Easier to introdu

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

2023-11-24 Thread Gyula Fóra
Hi Surendra! The resource configuration in Flink is pretty well established and covers setting both memory requests and limits (through the limit factor.) Could you please elaborate why you think this change is a good addition? I see a few downsides: - It complicates memory configuration by add

Re: [DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-11-24 Thread Leonard Xu
Thanks Xia and Zhu Zhu for driving this work, It will help unify the parallelism inference for all operators of batch job, the updated FLIP looks good to me. Best, Leonard > 2023年11月24日 下午5:53,Xia Sun 写道: > > Hi all, > Offline discussed with Zhu Zhu and Leonard Xu and we have reached the > f

[jira] [Created] (FLINK-33641) RankITCase.testMultipleUnaryTopNAfterAgg fails on AZP

2023-11-24 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33641: --- Summary: RankITCase.testMultipleUnaryTopNAfterAgg fails on AZP Key: FLINK-33641 URL: https://issues.apache.org/jira/browse/FLINK-33641 Project: Flink I

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2023-11-24 Thread Leonard Xu
Thanks Danny for driving this. +1 (binding) - built from source code succeeded - verified signatures - verified hashsums - checked the contents contains jar and pom files in apache repo - checked the lowest supported flink version issue has been fixed - checked Github release tag - checked re

[jira] [Created] (FLINK-33642) Use docker image with jdk21 in CI

2023-11-24 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33642: --- Summary: Use docker image with jdk21 in CI Key: FLINK-33642 URL: https://issues.apache.org/jira/browse/FLINK-33642 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33643) Allow StreamExecutionEnvironment's executeAsync API to use default JobName

2023-11-24 Thread Matt Wang (Jira)
Matt Wang created FLINK-33643: - Summary: Allow StreamExecutionEnvironment's executeAsync API to use default JobName Key: FLINK-33643 URL: https://issues.apache.org/jira/browse/FLINK-33643 Project: Flink

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-24 Thread Maximilian Michels
Thanks for reviving the efforts here Matthias! +1 for the transition to GitHub Actions. As for ASF Infra Jenkins, it works fine. Jenkins is extremely feature-rich. Not sure about the spare capacity though. I know that for Apache Beam, Google donated a bunch of servers to get additional build capac

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz
+1(binding) And closing this vote thread, results will be announced in a separate email. Best, Dawid On Wed, 22 Nov 2023 at 14:17, Martijn Visser wrote: > +1 (binding) > > On Wed, Nov 22, 2023 at 2:05 AM Lincoln Lee > wrote: > > > > +1 (binding) > > > > Best, > > Lincoln Lee > > > > > > Serge

Re: [VOTE] Release 1.17.2, release candidate #1

2023-11-24 Thread Lincoln Lee
+1 (non-binding) - Verified signatures - Build the source with Maven - start cluster bin/start-cluster.sh with java8, it works fine - started SQL Client, used datagen source to verify several SQL cases(FLINK-33010,FLINK-32456,FLINK-30966), result seems ok. Best, Lincoln Lee Martijn Visser 于202

[RESULT][VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz
FLIP-393 [1] has been accepted and voted through this thread [2]. The proposal received 10 approving votes, 7 of which are binding, and there is no disapproval. Benchao Li (binding) Martijn Visser (binding) Gyula Fora (binding) Timo Walther (binding) Sergey Nuyanzin (binding) Lincoln Lee (binding

[jira] [Created] (FLINK-33644) FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33644: Summary: FLIP-393: Make QueryOperations SQL serializable Key: FLINK-33644 URL: https://issues.apache.org/jira/browse/FLINK-33644 Project: Flink Issue

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2023-11-24 Thread Martijn Visser
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Fri, Nov 24, 2023 at 12:38 PM Leonard Xu wrote: > > Thanks Danny for driving this. > > > +1 (binding) > > - built f

[jira] [Created] (FLINK-33645) Env vars in config not added to Taskmanagers in Standalone mode

2023-11-24 Thread Tony Garrard (Jira)
Tony Garrard created FLINK-33645: Summary: Env vars in config not added to Taskmanagers in Standalone mode Key: FLINK-33645 URL: https://issues.apache.org/jira/browse/FLINK-33645 Project: Flink

[VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-24 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on release candidate #1 for version 3.0.2 of the Apache Flink Kafka Connector, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Compared to the previous hotfix release, this release only additionally

Re: [DISCUSS] Towards flink-shaded release 18.0

2023-11-24 Thread Sergey Nuyanzin
Thanks everyone for responses I've started preparation steps to make it happened by following steps [1] One of the preliminary steps is publishing GPG key dist.apache.org I don't have write access there and as it is also mentioned at wiki page >Note: Only PMC members have write access to the rel

Re: [VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-24 Thread Tzu-Li (Gordon) Tai
+1 (binding) - Verified signature and hashes - Verified mvn dependency:tree for a typical user job jar [1]. When using Flink 1.18.0, flink-connector-base is no longer getting bundled, and all Flink dependencies resolve as 1.18.0 / provided. - Submitting user job jar to local Flink 1.18.0 cluster w

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

2023-11-24 Thread Surendra Singh Lilhore
Hi Gyula, Thank you for raising these valid concerns and sharing your perspective. I agree that this new change will impact the existing pipelines. I brought up this issue because in Kubernetes environments, resource configuration is typically managed through Kubernetes objects like container tem

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

2023-11-24 Thread Gyula Fóra
I think a better and more Kubernetes canonical approach would be what is outlined in this ticket: https://issues.apache.org/jira/browse/FLINK-33548 The upside is that it will work for all operator supported Flink versions and is much simpler to use than the pod template. What do you think? Gyul

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

2023-11-24 Thread Surendra Singh Lilhore
Hi Gyula, Yes, I agree with this approach. Thanks Surendra On Sat, Nov 25, 2023, 12:28 AM Gyula Fóra wrote: > I think a better and more Kubernetes canonical approach would be what is > outlined in this ticket: > > https://issues.apache.org/jira/browse/FLINK-33548 > > The upside is that it w

Re: Discussion: [FLINK-33609] Take into account the resource limit specified in the pod template

2023-11-24 Thread Gyula Fóra
Hi In that case we could also close your ticket and the one you linked and refer to this one as the selected approach :) Gyula On Fri, 24 Nov 2023 at 20:19, Surendra Singh Lilhore < surendralilh...@gmail.com> wrote: > Hi Gyula, > > > Yes, I agree with this approach. > > > Thanks > Surendra > >

[jira] [Created] (FLINK-33646) Cleanup the rest usage of deprecated TableEnvironment#registerFunction and in docs

2023-11-24 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33646: --- Summary: Cleanup the rest usage of deprecated TableEnvironment#registerFunction and in docs Key: FLINK-33646 URL: https://issues.apache.org/jira/browse/FLINK-33646

[jira] [Created] (FLINK-33647) Implement restore tests for LookupJoin node

2023-11-24 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-33647: --- Summary: Implement restore tests for LookupJoin node Key: FLINK-33647 URL: https://issues.apache.org/jira/browse/FLINK-33647 Project: Flink Issue Type:

[jira] [Created] (FLINK-33648) table usfs doc in Chinese is out-of-date

2023-11-24 Thread Jing Ge (Jira)
Jing Ge created FLINK-33648: --- Summary: table usfs doc in Chinese is out-of-date Key: FLINK-33648 URL: https://issues.apache.org/jira/browse/FLINK-33648 Project: Flink Issue Type: Improvement

Re: [VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-24 Thread Rui Fan
+1 (non-binding) - Validated checksum hash - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven and jdk8 - Verified licenses - Verified web PRs Best, Rui On Sat, Nov 25, 2023 at 2:05 AM Tzu-Li (Gordon) Tai wrote: > +1 (binding) > > - Verifi

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2023-11-24 Thread Rui Fan
+1 (non-binding) - Validated checksum hash - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven and jdk8 - Verified licenses - Verified web PRs, left a minor comment Best, Rui On Fri, Nov 24, 2023 at 10:26 PM Martijn Visser wrote: > +1 (bin

[jira] [Created] (FLINK-33649) Exponential Delay doc is missed

2023-11-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-33649: --- Summary: Exponential Delay doc is missed Key: FLINK-33649 URL: https://issues.apache.org/jira/browse/FLINK-33649 Project: Flink Issue Type: Technical Debt Co