[jira] [Created] (FLINK-26041) AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure hang on azure

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26041: --- Summary: AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure hang on azure Key: FLINK-26041 URL: https://issues.apache.org/jira/browse/FLINK-26041 Proj

[jira] [Created] (FLINK-26042) PyFlinkEmbeddedSubInterpreterTests. test_udf_without_arguments failed on azure

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26042: --- Summary: PyFlinkEmbeddedSubInterpreterTests. test_udf_without_arguments failed on azure Key: FLINK-26042 URL: https://issues.apache.org/jira/browse/FLINK-26042 Project: Flink

[jira] [Created] (FLINK-26043) Add periodic kerberos relogin to DelegationTokenManager

2022-02-09 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-26043: - Summary: Add periodic kerberos relogin to DelegationTokenManager Key: FLINK-26043 URL: https://issues.apache.org/jira/browse/FLINK-26043 Project: Flink Iss

[jira] [Created] (FLINK-26044) Properly declare internal UNNEST function

2022-02-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-26044: Summary: Properly declare internal UNNEST function Key: FLINK-26044 URL: https://issues.apache.org/jira/browse/FLINK-26044 Project: Flink Issue Type: Sub-tas

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-09 Thread Till Rohrmann
Hi Galen, Great to hear it :-) I've assigned you to the ticket [1]. Next you can open a PR against the repository and we will review it. [1] https://issues.apache.org/jira/browse/FLINK-25933 Cheers, Till On Tue, Feb 8, 2022 at 11:58 PM Galen Warren wrote: > I'm ready to pick this one up, I ha

[jira] [Created] (FLINK-26045) Document error status codes for the REST API

2022-02-09 Thread Jira
David Morávek created FLINK-26045: - Summary: Document error status codes for the REST API Key: FLINK-26045 URL: https://issues.apache.org/jira/browse/FLINK-26045 Project: Flink Issue Type: Te

[jira] [Created] (FLINK-26046) Link to OpenAPI specification is dead

2022-02-09 Thread Jira
David Morávek created FLINK-26046: - Summary: Link to OpenAPI specification is dead Key: FLINK-26046 URL: https://issues.apache.org/jira/browse/FLINK-26046 Project: Flink Issue Type: Technical

Azure Pipelines are dealing with an incident, causing pipeline runs to fail

2022-02-09 Thread Martijn Visser
Hi everyone, Please keep in mind that Azure Pipelines currently is dealing with an incident [1] which causes all CI pipeline runs on Azure to fail. When the incident has been resolved, it will be required to retrigger your pipeline to see if the pipeline then passes. Best regards, Martijn Visser

[jira] [Created] (FLINK-26047) Support usrlib in HDFS for YARN application mode

2022-02-09 Thread Biao Geng (Jira)
Biao Geng created FLINK-26047: - Summary: Support usrlib in HDFS for YARN application mode Key: FLINK-26047 URL: https://issues.apache.org/jira/browse/FLINK-26047 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-26048) Yarn access should be separated by semicolon

2022-02-09 Thread MichealShin (Jira)
MichealShin created FLINK-26048: --- Summary: Yarn access should be separated by semicolon Key: FLINK-26048 URL: https://issues.apache.org/jira/browse/FLINK-26048 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-26049) The tolerable-failed-checkpoints logic is invalid when checkpoint trigger failed

2022-02-09 Thread fanrui (Jira)
fanrui created FLINK-26049: -- Summary: The tolerable-failed-checkpoints logic is invalid when checkpoint trigger failed Key: FLINK-26049 URL: https://issues.apache.org/jira/browse/FLINK-26049 Project: Flink

[jira] [Created] (FLINK-26051) row_number =1 and Subsequent SQL has "case when" and "where" statement : The window can only be ordered in ASCENDING mode

2022-02-09 Thread chuncheng wu (Jira)
chuncheng wu created FLINK-26051: Summary: row_number =1 and Subsequent SQL has "case when" and "where" statement : The window can only be ordered in ASCENDING mode Key: FLINK-26051 URL: https://issues.apache.org/

[jira] [Created] (FLINK-26050) Too Many small sst files in rocksdb state backend when using processing time window

2022-02-09 Thread shen (Jira)
shen created FLINK-26050: Summary: Too Many small sst files in rocksdb state backend when using processing time window Key: FLINK-26050 URL: https://issues.apache.org/jira/browse/FLINK-26050 Project: Flink

Re: Azure Pipelines are dealing with an incident, causing pipeline runs to fail

2022-02-09 Thread Martijn Visser
Hi everyone, The issue should now be resolved. Best regards, Martijn On Wed, 9 Feb 2022 at 10:55, Martijn Visser wrote: > Hi everyone, > > Please keep in mind that Azure Pipelines currently is dealing with an > incident [1] which causes all CI pipeline runs on Azure to fail. When the > incide

[jira] [Created] (FLINK-26052) Update chinese documentation regarding FLIP-203

2022-02-09 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-26052: Summary: Update chinese documentation regarding FLIP-203 Key: FLINK-26052 URL: https://issues.apache.org/jira/browse/FLINK-26052 Project: Flink Issue

[DISCUSS] FLINK-25927: Make flink-connector-base dependency usage consistent across all connectors

2022-02-09 Thread Alexander Fedulov
Hi everyone, I would like to discuss the best approach to address the issue raised in FLINK-25927 [1]. It can be summarized as follows: > flink-connector-base is currently inconsistently used in connectors (directly shaded in some and transitively pulled in via flink-connector-files which is itse

[DISCUSS] Drop Jepsen tests

2022-02-09 Thread Chesnay Schepler
For a few years by now we had a set of Jepsen tests that verify the correctness of Flinks coordination layer in the case of process crashes. In the past it has indeed found issues and thus provided value to the project, and in general the core idea of it (and Jepsen for that matter) is very soun

Re: Azure Pipelines are dealing with an incident, causing pipeline runs to fail

2022-02-09 Thread Martijn Visser
Unfortunately it looks like there are still failures. Will keep you posted On Wed, 9 Feb 2022 at 11:51, Martijn Visser wrote: > Hi everyone, > > The issue should now be resolved. > > Best regards, > > Martijn > > On Wed, 9 Feb 2022 at 10:55, Martijn Visser wrote: > >> Hi everyone, >> >> Please

Re: Azure Pipelines are dealing with an incident, causing pipeline runs to fail

2022-02-09 Thread Robert Metzger
I filed a support request with Microsoft: https://developercommunity.visualstudio.com/t/Number-of-Microsoft-hosted-agents-droppe/1658827?from=email&space=21&sort=newest On Wed, Feb 9, 2022 at 1:04 PM Martijn Visser wrote: > Unfortunately it looks like there are still failures. Will keep you post

[jira] [Created] (FLINK-26053) Fix parser generator warnings

2022-02-09 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-26053: --- Summary: Fix parser generator warnings Key: FLINK-26053 URL: https://issues.apache.org/jira/browse/FLINK-26053 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-26054) Enable maven-enforcer to disable table-planner and table-runtime as dependencies

2022-02-09 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-26054: --- Summary: Enable maven-enforcer to disable table-planner and table-runtime as dependencies Key: FLINK-26054 URL: https://issues.apache.org/jira/browse/FLINK-26054

[jira] [Created] (FLINK-26055) Use new FlinkVersion enum for serde plans

2022-02-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-26055: Summary: Use new FlinkVersion enum for serde plans Key: FLINK-26055 URL: https://issues.apache.org/jira/browse/FLINK-26055 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-26056) CLONE - [FLIP-171] KDS implementation of Async Sink Table API

2022-02-09 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-26056: --- Summary: CLONE - [FLIP-171] KDS implementation of Async Sink Table API Key: FLINK-26056 URL: https://issues.apache.org/jira/browse/FLINK-26056 Project: Flink

[jira] [Created] (FLINK-26057) CLONE - [FLIP-171] Add SQL Client Uber jar for KinesisDataStreams Sink.

2022-02-09 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-26057: --- Summary: CLONE - [FLIP-171] Add SQL Client Uber jar for KinesisDataStreams Sink. Key: FLINK-26057 URL: https://issues.apache.org/jira/browse/FLINK-26057 Project: Flink

[jira] [Created] (FLINK-26058) KinesisDynamicTableFactory implements DynamicSourceFactory but uses sink options

2022-02-09 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-26058: --- Summary: KinesisDynamicTableFactory implements DynamicSourceFactory but uses sink options Key: FLINK-26058 URL: https://issues.apache.org/jira/browse/FLINK-26058 Projec

[jira] [Created] (FLINK-26059) ResourceProfile#ANY can cause overflow exceptions

2022-02-09 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-26059: Summary: ResourceProfile#ANY can cause overflow exceptions Key: FLINK-26059 URL: https://issues.apache.org/jira/browse/FLINK-26059 Project: Flink Iss

[jira] [Created] (FLINK-26060) Make Python specific exec nodes unsupported

2022-02-09 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-26060: --- Summary: Make Python specific exec nodes unsupported Key: FLINK-26060 URL: https://issues.apache.org/jira/browse/FLINK-26060 Project: Flink Is

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread Konstantin Knauf
Thank you for raising this issue. What risks do you see if we drop it? Do you see any cheaper alternative to (partially) mitigate those risks? On Wed, Feb 9, 2022 at 12:40 PM Chesnay Schepler wrote: > For a few years by now we had a set of Jepsen tests that verify the > correctness of Flinks coo

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread Chesnay Schepler
The jepsen tests cover 3 cases: a) JM/TM crashes b) HDFS namenode crash (aka, can't checkpoint because HDFS is down) c) network partitions a) can (and probably is) reasonably covered by existing ITCases and e2e tests b) We could probably figure this out ourselves if we wanted to. c) is the diff

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread Chesnay Schepler
b/c are part of the same test. * We have a job running, * trigger a network partition (failing the job), * then crash HDFS (preventing checkpoints and access to the HA storageDir), * then the partition is resolved and HDFS is started again. Conceptually I would think we can replicate this

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread David Morávek
Network partitions are trickier than simply crashing process. For example these can be asymmetric -> as a TM you're still able to talk to the JM, but you're not able to talk to other TMs. In general this could be achieved by manipulating iptables on the host machine (considering we spawn all the p

Re: Statefun async http request via RequestReplyFunctionBuilder

2022-02-09 Thread Galen Warren
Thanks! PR created and noted on the ticket. On Wed, Feb 9, 2022 at 3:42 AM Till Rohrmann wrote: > Hi Galen, > > Great to hear it :-) I've assigned you to the ticket [1]. Next you can open > a PR against the repository and we will review it. > > [1] https://issues.apache.org/jira/browse/FLINK-259

[jira] [Created] (FLINK-26061) FileSystem.getFileStatus fails for directories

2022-02-09 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-26061: - Summary: FileSystem.getFileStatus fails for directories Key: FLINK-26061 URL: https://issues.apache.org/jira/browse/FLINK-26061 Project: Flink Issue Type:

[jira] [Created] (FLINK-26062) [Changelog] Non-deterministic recovery of PriorityQueue states

2022-02-09 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-26062: - Summary: [Changelog] Non-deterministic recovery of PriorityQueue states Key: FLINK-26062 URL: https://issues.apache.org/jira/browse/FLINK-26062 Project: Fli

[jira] [Created] (FLINK-26063) [Changelog] Incorrect key group logged for PQ.poll and remove

2022-02-09 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-26063: - Summary: [Changelog] Incorrect key group logged for PQ.poll and remove Key: FLINK-26063 URL: https://issues.apache.org/jira/browse/FLINK-26063 Project: Flin

[jira] [Created] (FLINK-26064) KinesisFirehoseSinkITCase IllegalStateException: Trying to access closed classloader

2022-02-09 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-26064: -- Summary: KinesisFirehoseSinkITCase IllegalStateException: Trying to access closed classloader Key: FLINK-26064 URL: https://issues.apache.org/jira/browse/FLINK-26064

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread Austin Cawley-Edwards
Are there e2e tests that run on kubernetes? Perhaps k8s network policies[1] would be an option to simulate asymmetric network partitions without modifying iptables in a more approachable way? Austin [1]: https://kubernetes.io/docs/concepts/services-networking/network-policies/ On Wed, Feb 9, 20

Re: [DISCUSS] Drop Jepsen tests

2022-02-09 Thread Yang Wang
@Austin We already have some e2e tests[1] that guards k8s deployment(both session and application, with or without HA). And I agree with you that network partition could be simulated by K8s network policy. [1]. https://github.com/apache/flink/blob/master/flink-end-to-end-tests/test-scripts/test_k

[jira] [Created] (FLINK-26065) org.apache.flink.table.api.PlanReference$ContentPlanReference $FilePlanReference $ResourcePlanReference violation the api rules

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26065: --- Summary: org.apache.flink.table.api.PlanReference$ContentPlanReference $FilePlanReference $ResourcePlanReference violation the api rules Key: FLINK-26065 URL: https://issues.apache.org/jir

Re: [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-09 Thread Thomas Weise
+1 (binding) On Mon, Feb 7, 2022 at 1:10 AM Yangze Guo wrote: > > +1 (binding) > > Best, > Yangze Guo > > On Mon, Feb 7, 2022 at 5:04 PM K Fred wrote: > > > > +1 (non-binding) > > > > Best Regards > > Peng Yuan > > > > On Mon, Feb 7, 2022 at 4:49 PM Danny Cranmer > > wrote: > > > > > +1 (bindin

[RESULT] [VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-09 Thread Thomas Weise
Hi everyone, FLIP-212 [1] has been accepted. There were 7 binding +1 votes and 6 non-binding +1 votes. No other votes. binding: Gyula Fóra Márton Balassi Xintong Song Yang Wang Danny Cranmer Yangze Guo Thomas Weise non-binding: Chenya Zhang Shqiprim Bunjaku Israel Ekpo Peter Huang Biao Geng K Fr

[jira] [Created] (FLINK-26066) Introduce FileStoreRead

2022-02-09 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-26066: --- Summary: Introduce FileStoreRead Key: FLINK-26066 URL: https://issues.apache.org/jira/browse/FLINK-26066 Project: Flink Issue Type: Sub-task Componen

[jira] [Created] (FLINK-26067) ZooKeeperLeaderElectionConnectionHandlingTest. testLoseLeadershipOnLostConnectionIfTolerateSuspendedConnectionsIsEnabled failed due to timeout

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26067: --- Summary: ZooKeeperLeaderElectionConnectionHandlingTest. testLoseLeadershipOnLostConnectionIfTolerateSuspendedConnectionsIsEnabled failed due to timeout Key: FLINK-26067 URL: https://issues

[jira] [Created] (FLINK-26068) ZooKeeperJobGraphsStoreITCase.testPutAndRemoveJobGraph failed on azure

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26068: --- Summary: ZooKeeperJobGraphsStoreITCase.testPutAndRemoveJobGraph failed on azure Key: FLINK-26068 URL: https://issues.apache.org/jira/browse/FLINK-26068 Project: Flink

[jira] [Created] (FLINK-26069) KinesisFirehoseSinkITCase failed due to org.testcontainers.containers.ContainerLaunchException: Container startup failed

2022-02-09 Thread Yun Gao (Jira)
Yun Gao created FLINK-26069: --- Summary: KinesisFirehoseSinkITCase failed due to org.testcontainers.containers.ContainerLaunchException: Container startup failed Key: FLINK-26069 URL: https://issues.apache.org/jira/browse

Re: Azure Pipelines are dealing with an incident, causing pipeline runs to fail

2022-02-09 Thread Martijn Visser
It looks like the issues haven't been resolved yet unfortunately. On Wed, 9 Feb 2022 at 13:15, Robert Metzger wrote: > I filed a support request with Microsoft: > > https://developercommunity.visualstudio.com/t/Number-of-Microsoft-hosted-agents-droppe/1658827?from=email&space=21&sort=newest > >