Re: [PR] [hotfix] Add flink-connector-hive in README [flink]

2024-01-12 Thread via GitHub
caicancai commented on PR #24081: URL: https://github.com/apache/flink/pull/24081#issuecomment-1890354118 @RocMarshal @1996fanrui Hello, if you have time, can you review this PR? I’m not sure whether this connector needs to be added, thank you. -- This is an automated message from the

Re: [PR] [FLINK-33768] Support dynamic source parallelism inference for batch jobs [flink]

2024-01-12 Thread via GitHub
SinBex commented on PR #24078: URL: https://github.com/apache/flink/pull/24078#issuecomment-1890349712 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
dmvk commented on PR #24079: URL: https://github.com/apache/flink/pull/24079#issuecomment-1890348927 `testOperatorEventLostWithReaderFailure` ... reliably fails when trying to restore from compressed state, will investigate further :/ -- This is an automated message from the Apache Git Se

Re: [PR] [FLINK-33980][core] Reorganize job configuration [flink]

2024-01-12 Thread via GitHub
1996fanrui commented on code in PR #24025: URL: https://github.com/apache/flink/pull/24025#discussion_r1451171109 ## flink-streaming-java/src/test/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironmentComplexConfigurationTest.java: ## @@ -222,7 +223,7 @@ publ

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
1996fanrui commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1451108521 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingExecutor.java: ## @@ -247,6 +254,69 @@ private boolean isJobUnderMemoryPressure

[jira] [Updated] (FLINK-33230) Support Expanding ExecutionGraph to StreamGraph in Web UI

2024-01-12 Thread macdoor615 (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] macdoor615 updated FLINK-33230: --- Attachment: (was: image.png) > Support Expanding ExecutionGraph to StreamGraph in Web UI > -

[jira] [Updated] (FLINK-33230) Support Expanding ExecutionGraph to StreamGraph in Web UI

2024-01-12 Thread macdoor615 (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] macdoor615 updated FLINK-33230: --- Attachment: image.png > Support Expanding ExecutionGraph to StreamGraph in Web UI >

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
AlanConfluent commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1451023480 ## flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/extraction/ExtractionUtils.java: ## @@ -348,6 +350,53 @@ public static Optional> extr

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
AlanConfluent commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1451023480 ## flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/extraction/ExtractionUtils.java: ## @@ -348,6 +350,53 @@ public static Optional> extr

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
AlanConfluent commented on PR #23975: URL: https://github.com/apache/flink/pull/23975#issuecomment-1890159845 >The only issue currently is a green build. Is the error in the Python related to your changes? I'm pretty sure it's not. I can't make any sense of it at all anyhow. Will r

Re: [PR] [FLINK-33915][ci] Adds nightly workflow for the master branch [flink]

2024-01-12 Thread via GitHub
XComp commented on code in PR #23971: URL: https://github.com/apache/flink/pull/23971#discussion_r1450880042 ## .github/actions/select_workflow_configs/action.yml: ## @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] [BP-1.18][FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
flinkbot commented on PR #24083: URL: https://github.com/apache/flink/pull/24083#issuecomment-1889725940 ## CI report: * cb3b8abdeba9c5153e76062a54aabfcb0185cdf1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] Add GCP PubSub connector v3.0.2 [flink-web]

2024-01-12 Thread via GitHub
dannycranmer merged PR #695: URL: https://github.com/apache/flink-web/pull/695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450742988 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450738536 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450737039 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

[jira] [Commented] (FLINK-34063) When snapshot compression is enabled, rescaling of a source operator leads to some splits getting lost

2024-01-12 Thread Jira
[ https://issues.apache.org/jira/browse/FLINK-34063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806131#comment-17806131 ] David Morávek commented on FLINK-34063: --- Hi [~yang], it would be great if you open

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
dmvk commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450722556 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream( @Ove

[jira] [Comment Edited] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Jiang Xin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806125#comment-17806125 ] Jiang Xin edited comment on FLINK-34070 at 1/12/24 5:08 PM:

[jira] [Commented] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Jiang Xin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806125#comment-17806125 ] Jiang Xin commented on FLINK-34070: --- [~mapohl] Thanks for the clarification of FLINK-3

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
gyfora commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450665802 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * Lice

Re: [PR] [FLINK-33915][ci] Adds nightly workflow for the master branch [flink]

2024-01-12 Thread via GitHub
XComp commented on code in PR #23971: URL: https://github.com/apache/flink/pull/23971#discussion_r1450664278 ## .github/actions/select_workflow_configs/action.yml: ## @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450661344 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-34070][test] Adds dedicated tests in MiniClusterITCase for scenarios where there are not enough slots available. [flink]

2024-01-12 Thread via GitHub
flinkbot commented on PR #24082: URL: https://github.com/apache/flink/pull/24082#issuecomment-1889580149 ## CI report: * 44d3183ada4a4a65444f1ceba66ce83d3079dbba UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [hotfix] Fix incorrect echo in python_ci.yml [flink-connector-shared-utils]

2024-01-12 Thread via GitHub
MartijnVisser merged PR #32: URL: https://github.com/apache/flink-connector-shared-utils/pull/32 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-

[PR] [FLINK-34070][test] Adds dedicated tests in MiniClusterITCase for scenarios where there are not enough slots available. [flink]

2024-01-12 Thread via GitHub
XComp opened a new pull request, #24082: URL: https://github.com/apache/flink/pull/24082 ## What is the purpose of the change See FLINK-34070 for more context on this change. ## Brief change log * Adds dedicated test for no-resource-available handling through timeout *

[jira] [Updated] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34070: --- Labels: pull-request-available test-stability (was: test-stability) > MiniClusterITCase.tes

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
gyfora commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450645183 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * Lice

[jira] [Commented] (FLINK-34063) When snapshot compression is enabled, rescaling of a source operator leads to some splits getting lost

2024-01-12 Thread Yang LI (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806111#comment-17806111 ] Yang LI commented on FLINK-34063: - FYI, I have encountered also the issue of losing kafk

[jira] [Commented] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806110#comment-17806110 ] Matthias Pohl commented on FLINK-34070: --- [~chesnay] Do you remember whether there

[jira] [Commented] (FLINK-33705) Upgrade flink-shaded to 18.0

2024-01-12 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806106#comment-17806106 ] Sergey Nuyanzin commented on FLINK-33705: - Merged to master as [5d9d8748b64ff1a

[jira] [Resolved] (FLINK-33705) Upgrade flink-shaded to 18.0

2024-01-12 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-33705. - Fix Version/s: 1.19.0 Resolution: Fixed > Upgrade flink-shaded to 18.0 >

[jira] [Closed] (FLINK-33705) Upgrade flink-shaded to 18.0

2024-01-12 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin closed FLINK-33705. --- > Upgrade flink-shaded to 18.0 > > > Key: FLINK-337

Re: [PR] [FLINK-33705] Upgrade to flink-shaded 18.0 [flink]

2024-01-12 Thread via GitHub
snuyanzin merged PR #23838: URL: https://github.com/apache/flink/pull/23838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Closed] (FLINK-33979) Implement restore tests for TableSink node

2024-01-12 Thread Dawid Wysakowicz (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Wysakowicz closed FLINK-33979. Fix Version/s: 1.19.0 Resolution: Implemented Implemented in 9a9b9ce81ca05398f8891

Re: [PR] [FLINK-33979] Implement restore tests for TableSink node [flink]

2024-01-12 Thread via GitHub
dawidwys merged PR #24024: URL: https://github.com/apache/flink/pull/24024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apac

Re: [PR] [FLINK-32416] initial implementation of DynamicKafkaSource with bound… [flink-connector-kafka]

2024-01-12 Thread via GitHub
mxm merged PR #44: URL: https://github.com/apache/flink-connector-kafka/pull/44 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450588321 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450588940 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450588671 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/FlinkOperator.java: ## @@ -165,7 +167,12 @@ void registerDeploymentControll

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450588017 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/resources/ClusterResourceManager.java: ## @@ -0,0 +1,282 @@ +/* + * License

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450587771 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingExecutor.java: ## @@ -247,6 +254,69 @@ private boolean isJobUnderMemoryPressure(

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450586863 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/resources/ResourceRequirements.java: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450585907 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/resources/ResourceCheck.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [FLINK-33771] Add cluster capacity awareness to autoscaler [flink-kubernetes-operator]

2024-01-12 Thread via GitHub
mxm commented on code in PR #751: URL: https://github.com/apache/flink-kubernetes-operator/pull/751#discussion_r1450585550 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/resources/ResourceCheck.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [hotfix] Add flink-connector-hive in README [flink]

2024-01-12 Thread via GitHub
flinkbot commented on PR #24081: URL: https://github.com/apache/flink/pull/24081#issuecomment-1889457796 ## CI report: * 4147aacae70c6407f625c88ffdf1af0fdab4c4a6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
twalthr commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1450572179 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/runtime/stream/table/AsyncCalcITCase.java: ## @@ -0,0 +1,561 @@ +/* + * Licensed to the

Re: [PR] [FLINK-33365] include filters with Lookup joins [flink-connector-jdbc]

2024-01-12 Thread via GitHub
davidradl commented on PR #79: URL: https://github.com/apache/flink-connector-jdbc/pull/79#issuecomment-1889448042 > JdbcRowDataInputFormat @libenchao This sounds interesting. An early fix that I produced text replaced ? for the parameters; this was a simple change and did not need

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
twalthr commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1450566541 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/plan/rules/logical/AsyncCalcSplitRuleTest.java: ## @@ -0,0 +1,274 @@ +/* + * Licensed to

[PR] [hotfix] Add flink-connector-hive in README [flink]

2024-01-12 Thread via GitHub
caicancai opened a new pull request, #24081: URL: https://github.com/apache/flink/pull/24081 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450546553 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
twalthr commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1450552569 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/rules/logical/AsyncCalcSplitRuleTest.xml: ## @@ -0,0 +1,536 @@ + + + +

[jira] [Commented] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806077#comment-17806077 ] Gyula Fora commented on FLINK-33907: Are you using the maven wrapper?  ./mvnw -pl fl

Re: [PR] [FLINK-34027] Introduces AsyncScalarFunction as a new UDF type [flink]

2024-01-12 Thread via GitHub
twalthr commented on code in PR #23975: URL: https://github.com/apache/flink/pull/23975#discussion_r1450552569 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/rules/logical/AsyncCalcSplitRuleTest.xml: ## @@ -0,0 +1,536 @@ + + + +

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450546553 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

[jira] [Commented] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806072#comment-17806072 ] Matthias Pohl commented on FLINK-34070: --- The {{AdaptiveScheduler}} uses {{Declarat

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450546553 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

Re: [PR] [FLINK-33980][core] Reorganize job configuration [flink]

2024-01-12 Thread via GitHub
JunRuiLee commented on code in PR #24025: URL: https://github.com/apache/flink/pull/24025#discussion_r1450526101 ## flink-tests/src/test/java/org/apache/flink/test/checkpointing/CheckpointAfterAllTasksFinishedITCase.java: ## @@ -82,7 +83,8 @@ public void setUp() { @Test

Re: [PR] FLK-1186 implementation for the 4 labs in module 1 [flink-training]

2024-01-12 Thread via GitHub
ness-TraianBoboc closed pull request #72: FLK-1186 implementation for the 4 labs in module 1 URL: https://github.com/apache/flink-training/pull/72 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [FLINK-34067] Fix javacc warnings in flink-sql-parser [flink]

2024-01-12 Thread via GitHub
jnh5y commented on PR #24076: URL: https://github.com/apache/flink/pull/24076#issuecomment-1889340287 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450506731 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

[PR] FLK-1186 implementation for the 4 labs in module 1 [flink-training]

2024-01-12 Thread via GitHub
ness-TraianBoboc opened a new pull request, #72: URL: https://github.com/apache/flink-training/pull/72 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] [FLINK-33268][rest] Skip unknown fields in REST response deserialization [flink]

2024-01-12 Thread via GitHub
gaborgsomogyi commented on PR #23930: URL: https://github.com/apache/flink/pull/23930#issuecomment-1889300134 Unrelated error again: ``` Jan 12 13:24:31 mypy: can't read file '/__w/3/s/flink-python/dev/.conda/lib/python3.10/site-packages//google': No such file or directory ``` R

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
dmvk commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450490276 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream( @Ove

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-01-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1450463846 ## prometheus-connector/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusSinkWriter.java: ## @@ -0,0 +1,267 @@ +/* + * Licensed to the

[jira] [Assigned] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl reassigned FLINK-34070: - Assignee: Matthias Pohl > MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fa

[jira] [Commented] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806055#comment-17806055 ] Matthias Pohl commented on FLINK-34070: --- This issue can be reproduced locally: {co

Re: [PR] [FLINK-33365] include filters with Lookup joins [flink-connector-jdbc]

2024-01-12 Thread via GitHub
libenchao commented on PR #79: URL: https://github.com/apache/flink-connector-jdbc/pull/79#issuecomment-1889182758 @snuyanzin My rough idea is like: * Pass `resolvedPredicates` and `pushdownParams` to `JdbcRowDataLookupFunction` * `FieldNamedPreparedStatement` may be need to adapt to

[jira] [Comment Edited] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806052#comment-17806052 ] Matthias Pohl edited comment on FLINK-33907 at 1/12/24 1:06 PM: --

[jira] [Commented] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806052#comment-17806052 ] Matthias Pohl commented on FLINK-33907: --- I tried the same with a clean Maven repos

[jira] [Commented] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806051#comment-17806051 ] Matthias Pohl commented on FLINK-33907: --- Hm, it works for me locally: {code:java}

Re: [PR] [FLINK-33980][core] Reorganize job configuration [flink]

2024-01-12 Thread via GitHub
zhuzhurk commented on code in PR #24025: URL: https://github.com/apache/flink/pull/24025#discussion_r1448756961 ## flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java: ## @@ -2562,19 +2505,26 @@ private StreamGraphGenerat

[PR] Add RabbitMQ 3.0.2 connector [flink-web]

2024-01-12 Thread via GitHub
MartijnVisser opened a new pull request, #712: URL: https://github.com/apache/flink-web/pull/712 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[jira] [Commented] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806045#comment-17806045 ] Matthias Pohl commented on FLINK-33907: --- Yikes, thanks for letting me know. I will

[jira] [Commented] (FLINK-33907) Makes copying test jars being done later

2024-01-12 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806044#comment-17806044 ] Gyula Fora commented on FLINK-33907: [~mapohl] this change seems to break the local

[jira] [Commented] (FLINK-33998) Flink Job Manager restarted after kube-apiserver connection intermittent

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806039#comment-17806039 ] Matthias Pohl commented on FLINK-33998: --- the community provides bugfixes to the tw

[jira] [Commented] (FLINK-33998) Flink Job Manager restarted after kube-apiserver connection intermittent

2024-01-12 Thread Xiangyan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806037#comment-17806037 ] Xiangyan commented on FLINK-33998: -- Sure, which version do you suggest? [~mapohl]  > F

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450346151 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

[jira] [Resolved] (FLINK-33414) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl resolved FLINK-33414. --- Resolution: Fixed I'm closing this one again in favor of FLINK-34070. Because it's rather a

[jira] [Created] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34070: - Summary: MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler Key: FLINK-34070 URL: https://issues.apache.org/jira/browse/FLINK-34070

[jira] [Updated] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-34070: -- Description: We experience test failures of {{MiniClusterITCase.testHandleStreamingJobsWhenNo

Re: [PR] [FLINK-33365] include filters with Lookup joins [flink-connector-jdbc]

2024-01-12 Thread via GitHub
snuyanzin commented on PR #79: URL: https://github.com/apache/flink-connector-jdbc/pull/79#issuecomment-1888940790 @libenchao thanks for your comments/feedback >I'll try to come out with a solution with FieldNamedPreparedStatement in the coming days, but I cannot guarantee that I have en

Re: [PR] [FLINK-33915][ci] Adds nightly workflow for the master branch [flink]

2024-01-12 Thread via GitHub
zentol commented on code in PR #23971: URL: https://github.com/apache/flink/pull/23971#discussion_r1450284331 ## .github/actions/select_workflow_configs/action.yml: ## @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license a

Re: [PR] [FLINK-33915][ci] Adds nightly workflow for the master branch [flink]

2024-01-12 Thread via GitHub
zentol commented on code in PR #23971: URL: https://github.com/apache/flink/pull/23971#discussion_r1450284331 ## .github/actions/select_workflow_configs/action.yml: ## @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license a

Re: [PR] [FLINK-33365] include filters with Lookup joins [flink-connector-jdbc]

2024-01-12 Thread via GitHub
libenchao commented on PR #79: URL: https://github.com/apache/flink-connector-jdbc/pull/79#issuecomment-1888926645 > I am new to this area. I was looking to work within the existing design with minimal changes (as the design was added in a flip and works for scan queries. I think we have s

[jira] [Comment Edited] (FLINK-33414) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806005#comment-17806005 ] Matthias Pohl edited comment on FLINK-33414 at 1/12/24 11:24 AM: -

[jira] [Commented] (FLINK-33414) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806005#comment-17806005 ] Matthias Pohl commented on FLINK-33414: --- [~Jiang Xin] The issue only appears with

[jira] [Commented] (FLINK-33414) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails due to unexpected TimeoutException

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806004#comment-17806004 ] Matthias Pohl commented on FLINK-33414: --- [~Jiang Xin] The issue only appears with

[jira] [Commented] (FLINK-33783) Add options to ignore parsing error in Kafka SQL Connector

2024-01-12 Thread Bartosz Mikulski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806001#comment-17806001 ] Bartosz Mikulski commented on FLINK-33783: -- Sure, sounds great [~martijnvisser]

Re: [PR] [FLINK-32815] Implement internal HASHCODE function [flink]

2024-01-12 Thread via GitHub
twalthr commented on code in PR #24070: URL: https://github.com/apache/flink/pull/24070#discussion_r1450259392 ## flink-table/flink-table-common/src/main/java/org/apache/flink/table/functions/BuiltInFunctionDefinitions.java: ## @@ -2292,6 +2292,16 @@ ANY, and(logical(LogicalType

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450246875 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

Re: [PR] [FLINK-34063][runtime] Always flush compression buffers, when retrieving stream position during OperatorState snapshot. [flink]

2024-01-12 Thread via GitHub
isburmistrov commented on code in PR #24079: URL: https://github.com/apache/flink/pull/24079#discussion_r1450246875 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/CompressibleFSDataOutputStream.java: ## @@ -41,6 +41,10 @@ public CompressibleFSDataOutputStream(

Re: [PR] [FLINK-32815] Implement internal HASHCODE function [flink]

2024-01-12 Thread via GitHub
dawidwys commented on PR #24070: URL: https://github.com/apache/flink/pull/24070#issuecomment-175890 CI seems broken by https://github.com/apache/flink/pull/24022#issuecomment-111191 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [FLINK-33973] Add new interfaces for SinkV2 to synchronize the API with the SourceV2 API [flink]

2024-01-12 Thread via GitHub
flinkbot commented on PR #24080: URL: https://github.com/apache/flink/pull/24080#issuecomment-174136 ## CI report: * ee54a07048685165fb17e3c57ebcd968e959deac UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Commented] (FLINK-31472) AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805992#comment-17805992 ] Matthias Pohl commented on FLINK-31472: --- 1.18: https://dev.azure.com/apache-flink

[jira] [Updated] (FLINK-31472) AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-31472: -- Affects Version/s: 1.18.0 > AsyncSinkWriterThrottlingTest failed with Illegal mailbox thread >

[jira] [Commented] (FLINK-31724) SqlClientITCase.testMatchRecognize fails with "bash -c rm -rf /opt/flink/checkpoint/*" returned non-zero exit code 1

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805990#comment-17805990 ] Matthias Pohl commented on FLINK-31724: --- Happened again: https://dev.azure.com/ap

[jira] [Updated] (FLINK-31724) SqlClientITCase.testMatchRecognize fails with "bash -c rm -rf /opt/flink/checkpoint/*" returned non-zero exit code 1

2024-01-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-31724: -- Affects Version/s: 1.19.0 > SqlClientITCase.testMatchRecognize fails with "bash -c rm -rf > /

[jira] [Created] (FLINK-34069) build_wheels_on_macos times out and gets cancelled

2024-01-12 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34069: - Summary: build_wheels_on_macos times out and gets cancelled Key: FLINK-34069 URL: https://issues.apache.org/jira/browse/FLINK-34069 Project: Flink Issue Ty

[PR] [FLINK-33973] Add new interfaces for SinkV2 to synchronize the API with the SourceV2 API [flink]

2024-01-12 Thread via GitHub
pvary opened a new pull request, #24080: URL: https://github.com/apache/flink/pull/24080 ## What is the purpose of the change Add new SinkV2 API interfaces, set inheritance and deprecation for the old interfaces. Added fix for the concurrent commit ## Brief change log

  1   2   >