[jira] [Created] (FLINK-25770) Delete file is not correct in MergeTreeWriter
Jingsong Lee created FLINK-25770: Summary: Delete file is not correct in MergeTreeWriter Key: FLINK-25770 URL: https://issues.apache.org/jira/browse/FLINK-25770 Project: Flink Issue Type: Bug Components: Table Store Reporter: Jingsong Lee Assignee: Jingsong Lee Fix For: table-store-0.1.0 The deletion in MergeTreeWriter.updateCompactResult dose not consider upgrade case, the upgrade file is required by previous snapshot and following snapshot, we should ensure: 1. This file is not the output of upgraded. 2. This file is not the input of upgraded. Otherwise, the file will be deleted incorrectly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink-table-store] JingsongLi opened a new pull request #13: [FLINK-25770] Delete file is not correct in MergeTreeWriter
JingsongLi opened a new pull request #13: URL: https://github.com/apache/flink-table-store/pull/13 The deletion in MergeTreeWriter.updateCompactResult dose not consider upgrade case, the upgrade file is required by previous snapshot and following snapshot, we should ensure: 1. This file is not the output of upgraded. 2. This file is not the input of upgraded. Otherwise, the file will be deleted incorrectly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25770) Delete file is not correct in MergeTreeWriter
[ https://issues.apache.org/jira/browse/FLINK-25770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25770: --- Labels: pull-request-available (was: ) > Delete file is not correct in MergeTreeWriter > - > > Key: FLINK-25770 > URL: https://issues.apache.org/jira/browse/FLINK-25770 > Project: Flink > Issue Type: Bug > Components: Table Store >Reporter: Jingsong Lee >Assignee: Jingsong Lee >Priority: Major > Labels: pull-request-available > Fix For: table-store-0.1.0 > > > The deletion in MergeTreeWriter.updateCompactResult dose not consider upgrade > case, the upgrade file is required by previous snapshot and following > snapshot, we should ensure: > 1. This file is not the output of upgraded. > 2. This file is not the input of upgraded. > Otherwise, the file will be deleted incorrectly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] LadyForest commented on a change in pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
LadyForest commented on a change in pull request #18394:
URL: https://github.com/apache/flink/pull/18394#discussion_r789511549
##
File path:
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/utils/TableTestBase.scala
##
@@ -47,21 +53,30 @@ import org.apache.flink.table.expressions.Expression
import org.apache.flink.table.factories.{FactoryUtil, PlannerFactoryUtil,
StreamTableSourceFactory}
import org.apache.flink.table.functions._
import org.apache.flink.table.module.ModuleManager
-import org.apache.flink.table.operations.{ModifyOperation, Operation,
QueryOperation, SinkModifyOperation}
+import org.apache.flink.table.operations.ModifyOperation
+import org.apache.flink.table.operations.Operation
+import org.apache.flink.table.operations.QueryOperation
+import org.apache.flink.table.operations.SinkModifyOperation
Review comment:
I diffed the scala-checkstyle config file under the .idea and tools dir
and got the reason, it's so magic...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18388: [FLINK-25530][python][connector/pulsar] Support Pulsar source connector in Python DataStream API.
flinkbot edited a comment on pull request #18388: URL: https://github.com/apache/flink/pull/18388#issuecomment-1015205179 ## CI report: * 68b840fe2df44305be126b638e965bf8b3433b88 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29976) * d7c57c901a1d061766c39d1572885b6df3c6fc1e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29983) * 11799600e83d55b8150cdb93ad90d1a55fd2651f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29989) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) * c664a06bc46a706892eb58a86701a477f35d6b77 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wanglijie95 commented on a change in pull request #18130: [FLINK-25035][runtime] Shuffle service supports consuming subpartition range
wanglijie95 commented on a change in pull request #18130:
URL: https://github.com/apache/flink/pull/18130#discussion_r790491207
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGateTest.java
##
@@ -645,13 +616,13 @@ public void testRequestBackoffConfiguration() throws
Exception {
throw new RuntimeException(e);
}
});
-InputChannel localChannel = channelMap.get(partitionIds[0]);
+InputChannel localChannel = channelMap.get(new
SubpartitionInfo(partitionIds[0], 0));
Review comment:
fixed
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGateTest.java
##
@@ -645,13 +616,13 @@ public void testRequestBackoffConfiguration() throws
Exception {
throw new RuntimeException(e);
}
});
-InputChannel localChannel = channelMap.get(partitionIds[0]);
+InputChannel localChannel = channelMap.get(new
SubpartitionInfo(partitionIds[0], 0));
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] JingsongLi commented on pull request #18412: [FLINK-25696][datastream] Introduce MetadataPublisher interface to SinkWriter
JingsongLi commented on pull request #18412: URL: https://github.com/apache/flink/pull/18412#issuecomment-1019821611 > I think here we have two different thread-safety issues. One is that the consumer is invoked by the KafkaProducer that you probably need to fix and trigger the callback via the Mailbox. The other issue is around adding the subscribers concurrently. I don't think we need to put it inside the mailbox, it would be very performance intensive, it's a per record operation. A callback consumer, which I think has asynchronous processing reasonable. > I agree the Consumer needs to be serializable but if we pass an unmodifiable list to the sink writer it solves a lot of the thread-safety issues because it is not possible to add new subscribers after the sink translation. I guess if that works depends on your implementation that you have in mind to use the metadata. Yes, the problem is that this is a runtime statistic, which needs to expose information to an external caller, and it's hard to implement by a serializable class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18130: [FLINK-25035][runtime] Shuffle service supports consuming subpartition range
flinkbot edited a comment on pull request #18130: URL: https://github.com/apache/flink/pull/18130#issuecomment-995629902 ## CI report: * 41eeab31ee52a29d71acf018f9d1ef5a7750bb85 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29515) * a01ee8682f200b0332bccc8d96f2e25fe3a056f2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18246: [FLINK-25249][connector/kafka] Introduce KafkaExtension for all Kafka connector tests
flinkbot edited a comment on pull request #18246: URL: https://github.com/apache/flink/pull/18246#issuecomment-1002929103 ## CI report: * ad18f7d233edacae59cd55c0d56f09f25f2f1658 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28919) * 90ab7151072dd7e9b9e3cecdfab85cb8002f52b7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingsongLi commented on pull request #18359: [FLINK-25484][connectors/filesystem] Support inactivityInterval config in table api
JingsongLi commented on pull request #18359: URL: https://github.com/apache/flink/pull/18359#issuecomment-1019822883 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] PatrickRen commented on pull request #18246: [FLINK-25249][connector/kafka] Introduce KafkaExtension for all Kafka connector tests
PatrickRen commented on pull request #18246: URL: https://github.com/apache/flink/pull/18246#issuecomment-1019822936 @fapaul @JingGe Sorry for my late response! I made a new push just now. Hope this round is acceptable. Basically I changed the usage of `@Topic` annotation, and adopt advices from @JingGe. You can check the JavaDoc of new classes in the first commit to get the usage of the new infra. Please take a look when you are available. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) * a4acd8e2b6d14724243eddfdd22caa8bf11890e5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21608) YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at least one application..."
[
https://issues.apache.org/jira/browse/FLINK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480896#comment-17480896
]
Till Rohrmann commented on FLINK-21608:
---
Another instance:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29963&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=62110053-334f-5295-a0ab-80dd7e2babbf&l=28968
> YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at
> least one application..."
> ---
>
> Key: FLINK-21608
> URL: https://issues.apache.org/jira/browse/FLINK-21608
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Tests
>Affects Versions: 1.13.0
>Reporter: Dawid Wysakowicz
>Priority: Minor
> Labels: auto-deprioritized-major, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=14108&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=62110053-334f-5295-a0ab-80dd7e2babbf
> {code}
> 2021-03-04T10:45:47.3523930Z INFO: Binding
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:45:47.4240091Z Mar 04, 2021 10:45:47 AM
> com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
> getComponentProvider
> 2021-03-04T10:45:47.4241009Z INFO: Binding
> org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:47:53.6102867Z [ERROR] Tests run: 3, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 132.302 s <<< FAILURE! - in
> org.apache.flink.yarn.YARNHighAvailabilityITCase
> 2021-03-04T10:47:53.6103745Z [ERROR]
> testClusterClientRetrieval(org.apache.flink.yarn.YARNHighAvailabilityITCase)
> Time elapsed: 15.906 s <<< FAILURE!
> 2021-03-04T10:47:53.6104784Z java.lang.AssertionError: There is at least one
> application on the cluster that is not finished.[App
> application_1614854744820_0003 is in state RUNNING.]
> 2021-03-04T10:47:53.6106075Z at org.junit.Assert.fail(Assert.java:88)
> 2021-03-04T10:47:53.6108977Z at
> org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:322)
> 2021-03-04T10:47:53.6109784Z at
> org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:286)
> 2021-03-04T10:47:53.6110493Z at
> org.apache.flink.yarn.YARNHighAvailabilityITCase.testClusterClientRetrieval(YARNHighAvailabilityITCase.java:219)
> 2021-03-04T10:47:53.6111446Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-04T10:47:53.6111871Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-04T10:47:53.6112360Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-04T10:47:53.6112784Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-04T10:47:53.6113210Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-04T10:47:53.6114001Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-04T10:47:53.6114796Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-04T10:47:53.6115388Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-04T10:47:53.6116123Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6116995Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-04T10:47:53.6117810Z at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-03-04T10:47:53.6118621Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-04T10:47:53.6119311Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-04T10:47:53.6119840Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-04T10:47:53.6120279Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-04T10:47:53.6120739Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-04T10:47:53.6121173Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-04T10:47:53.6121692Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-03-04T10:47:53.6122128Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-03-04T10:47:53.6122594Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-03-04T10:47:53.6123005Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-03
[jira] [Updated] (FLINK-21608) YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at least one application..."
[
https://issues.apache.org/jira/browse/FLINK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-21608:
--
Affects Version/s: 1.15.0
> YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at
> least one application..."
> ---
>
> Key: FLINK-21608
> URL: https://issues.apache.org/jira/browse/FLINK-21608
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Tests
>Affects Versions: 1.13.0, 1.15.0
>Reporter: Dawid Wysakowicz
>Priority: Minor
> Labels: auto-deprioritized-major, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=14108&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=62110053-334f-5295-a0ab-80dd7e2babbf
> {code}
> 2021-03-04T10:45:47.3523930Z INFO: Binding
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:45:47.4240091Z Mar 04, 2021 10:45:47 AM
> com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
> getComponentProvider
> 2021-03-04T10:45:47.4241009Z INFO: Binding
> org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:47:53.6102867Z [ERROR] Tests run: 3, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 132.302 s <<< FAILURE! - in
> org.apache.flink.yarn.YARNHighAvailabilityITCase
> 2021-03-04T10:47:53.6103745Z [ERROR]
> testClusterClientRetrieval(org.apache.flink.yarn.YARNHighAvailabilityITCase)
> Time elapsed: 15.906 s <<< FAILURE!
> 2021-03-04T10:47:53.6104784Z java.lang.AssertionError: There is at least one
> application on the cluster that is not finished.[App
> application_1614854744820_0003 is in state RUNNING.]
> 2021-03-04T10:47:53.6106075Z at org.junit.Assert.fail(Assert.java:88)
> 2021-03-04T10:47:53.6108977Z at
> org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:322)
> 2021-03-04T10:47:53.6109784Z at
> org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:286)
> 2021-03-04T10:47:53.6110493Z at
> org.apache.flink.yarn.YARNHighAvailabilityITCase.testClusterClientRetrieval(YARNHighAvailabilityITCase.java:219)
> 2021-03-04T10:47:53.6111446Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-04T10:47:53.6111871Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-04T10:47:53.6112360Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-04T10:47:53.6112784Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-04T10:47:53.6113210Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-04T10:47:53.6114001Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-04T10:47:53.6114796Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-04T10:47:53.6115388Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-04T10:47:53.6116123Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6116995Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-04T10:47:53.6117810Z at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-03-04T10:47:53.6118621Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-04T10:47:53.6119311Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-04T10:47:53.6119840Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-04T10:47:53.6120279Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-04T10:47:53.6120739Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-04T10:47:53.6121173Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-04T10:47:53.6121692Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-03-04T10:47:53.6122128Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-03-04T10:47:53.6122594Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-03-04T10:47:53.6123005Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-03-04T10:47:53.6123432Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6123978Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.
[jira] [Updated] (FLINK-21608) YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at least one application..."
[
https://issues.apache.org/jira/browse/FLINK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-21608:
--
Labels: test-stability (was: auto-deprioritized-major test-stability)
> YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at
> least one application..."
> ---
>
> Key: FLINK-21608
> URL: https://issues.apache.org/jira/browse/FLINK-21608
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Tests
>Affects Versions: 1.13.0, 1.15.0
>Reporter: Dawid Wysakowicz
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=14108&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=62110053-334f-5295-a0ab-80dd7e2babbf
> {code}
> 2021-03-04T10:45:47.3523930Z INFO: Binding
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:45:47.4240091Z Mar 04, 2021 10:45:47 AM
> com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
> getComponentProvider
> 2021-03-04T10:45:47.4241009Z INFO: Binding
> org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:47:53.6102867Z [ERROR] Tests run: 3, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 132.302 s <<< FAILURE! - in
> org.apache.flink.yarn.YARNHighAvailabilityITCase
> 2021-03-04T10:47:53.6103745Z [ERROR]
> testClusterClientRetrieval(org.apache.flink.yarn.YARNHighAvailabilityITCase)
> Time elapsed: 15.906 s <<< FAILURE!
> 2021-03-04T10:47:53.6104784Z java.lang.AssertionError: There is at least one
> application on the cluster that is not finished.[App
> application_1614854744820_0003 is in state RUNNING.]
> 2021-03-04T10:47:53.6106075Z at org.junit.Assert.fail(Assert.java:88)
> 2021-03-04T10:47:53.6108977Z at
> org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:322)
> 2021-03-04T10:47:53.6109784Z at
> org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:286)
> 2021-03-04T10:47:53.6110493Z at
> org.apache.flink.yarn.YARNHighAvailabilityITCase.testClusterClientRetrieval(YARNHighAvailabilityITCase.java:219)
> 2021-03-04T10:47:53.6111446Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-04T10:47:53.6111871Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-04T10:47:53.6112360Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-04T10:47:53.6112784Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-04T10:47:53.6113210Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-04T10:47:53.6114001Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-04T10:47:53.6114796Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-04T10:47:53.6115388Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-04T10:47:53.6116123Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6116995Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-04T10:47:53.6117810Z at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-03-04T10:47:53.6118621Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-04T10:47:53.6119311Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-04T10:47:53.6119840Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-04T10:47:53.6120279Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-04T10:47:53.6120739Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-04T10:47:53.6121173Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-04T10:47:53.6121692Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-03-04T10:47:53.6122128Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-03-04T10:47:53.6122594Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-03-04T10:47:53.6123005Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-03-04T10:47:53.6123432Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6123978Z at
> org.junit.internal.runners.statements.RunAfte
[jira] [Updated] (FLINK-21608) YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at least one application..."
[
https://issues.apache.org/jira/browse/FLINK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-21608:
--
Priority: Critical (was: Minor)
> YARNHighAvailabilityITCase.testClusterClientRetrieval fails with "There is at
> least one application..."
> ---
>
> Key: FLINK-21608
> URL: https://issues.apache.org/jira/browse/FLINK-21608
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Tests
>Affects Versions: 1.13.0, 1.15.0
>Reporter: Dawid Wysakowicz
>Priority: Critical
> Labels: auto-deprioritized-major, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=14108&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=62110053-334f-5295-a0ab-80dd7e2babbf
> {code}
> 2021-03-04T10:45:47.3523930Z INFO: Binding
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:45:47.4240091Z Mar 04, 2021 10:45:47 AM
> com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
> getComponentProvider
> 2021-03-04T10:45:47.4241009Z INFO: Binding
> org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to
> GuiceManagedComponentProvider with the scope "Singleton"
> 2021-03-04T10:47:53.6102867Z [ERROR] Tests run: 3, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 132.302 s <<< FAILURE! - in
> org.apache.flink.yarn.YARNHighAvailabilityITCase
> 2021-03-04T10:47:53.6103745Z [ERROR]
> testClusterClientRetrieval(org.apache.flink.yarn.YARNHighAvailabilityITCase)
> Time elapsed: 15.906 s <<< FAILURE!
> 2021-03-04T10:47:53.6104784Z java.lang.AssertionError: There is at least one
> application on the cluster that is not finished.[App
> application_1614854744820_0003 is in state RUNNING.]
> 2021-03-04T10:47:53.6106075Z at org.junit.Assert.fail(Assert.java:88)
> 2021-03-04T10:47:53.6108977Z at
> org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:322)
> 2021-03-04T10:47:53.6109784Z at
> org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:286)
> 2021-03-04T10:47:53.6110493Z at
> org.apache.flink.yarn.YARNHighAvailabilityITCase.testClusterClientRetrieval(YARNHighAvailabilityITCase.java:219)
> 2021-03-04T10:47:53.6111446Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-04T10:47:53.6111871Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-04T10:47:53.6112360Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-04T10:47:53.6112784Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-04T10:47:53.6113210Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-04T10:47:53.6114001Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-04T10:47:53.6114796Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-04T10:47:53.6115388Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-04T10:47:53.6116123Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6116995Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-04T10:47:53.6117810Z at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-03-04T10:47:53.6118621Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-04T10:47:53.6119311Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-04T10:47:53.6119840Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-04T10:47:53.6120279Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-04T10:47:53.6120739Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-04T10:47:53.6121173Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-04T10:47:53.6121692Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-03-04T10:47:53.6122128Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-03-04T10:47:53.6122594Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-03-04T10:47:53.6123005Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-03-04T10:47:53.6123432Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-04T10:47:53.6123978Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(
[GitHub] [flink] flinkbot edited a comment on pull request #18130: [FLINK-25035][runtime] Shuffle service supports consuming subpartition range
flinkbot edited a comment on pull request #18130: URL: https://github.com/apache/flink/pull/18130#issuecomment-995629902 ## CI report: * 41eeab31ee52a29d71acf018f9d1ef5a7750bb85 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29515) * a01ee8682f200b0332bccc8d96f2e25fe3a056f2 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18246: [FLINK-25249][connector/kafka] Introduce KafkaExtension for all Kafka connector tests
flinkbot edited a comment on pull request #18246: URL: https://github.com/apache/flink/pull/18246#issuecomment-1002929103 ## CI report: * ad18f7d233edacae59cd55c0d56f09f25f2f1658 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28919) * 90ab7151072dd7e9b9e3cecdfab85cb8002f52b7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=3) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-18356) Exit code 137 returned from process
[
https://issues.apache.org/jira/browse/FLINK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480899#comment-17480899
]
Till Rohrmann commented on FLINK-18356:
---
Another instance
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29964&view=logs&j=a9db68b9-a7e0-54b6-0f98-010e0aff39e2&t=cdd32e0b-6047-565b-c58f-14054472f1be&l=9994.
The table tests causing the exit code 137.
> Exit code 137 returned from process
> ---
>
> Key: FLINK-18356
> URL: https://issues.apache.org/jira/browse/FLINK-18356
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Tests
>Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0
>Reporter: Piotr Nowojski
>Assignee: Dawid Wysakowicz
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> {noformat}
> = test session starts
> ==
> platform linux -- Python 3.7.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
> cachedir: .tox/py37-cython/.pytest_cache
> rootdir: /__w/3/s/flink-python
> collected 568 items
> pyflink/common/tests/test_configuration.py ..[
> 1%]
> pyflink/common/tests/test_execution_config.py ...[
> 5%]
> pyflink/dataset/tests/test_execution_environment.py .
> ##[error]Exit code 137 returned from process: file name '/bin/docker',
> arguments 'exec -i -u 1002
> 97fc4e22522d2ced1f4d23096b8929045d083dd0a99a4233a8b20d0489e9bddb
> /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'.
> Finishing: Test - python
> {noformat}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=3729&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18359: [FLINK-25484][connectors/filesystem] Support inactivityInterval config in table api
flinkbot edited a comment on pull request #18359: URL: https://github.com/apache/flink/pull/18359#issuecomment-1012981632 ## CI report: * 82871d3416aefbce86d6e32e3e26a51c12841534 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29768) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) * b9b29ad0b3697de70c570ee08d4ccf45f1c3e0bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-25762) Native k8s- User defined system properties get overriden when cluster is deployed
[
https://issues.apache.org/jira/browse/FLINK-25762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Wang reassigned FLINK-25762:
-
Assignee: Tamir Sagi
> Native k8s- User defined system properties get overriden when cluster is
> deployed
> --
>
> Key: FLINK-25762
> URL: https://issues.apache.org/jira/browse/FLINK-25762
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
>Affects Versions: 1.13.0, 1.14.0
>Reporter: Tamir Sagi
>Assignee: Tamir Sagi
>Priority: Major
> Labels: Flink, flink-k8s, kubernetes, native-k8s,
> native-kubernetes, pull-request-available
> Attachments: log.png, stacktrace.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Running Flink 1.14.2 in native k8s mode, User defined system properties get
> ignores. i.e, -Dlog4j.configurationFile, it falls back to
> /opt/flink/conf/log4j-console.properties.
> That happens due to the order of exec command provided in flink-console.sh
> file.
> [https://github.com/apache/flink/blob/release-1.14.2/flink-dist/src/main/flink-bin/bin/flink-console.sh#L114]
> exec "$JAVA_RUN" $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "$\{log_setting[@]}"
> -classpath "`manglePathList
> "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" ${CLASS_TO_RUN}
> "$\{ARGS[@]}"
> Apart from logging, All user defined properties(FLINK_ENV_JAVA_OPTS) might
> get overriden by other args.
> Discussion
> https://lists.apache.org/thread/b24g1nd00q5pln5h9w2mh1s3ocxwb61b
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) * c664a06bc46a706892eb58a86701a477f35d6b77 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-19615) HBaseConnectorITCase.testTableSink is unstable
[
https://issues.apache.org/jira/browse/FLINK-19615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser closed FLINK-19615.
--
Resolution: Cannot Reproduce
No issues reported since November, therefore closing the ticket
> HBaseConnectorITCase.testTableSink is unstable
> --
>
> Key: FLINK-19615
> URL: https://issues.apache.org/jira/browse/FLINK-19615
> Project: Flink
> Issue Type: Bug
> Components: Connectors / HBase
>Affects Versions: 1.12.0, 1.14.0
>Reporter: Dian Fu
>Priority: Major
> Labels: test-stability
> Fix For: 1.15.0
>
>
> This is a failed instance from the test of
> [PR|https://github.com/apache/flink/pull/13507], however, I'm 99% sure that
> the PR doesn't touch anything which could cause this failure:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=7494&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=03dca39c-73e8-5aaf-601d-328ae5c35f20
> {code}
> 020-10-13T07:47:41.3436206Z testTableSink[planner = BLINK_PLANNER, legacy =
> false](org.apache.flink.connector.hbase2.HBaseConnectorITCase) Time elapsed:
> 3.805 sec <<< FAILURE!
> 2020-10-13T07:47:41.3436780Z org.junit.ComparisonFailure:
> 2020-10-13T07:47:41.3437071Z Different elements in arrays: expected 8
> elements and received 8
> 2020-10-13T07:47:41.3438174Z expected: [1,10,Hello-1,100,1.01,false,Welt-1,
> 2,20,Hello-2,200,2.02,true,Welt-2, 3,30,Hello-3,300,3.03,false,Welt-3,
> 4,40,null,400,4.04,true,Welt-4, 5,50,Hello-5,500,5.05,false,Welt-5,
> 6,60,Hello-6,600,6.06,true,Welt-6, 7,70,Hello-7,700,7.07,false,Welt-7,
> 8,80,null,800,8.08,true,Welt-8]
> 2020-10-13T07:47:41.3439849Z received: [1,10,Hello-1,100,1.01,false,Welt-1,
> 2,20,Hello-2,200,2.02,true,Welt-2, 3,30,Hello-3,300,3.03,false,Welt-3,
> 4,40,,400,4.04,true,Welt-4, 5,50,Hello-5,500,5.05,false,Welt-5,
> 6,60,Hello-6,600,6.06,true,Welt-6, 7,70,Hello-7,700,7.07,false,Welt-7,
> 8,80,,800,8.08,true,Welt-8] expected:<4,40,[null],400,4.04,true,Welt-...> but
> was:<4,40,[],400,4.04,true,Welt-...>
> 2020-10-13T07:47:41.3440777Z at
> org.junit.Assert.assertEquals(Assert.java:115)
> 2020-10-13T07:47:41.3441191Z at
> org.apache.flink.test.util.TestBaseUtils.compareResult(TestBaseUtils.java:400)
> 2020-10-13T07:47:41.3441656Z at
> org.apache.flink.test.util.TestBaseUtils.compareResultAsText(TestBaseUtils.java:347)
> 2020-10-13T07:47:41.3442153Z at
> org.apache.flink.connector.hbase2.HBaseConnectorITCase.testTableSink(HBaseConnectorITCase.java:386)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (FLINK-19615) HBaseConnectorITCase.testTableSink is unstable
[
https://issues.apache.org/jira/browse/FLINK-19615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser reassigned FLINK-19615:
--
Assignee: (was: Marios Trivyzas)
> HBaseConnectorITCase.testTableSink is unstable
> --
>
> Key: FLINK-19615
> URL: https://issues.apache.org/jira/browse/FLINK-19615
> Project: Flink
> Issue Type: Bug
> Components: Connectors / HBase
>Affects Versions: 1.12.0, 1.14.0
>Reporter: Dian Fu
>Priority: Major
> Labels: test-stability
> Fix For: 1.15.0
>
>
> This is a failed instance from the test of
> [PR|https://github.com/apache/flink/pull/13507], however, I'm 99% sure that
> the PR doesn't touch anything which could cause this failure:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=7494&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=03dca39c-73e8-5aaf-601d-328ae5c35f20
> {code}
> 020-10-13T07:47:41.3436206Z testTableSink[planner = BLINK_PLANNER, legacy =
> false](org.apache.flink.connector.hbase2.HBaseConnectorITCase) Time elapsed:
> 3.805 sec <<< FAILURE!
> 2020-10-13T07:47:41.3436780Z org.junit.ComparisonFailure:
> 2020-10-13T07:47:41.3437071Z Different elements in arrays: expected 8
> elements and received 8
> 2020-10-13T07:47:41.3438174Z expected: [1,10,Hello-1,100,1.01,false,Welt-1,
> 2,20,Hello-2,200,2.02,true,Welt-2, 3,30,Hello-3,300,3.03,false,Welt-3,
> 4,40,null,400,4.04,true,Welt-4, 5,50,Hello-5,500,5.05,false,Welt-5,
> 6,60,Hello-6,600,6.06,true,Welt-6, 7,70,Hello-7,700,7.07,false,Welt-7,
> 8,80,null,800,8.08,true,Welt-8]
> 2020-10-13T07:47:41.3439849Z received: [1,10,Hello-1,100,1.01,false,Welt-1,
> 2,20,Hello-2,200,2.02,true,Welt-2, 3,30,Hello-3,300,3.03,false,Welt-3,
> 4,40,,400,4.04,true,Welt-4, 5,50,Hello-5,500,5.05,false,Welt-5,
> 6,60,Hello-6,600,6.06,true,Welt-6, 7,70,Hello-7,700,7.07,false,Welt-7,
> 8,80,,800,8.08,true,Welt-8] expected:<4,40,[null],400,4.04,true,Welt-...> but
> was:<4,40,[],400,4.04,true,Welt-...>
> 2020-10-13T07:47:41.3440777Z at
> org.junit.Assert.assertEquals(Assert.java:115)
> 2020-10-13T07:47:41.3441191Z at
> org.apache.flink.test.util.TestBaseUtils.compareResult(TestBaseUtils.java:400)
> 2020-10-13T07:47:41.3441656Z at
> org.apache.flink.test.util.TestBaseUtils.compareResultAsText(TestBaseUtils.java:347)
> 2020-10-13T07:47:41.3442153Z at
> org.apache.flink.connector.hbase2.HBaseConnectorITCase.testTableSink(HBaseConnectorITCase.java:386)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25200) Implement duplicating for s3 filesystem
[ https://issues.apache.org/jira/browse/FLINK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480904#comment-17480904 ] Yun Tang commented on FLINK-25200: -- [~pnowojski] From my knowledge, S3 will involve copy on server side if calling #copyObject, which cannot be really fast. And OSS would use [shallow copy|https://issues.apache.org/jira/browse/HADOOP-15323] for normal foramt file under 32MB or multipat format file, which could finish the copying within hundreds of milliseconds (can achive 10 millisecond at least), which deserves to be implemented first. > Implement duplicating for s3 filesystem > --- > > Key: FLINK-25200 > URL: https://issues.apache.org/jira/browse/FLINK-25200 > Project: Flink > Issue Type: Sub-task > Components: FileSystems >Reporter: Dawid Wysakowicz >Priority: Major > Fix For: 1.15.0 > > > We can use https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] xintongsong commented on a change in pull request #18430: [FLINK-25577][docs] Update GCS documentation
xintongsong commented on a change in pull request #18430:
URL: https://github.com/apache/flink/pull/18430#discussion_r790487706
##
File path: docs/content/docs/deployment/filesystems/gcs.md
##
@@ -50,57 +48,40 @@
env.getCheckpointConfig().setCheckpointStorage("gs:///");
```
-### Libraries
+Note that these examples are *not* exhaustive and you can use GCS in other
places as well, including your [high availability setup]({{< ref
"docs/deployment/ha/overview" >}}) or the [EmbeddedRocksDBStateBackend]({{< ref
"docs/ops/state/state_backends" >}}#the-rocksdbstatebackend); everywhere that
Flink expects a FileSystem URI.
-You must include the following jars in Flink's `lib` directory to connect
Flink with gcs:
+### GCS File System plugin
-```xml
-
- org.apache.flink
- flink-shaded-hadoop2-uber
- ${flink.shared_hadoop_latest_version}
-
+Flink provides the `flink-gs-fs-hadoop` file system to write to GCS.
+This implementation is self-contained with no dependency footprint, so there
is no need to add Hadoop to the classpath to use it.
-
- com.google.cloud.bigdataoss
- gcs-connector
- hadoop2-2.2.0
-
-```
+`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the
*gs://* scheme. It uses Google's
[gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector)
Hadoop library to access GCS. It also uses Google's
[google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage)
library to provide `RecoverableWriter` support.
-We have tested with `flink-shared-hadoop2-uber` version >= `2.8.5-1.8.3`.
-You can track the latest version of the [gcs-connector hadoop
2](https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop2.jar).
+This file system supports the [StreamingFileSink]({{< ref
"docs/connectors/datastream/streamfile_sink" >}}) and the [FileSink]({{< ref
"docs/connectors/datastream/file_sink" >}}).
Review comment:
These two references are invalid, as CI complained. They are removed /
renamed in FLINK-20188. I think we should now refers to
"docs/content/docs/connectors/datastream/filesystem.md"
##
File path: docs/content/docs/deployment/filesystems/gcs.md
##
@@ -50,57 +48,40 @@
env.getCheckpointConfig().setCheckpointStorage("gs:///");
```
-### Libraries
+Note that these examples are *not* exhaustive and you can use GCS in other
places as well, including your [high availability setup]({{< ref
"docs/deployment/ha/overview" >}}) or the [EmbeddedRocksDBStateBackend]({{< ref
"docs/ops/state/state_backends" >}}#the-rocksdbstatebackend); everywhere that
Flink expects a FileSystem URI.
-You must include the following jars in Flink's `lib` directory to connect
Flink with gcs:
+### GCS File System plugin
-```xml
-
- org.apache.flink
- flink-shaded-hadoop2-uber
- ${flink.shared_hadoop_latest_version}
-
+Flink provides the `flink-gs-fs-hadoop` file system to write to GCS.
+This implementation is self-contained with no dependency footprint, so there
is no need to add Hadoop to the classpath to use it.
-
- com.google.cloud.bigdataoss
- gcs-connector
- hadoop2-2.2.0
-
-```
+`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the
*gs://* scheme. It uses Google's
[gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector)
Hadoop library to access GCS. It also uses Google's
[google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage)
library to provide `RecoverableWriter` support.
-We have tested with `flink-shared-hadoop2-uber` version >= `2.8.5-1.8.3`.
-You can track the latest version of the [gcs-connector hadoop
2](https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop2.jar).
+This file system supports the [StreamingFileSink]({{< ref
"docs/connectors/datastream/streamfile_sink" >}}) and the [FileSink]({{< ref
"docs/connectors/datastream/file_sink" >}}).
-### Authentication to access GCS
+To use `flink-gs-fs-hadoop`, copy the JAR file from the `opt` directory to the
`plugins` directory of your Flink distribution before starting Flink, i.e.
-Most operations on GCS require authentication. Please see [the documentation
on Google Cloud Storage
authentication](https://cloud.google.com/storage/docs/authentication) for more
information.
+```bash
+mkdir ./plugins/gs-fs-hadoop
+cp ./opt/flink-gs-fs-hadoop-{{< version >}}.jar ./plugins/gs-fs-hadoop/
+```
+
+### Configuration
-You can use the following method for authentication
-* Configure via core-site.xml
- You would need to add the following properties to `core-site.xml`
+The underlying Hadoop file system can be [configured using Hadoop's gs
configuration
keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/master/gcs/CONFIGURATION.md)
by adding the configurations to your `flink-conf.yaml`.
Review comment:
Correct me if I'm wrong, I think the p
[GitHub] [flink] beyond1920 commented on pull request #18414: [hotfix][docs]fix flink sql Cascading Window TVF Aggregation exception
beyond1920 commented on pull request #18414: URL: https://github.com/apache/flink/pull/18414#issuecomment-1019837686 Merged. Thanks for contribution @simenliuxing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18359: [FLINK-25484][connectors/filesystem] Support inactivityInterval config in table api
flinkbot edited a comment on pull request #18359: URL: https://github.com/apache/flink/pull/18359#issuecomment-1012981632 ## CI report: * 82871d3416aefbce86d6e32e3e26a51c12841534 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29768) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) * a4acd8e2b6d14724243eddfdd22caa8bf11890e5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) * 0becff6fff82bb6d816b2d02f366a8de81bf8f24 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] beyond1920 merged pull request #18414: [hotfix][docs]fix flink sql Cascading Window TVF Aggregation exception
beyond1920 merged pull request #18414: URL: https://github.com/apache/flink/pull/18414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18456: [FLINK-21790][network] Shuffle data directories to make directory selection of different TaskManagers fairer
flinkbot edited a comment on pull request #18456: URL: https://github.com/apache/flink/pull/18456#issuecomment-1019691654 ## CI report: * cd74a88227edf6a00098a9d651634e009ac3fe9f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29977) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] paul8263 opened a new pull request #18460: [FLINK-25767][doc] Totally translated state.md into Chinese
paul8263 opened a new pull request #18460: URL: https://github.com/apache/flink/pull/18460 ## What is the purpose of the change Totally translated state.md into Chinese ## Brief change log - docs/content.zh/docs/dev/datastream/fault-tolerance/state.md ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25767) Translation of page 'Working with State' is incomplete
[ https://issues.apache.org/jira/browse/FLINK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25767: --- Labels: pull-request-available (was: ) > Translation of page 'Working with State' is incomplete > -- > > Key: FLINK-25767 > URL: https://issues.apache.org/jira/browse/FLINK-25767 > Project: Flink > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.14.3 >Reporter: Yao Zhang >Assignee: Yao Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The translation of page [Working with State | Apache > Flink|https://nightlies.apache.org/flink/flink-docs-release-1.14/zh/docs/dev/datastream/fault-tolerance/state/] > is incomplete. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25771) CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480906#comment-17480906
]
Till Rohrmann commented on FLINK-25771:
---
cc [~echauchot].
> CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
> --
>
> Key: FLINK-25771
> URL: https://issues.apache.org/jira/browse/FLINK-25771
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test {{CassandraConnectorITCase.testRetrialAndDropTables}} fails on AZP
> with
> {code}
> Jan 23 01:02:52 com.datastax.driver.core.exceptions.NoHostAvailableException:
> All host(s) tried for query failed (tried: /172.17.0.1:59220
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [/172.17.0.1] Timed out waiting for server response))
> Jan 23 01:02:52 at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> Jan 23 01:02:52 at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
> Jan 23 01:02:52 at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> Jan 23 01:02:52 at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> Jan 23 01:02:52 at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63)
> Jan 23 01:02:52 at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39)
> Jan 23 01:02:52 at
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testRetrialAndDropTables(CassandraConnectorITCase.java:554)
> Jan 23 01:02:52 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jan 23 01:02:52 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jan 23 01:02:52 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jan 23 01:02:52 at java.lang.reflect.Method.invoke(Method.java:498)
> Jan 23 01:02:52 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Jan 23 01:02:52 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jan 23 01:02:52 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Jan 23 01:02:52 at
> org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:196)
> Jan 23 01:02:52 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Jan 23 01:02:52 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Jan 23 01:02:52 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Jan 23 01:02:52 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Jan 23 01:02:52 at
> org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
> Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:128)
> Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:27)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.
[jira] [Created] (FLINK-25771) CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
Till Rohrmann created FLINK-25771:
-
Summary: CassandraConnectorITCase.testRetrialAndDropTables fails
on AZP
Key: FLINK-25771
URL: https://issues.apache.org/jira/browse/FLINK-25771
Project: Flink
Issue Type: Bug
Components: Connectors / Cassandra
Affects Versions: 1.15.0
Reporter: Till Rohrmann
The test {{CassandraConnectorITCase.testRetrialAndDropTables}} fails on AZP with
{code}
Jan 23 01:02:52 com.datastax.driver.core.exceptions.NoHostAvailableException:
All host(s) tried for query failed (tried: /172.17.0.1:59220
(com.datastax.driver.core.exceptions.OperationTimedOutException: [/172.17.0.1]
Timed out waiting for server response))
Jan 23 01:02:52 at
com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
Jan 23 01:02:52 at
com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
Jan 23 01:02:52 at
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
Jan 23 01:02:52 at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
Jan 23 01:02:52 at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63)
Jan 23 01:02:52 at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39)
Jan 23 01:02:52 at
org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testRetrialAndDropTables(CassandraConnectorITCase.java:554)
Jan 23 01:02:52 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
Jan 23 01:02:52 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jan 23 01:02:52 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jan 23 01:02:52 at java.lang.reflect.Method.invoke(Method.java:498)
Jan 23 01:02:52 at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
Jan 23 01:02:52 at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Jan 23 01:02:52 at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
Jan 23 01:02:52 at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
Jan 23 01:02:52 at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
Jan 23 01:02:52 at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
Jan 23 01:02:52 at
org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:196)
Jan 23 01:02:52 at
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
Jan 23 01:02:52 at
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
Jan 23 01:02:52 at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
Jan 23 01:02:52 at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
Jan 23 01:02:52 at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
Jan 23 01:02:52 at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
Jan 23 01:02:52 at
org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.run(ParentRunner.java:363)
Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:128)
Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:27)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
Jan 23 01:02:52 at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
Jan 23 01:02:52
[jira] [Commented] (FLINK-23047) CassandraConnectorITCase.testCassandraBatchTupleFormat fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480909#comment-17480909
]
Yun Gao commented on FLINK-23047:
-
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29945&view=logs&j=c91190b6-40ae-57b2-5999-31b869b0a7c1&t=43529380-51b4-5e90-5af4-2dccec0ef402&l=13601]
Hi [~echauchot] ~ there seems one fail case on January 23th on release-1.13,
could you have a double look~? Very thanks!
> CassandraConnectorITCase.testCassandraBatchTupleFormat fails on azure
> -
>
> Key: FLINK-23047
> URL: https://issues.apache.org/jira/browse/FLINK-23047
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
>Affects Versions: 1.14.0, 1.12.4, 1.13.2, 1.15.0
>Reporter: Xintong Song
>Assignee: Etienne Chauchot
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0, 1.13.6, 1.14.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19176&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=03dca39c-73e8-5aaf-601d-328ae5c35f20&l=13995
> {code}
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 157.28 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase
> [ERROR]
> testCassandraBatchTupleFormat(org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase)
> Time elapsed: 12.052 s <<< ERROR!
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: /127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException: [/127.0.0.1]
> Timed out waiting for server response))
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63)
> at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39)
> at
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.createTable(CassandraConnectorITCase.java:234)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.
[jira] [Commented] (FLINK-25683) wrong result if table transfrom to DataStream then window process in batch mode
[ https://issues.apache.org/jira/browse/FLINK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480910#comment-17480910 ] Dawid Wysakowicz commented on FLINK-25683: -- Fixed in: * master ** 9aa879f50c32de862831e82613f1cf1bc4d760f9 * 1.14.4 ** 2cca3a6609a030b6f03fedbaab51da3dce2962cd > wrong result if table transfrom to DataStream then window process in batch > mode > --- > > Key: FLINK-25683 > URL: https://issues.apache.org/jira/browse/FLINK-25683 > Project: Flink > Issue Type: Bug > Components: Table SQL / API, Table SQL / Runtime >Affects Versions: 1.14.2 > Environment: mac book pro m1 > jdk 8 > scala 2.11 > flink 1.14.2 > idea 2020 >Reporter: zhangzh >Assignee: Yao Zhang >Priority: Major > Labels: pull-request-available > Attachments: TableToDataStreamBatchWindowTest.scala, pom.xml > > > I have 5 line datas, > i first need to transform current data with SQL > then mix current data and historical data which is batch get from hbase > for some special reason the program must run in batch mode > i think the correct result should be like this： > (BOB,1) > (EMA,1) > (DOUG,1) > (ALICE,1) > (CENDI,1) > but the result is : > (EMA,1) > > if i set different parallelism ,the result is different. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25683) wrong result if table transfrom to DataStream then window process in batch mode
[ https://issues.apache.org/jira/browse/FLINK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Wysakowicz updated FLINK-25683: - Fix Version/s: 1.15.0 1.14.4 > wrong result if table transfrom to DataStream then window process in batch > mode > --- > > Key: FLINK-25683 > URL: https://issues.apache.org/jira/browse/FLINK-25683 > Project: Flink > Issue Type: Bug > Components: Table SQL / API, Table SQL / Runtime >Affects Versions: 1.14.2 > Environment: mac book pro m1 > jdk 8 > scala 2.11 > flink 1.14.2 > idea 2020 >Reporter: zhangzh >Assignee: Yao Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0, 1.14.4 > > Attachments: TableToDataStreamBatchWindowTest.scala, pom.xml > > > I have 5 line datas, > i first need to transform current data with SQL > then mix current data and historical data which is batch get from hbase > for some special reason the program must run in batch mode > i think the correct result should be like this： > (BOB,1) > (EMA,1) > (DOUG,1) > (ALICE,1) > (CENDI,1) > but the result is : > (EMA,1) > > if i set different parallelism ,the result is different. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot commented on pull request #18460: [FLINK-25767][doc] Totally translated state.md into Chinese
flinkbot commented on pull request #18460: URL: https://github.com/apache/flink/pull/18460#issuecomment-1019842362 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6385885a06a970cc31f450251e9f6b0d4150a0dd (Mon Jan 24 08:33:17 UTC 2022) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] (FLINK-23047) CassandraConnectorITCase.testCassandraBatchTupleFormat fails on azure
[ https://issues.apache.org/jira/browse/FLINK-23047 ]
Yun Gao deleted comment on FLINK-23047:
-
was (Author: gaoyunhaii):
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29945&view=logs&j=c91190b6-40ae-57b2-5999-31b869b0a7c1&t=43529380-51b4-5e90-5af4-2dccec0ef402&l=13601]
Hi [~echauchot] ~ there seems one fail case on January 23th on release-1.13,
could you have a double look~? Very thanks!
> CassandraConnectorITCase.testCassandraBatchTupleFormat fails on azure
> -
>
> Key: FLINK-23047
> URL: https://issues.apache.org/jira/browse/FLINK-23047
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
>Affects Versions: 1.14.0, 1.12.4, 1.13.2, 1.15.0
>Reporter: Xintong Song
>Assignee: Etienne Chauchot
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0, 1.13.6, 1.14.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19176&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=03dca39c-73e8-5aaf-601d-328ae5c35f20&l=13995
> {code}
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 157.28 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase
> [ERROR]
> testCassandraBatchTupleFormat(org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase)
> Time elapsed: 12.052 s <<< ERROR!
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: /127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException: [/127.0.0.1]
> Timed out waiting for server response))
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63)
> at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39)
> at
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.createTable(CassandraConnectorITCase.java:234)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.j
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) * a638e63713a9453e027d5c45aa6db05a9bec9230 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) * c664a06bc46a706892eb58a86701a477f35d6b77 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25771) CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25771:
Affects Version/s: 1.13.5
(was: 1.15.0)
> CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
> --
>
> Key: FLINK-25771
> URL: https://issues.apache.org/jira/browse/FLINK-25771
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
>Affects Versions: 1.13.5
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test {{CassandraConnectorITCase.testRetrialAndDropTables}} fails on AZP
> with
> {code}
> Jan 23 01:02:52 com.datastax.driver.core.exceptions.NoHostAvailableException:
> All host(s) tried for query failed (tried: /172.17.0.1:59220
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [/172.17.0.1] Timed out waiting for server response))
> Jan 23 01:02:52 at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> Jan 23 01:02:52 at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
> Jan 23 01:02:52 at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> Jan 23 01:02:52 at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
> Jan 23 01:02:52 at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63)
> Jan 23 01:02:52 at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39)
> Jan 23 01:02:52 at
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testRetrialAndDropTables(CassandraConnectorITCase.java:554)
> Jan 23 01:02:52 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jan 23 01:02:52 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jan 23 01:02:52 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jan 23 01:02:52 at java.lang.reflect.Method.invoke(Method.java:498)
> Jan 23 01:02:52 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Jan 23 01:02:52 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jan 23 01:02:52 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Jan 23 01:02:52 at
> org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:196)
> Jan 23 01:02:52 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Jan 23 01:02:52 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Jan 23 01:02:52 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Jan 23 01:02:52 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> Jan 23 01:02:52 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Jan 23 01:02:52 at
> org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
> Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:128)
> Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:27)
> Jan 23 01:02:52 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Jan
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) * b9b29ad0b3697de70c570ee08d4ccf45f1c3e0bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18460: [FLINK-25767][doc] Totally translated state.md into Chinese
flinkbot commented on pull request #18460: URL: https://github.com/apache/flink/pull/18460#issuecomment-1019844798 ## CI report: * 6385885a06a970cc31f450251e9f6b0d4150a0dd UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25673) AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond hangs on AZP
[
https://issues.apache.org/jira/browse/FLINK-25673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480912#comment-17480912
]
Till Rohrmann commented on FLINK-25673:
---
Another instance:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29946&view=logs&j=a57e0635-3fad-5b08-57c7-a4142d7d6fa9&t=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7&l=28462
> AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond
> hangs on AZP
> -
>
> Key: FLINK-25673
> URL: https://issues.apache.org/jira/browse/FLINK-25673
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test
> {{AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond}}
> hangs on AZP:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29512&view=logs&j=a57e0635-3fad-5b08-57c7-a4142d7d6fa9&t=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7&l=18044
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-18356) Exit code 137 returned from process
[
https://issues.apache.org/jira/browse/FLINK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480913#comment-17480913
]
Yangze Guo commented on FLINK-18356:
Another instance from my Azure:
https://dev.azure.com/karmagyz/Flink/_build/results?buildId=644&view=logs&j=43a593e7-535d-554b-08cc-244368da36b4&t=82d122c0-8bbf-56f3-4c0d-8e3d69630d0f
> Exit code 137 returned from process
> ---
>
> Key: FLINK-18356
> URL: https://issues.apache.org/jira/browse/FLINK-18356
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Tests
>Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0
>Reporter: Piotr Nowojski
>Assignee: Dawid Wysakowicz
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> {noformat}
> = test session starts
> ==
> platform linux -- Python 3.7.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
> cachedir: .tox/py37-cython/.pytest_cache
> rootdir: /__w/3/s/flink-python
> collected 568 items
> pyflink/common/tests/test_configuration.py ..[
> 1%]
> pyflink/common/tests/test_execution_config.py ...[
> 5%]
> pyflink/dataset/tests/test_execution_environment.py .
> ##[error]Exit code 137 returned from process: file name '/bin/docker',
> arguments 'exec -i -u 1002
> 97fc4e22522d2ced1f4d23096b8929045d083dd0a99a4233a8b20d0489e9bddb
> /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'.
> Finishing: Test - python
> {noformat}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=3729&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-25683) wrong result if table transfrom to DataStream then window process in batch mode
[ https://issues.apache.org/jira/browse/FLINK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Wysakowicz closed FLINK-25683. Resolution: Fixed > wrong result if table transfrom to DataStream then window process in batch > mode > --- > > Key: FLINK-25683 > URL: https://issues.apache.org/jira/browse/FLINK-25683 > Project: Flink > Issue Type: Bug > Components: Table SQL / API, Table SQL / Runtime >Affects Versions: 1.14.2 > Environment: mac book pro m1 > jdk 8 > scala 2.11 > flink 1.14.2 > idea 2020 >Reporter: zhangzh >Assignee: Yao Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0, 1.14.4 > > Attachments: TableToDataStreamBatchWindowTest.scala, pom.xml > > > I have 5 line datas, > i first need to transform current data with SQL > then mix current data and historical data which is batch get from hbase > for some special reason the program must run in batch mode > i think the correct result should be like this： > (BOB,1) > (EMA,1) > (DOUG,1) > (ALICE,1) > (CENDI,1) > but the result is : > (EMA,1) > > if i set different parallelism ,the result is different. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (FLINK-18356) Exit code 137 returned from process
[
https://issues.apache.org/jira/browse/FLINK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480913#comment-17480913
]
Yangze Guo edited comment on FLINK-18356 at 1/24/22, 8:38 AM:
--
Another two instances from my Azure:
https://dev.azure.com/karmagyz/Flink/_build/results?buildId=644&view=logs&j=43a593e7-535d-554b-08cc-244368da36b4&t=82d122c0-8bbf-56f3-4c0d-8e3d69630d0f
https://dev.azure.com/karmagyz/Flink/_build/results?buildId=645&view=logs&j=43a593e7-535d-554b-08cc-244368da36b4&t=82d122c0-8bbf-56f3-4c0d-8e3d69630d0f
was (Author: karmagyz):
Another instance from my Azure:
https://dev.azure.com/karmagyz/Flink/_build/results?buildId=644&view=logs&j=43a593e7-535d-554b-08cc-244368da36b4&t=82d122c0-8bbf-56f3-4c0d-8e3d69630d0f
> Exit code 137 returned from process
> ---
>
> Key: FLINK-18356
> URL: https://issues.apache.org/jira/browse/FLINK-18356
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Tests
>Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0
>Reporter: Piotr Nowojski
>Assignee: Dawid Wysakowicz
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> {noformat}
> = test session starts
> ==
> platform linux -- Python 3.7.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
> cachedir: .tox/py37-cython/.pytest_cache
> rootdir: /__w/3/s/flink-python
> collected 568 items
> pyflink/common/tests/test_configuration.py ..[
> 1%]
> pyflink/common/tests/test_execution_config.py ...[
> 5%]
> pyflink/dataset/tests/test_execution_environment.py .
> ##[error]Exit code 137 returned from process: file name '/bin/docker',
> arguments 'exec -i -u 1002
> 97fc4e22522d2ced1f4d23096b8929045d083dd0a99a4233a8b20d0489e9bddb
> /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'.
> Finishing: Test - python
> {noformat}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=3729&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) * a4acd8e2b6d14724243eddfdd22caa8bf11890e5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fapaul commented on a change in pull request #17598: [FLINK-24703][connectors][formats] Add CSV format support for filesystem based on StreamFormat and BulkWriter interfaces.
fapaul commented on a change in pull request #17598: URL: https://github.com/apache/flink/pull/17598#discussion_r790515844 ## File path: flink-formats/flink-csv/src/test/java/org/apache/flink/formats/csv/DataStreamCsvITCase.java ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.formats.csv; + +import org.apache.flink.api.common.eventtime.WatermarkStrategy; +import org.apache.flink.api.common.restartstrategy.RestartStrategies; +import org.apache.flink.api.common.serialization.BulkWriter; +import org.apache.flink.api.common.typeinfo.TypeInformation; +import org.apache.flink.connector.file.sink.FileSink; +import org.apache.flink.connector.file.src.FileSource; +import org.apache.flink.connector.file.src.reader.StreamFormat; +import org.apache.flink.core.fs.Path; +import org.apache.flink.core.testutils.AllCallbackWrapper; +import org.apache.flink.formats.common.Converter; +import org.apache.flink.runtime.minicluster.RpcServiceSharing; +import org.apache.flink.runtime.testutils.MiniClusterExtension; +import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration; +import org.apache.flink.streaming.api.datastream.DataStream; +import org.apache.flink.streaming.api.datastream.DataStreamUtils; +import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; +import org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.BasePathBucketAssigner; +import org.apache.flink.streaming.api.operators.collect.ClientAndIterator; +import org.apache.flink.util.TestLoggerExtension; +import org.apache.flink.util.function.FunctionWithException; + +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonPropertyOrder; +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvMapper; +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema; + +import org.apache.commons.io.FileUtils; +import org.jetbrains.annotations.NotNull; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.extension.RegisterExtension; +import org.junit.jupiter.api.io.TempDir; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.OutputStream; +import java.io.OutputStreamWriter; +import java.io.PrintWriter; +import java.io.Serializable; +import java.math.BigDecimal; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.UUID; +import java.util.stream.Collectors; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertEquals; Review comment: Please only use assertJ ## File path: flink-formats/flink-csv/src/main/java/org/apache/flink/formats/csv/CsvCommons.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.formats.csv; + +import org.apache.flink.configuration.ConfigOption; +import org.apache.flink.configuration.ReadableConfig; +import org.apache.flink.table.api.ValidationException; + +import org.apache.commons.lang3.StringEscapeUtils; + +import java.util.HashSet; +import java.util.Set; + +import static org.apache.flink.format
[GitHub] [flink] flinkbot edited a comment on pull request #18460: [FLINK-25767][doc] Totally translated state.md into Chinese
flinkbot edited a comment on pull request #18460: URL: https://github.com/apache/flink/pull/18460#issuecomment-1019844798 ## CI report: * 6385885a06a970cc31f450251e9f6b0d4150a0dd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30002) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25772) GCS filessystem fails license checker
Till Rohrmann created FLINK-25772:
-
Summary: GCS filessystem fails license checker
Key: FLINK-25772
URL: https://issues.apache.org/jira/browse/FLINK-25772
Project: Flink
Issue Type: Bug
Components: Connectors / FileSystem
Affects Versions: 1.15.0
Reporter: Till Rohrmann
Fix For: 1.15.0
FLINK-11838 made the {{LicenseChecker}} fail with the following exception when
compiling using Java 11.
{code}
00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
[] - File '/javax/annotation/security/package.html' in jar
'/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
contains match with forbidden regex 'gnu ?\R?[\s/#]*general ?\R?[\s/#]*public
?\R?[\s/#]*license'.
00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
[] - File '/javax/annotation/package.html' in jar
'/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
contains match with forbidden regex 'gnu ?\R?[\s/#]*general ?\R?[\s/#]*public
?\R?[\s/#]*license'.
00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
[] - Found a total of 3 severe license issues
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25772) GCS filessystem fails license checker
[
https://issues.apache.org/jira/browse/FLINK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-25772:
--
Labels: test-stability (was: )
> GCS filessystem fails license checker
> -
>
> Key: FLINK-25772
> URL: https://issues.apache.org/jira/browse/FLINK-25772
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> FLINK-11838 made the {{LicenseChecker}} fail with the following exception
> when compiling using Java 11.
> {code}
> 00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/security/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
> [] - Found a total of 3 severe license issues
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25772) GCS filessystem fails license checker
[
https://issues.apache.org/jira/browse/FLINK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480918#comment-17480918
]
Till Rohrmann commented on FLINK-25772:
---
cc [~xtsong], [~galenwarren] could you take a look?
> GCS filessystem fails license checker
> -
>
> Key: FLINK-25772
> URL: https://issues.apache.org/jira/browse/FLINK-25772
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Fix For: 1.15.0
>
>
> FLINK-11838 made the {{LicenseChecker}} fail with the following exception
> when compiling using Java 11.
> {code}
> 00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/security/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
> [] - Found a total of 3 severe license issues
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25772) GCS filesystem fails license checker
[
https://issues.apache.org/jira/browse/FLINK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-25772:
--
Summary: GCS filesystem fails license checker (was: GCS filessystem fails
license checker)
> GCS filesystem fails license checker
>
>
> Key: FLINK-25772
> URL: https://issues.apache.org/jira/browse/FLINK-25772
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> FLINK-11838 made the {{LicenseChecker}} fail with the following exception
> when compiling using Java 11.
> {code}
> 00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/security/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
> [] - Found a total of 3 severe license issues
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25758) GCS Filesystem implementation fails on Java 11 tests due to licensing issues
[
https://issues.apache.org/jira/browse/FLINK-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-25758:
--
Issue Type: Bug (was: Improvement)
> GCS Filesystem implementation fails on Java 11 tests due to licensing issues
>
>
> Key: FLINK-25758
> URL: https://issues.apache.org/jira/browse/FLINK-25758
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
>Affects Versions: 1.15.0
>Reporter: Martijn Visser
>Assignee: Galen Warren
>Priority: Blocker
> Labels: pull-request-available
>
> {code}
> 00:33:45,410 DEBUG org.apache.flink.tools.ci.licensecheck.NoticeFileChecker
> [] - Dependency io.netty:netty-common:4.1.51.Final is mentioned in NOTICE
> file /__w/2/s/flink-python/src/main/resources/META-INF/NOTICE, but was not
> mentioned by the build output as a bundled dependency
> 00:33:45,411 ERROR org.apache.flink.tools.ci.licensecheck.NoticeFileChecker
> [] - Could not find dependency javax.annotation:javax.annotation-api:1.3.2
> in NOTICE file
> /__w/2/s/flink-filesystems/flink-gs-fs-hadoop/src/main/resources/META-INF/NOTICE
> 00:33:45,536 INFO org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - Checking directory /tmp/flink-validation-deployment with a total of
> 197 jar files.
> 00:34:18,554 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/security/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220122.001944-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:34:18,555 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220122.001944-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:35:46,612 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
> [] - Found a total of 3 severe license issues
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29932&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-25772) GCS filesystem fails license checker
[
https://issues.apache.org/jira/browse/FLINK-25772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-25772.
-
Fix Version/s: (was: 1.15.0)
Resolution: Duplicate
> GCS filesystem fails license checker
>
>
> Key: FLINK-25772
> URL: https://issues.apache.org/jira/browse/FLINK-25772
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Labels: test-stability
>
> FLINK-11838 made the {{LicenseChecker}} fail with the following exception
> when compiling using Java 11.
> {code}
> 00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/security/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker
> [] - File '/javax/annotation/package.html' in jar
> '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar'
> contains match with forbidden regex 'gnu ?\R?[\s/#]*general
> ?\R?[\s/#]*public ?\R?[\s/#]*license'.
> 00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker
> [] - Found a total of 3 severe license issues
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) * a638e63713a9453e027d5c45aa6db05a9bec9230 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) * b9b29ad0b3697de70c570ee08d4ccf45f1c3e0bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) * c664a06bc46a706892eb58a86701a477f35d6b77 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-table-store] tsreaper commented on a change in pull request #13: [FLINK-25770] Delete file is not correct in MergeTreeWriter
tsreaper commented on a change in pull request #13:
URL: https://github.com/apache/flink-table-store/pull/13#discussion_r790492629
##
File path:
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/mergetree/MergeTreeWriter.java
##
@@ -134,26 +139,36 @@ public void sync() throws Exception {
}
private Increment drainIncrement() {
+// drain files to create Increment
Increment increment =
new Increment(
new ArrayList<>(newFiles),
-new ArrayList<>(compactBefore),
+new ArrayList<>(compactBefore.values()),
new ArrayList<>(compactAfter));
newFiles.clear();
compactBefore.clear();
compactAfter.clear();
+
+// return increment
return increment;
Review comment:
No need for this comment.
##
File path:
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/mergetree/MergeTreeWriter.java
##
@@ -112,8 +115,10 @@ private void flush() throws Exception {
Iterator iterator = memTable.iterator(keyComparator,
accumulator);
List files =
sstFile.write(CloseableIterator.adapterForIterator(iterator), 0);
-newFiles.addAll(files);
-files.forEach(levels::addLevel0File);
+for (SstFileMeta file : files) {
+newFiles.add(file);
+levels.addLevel0File(file);
+}
Review comment:
No need for this change?
##
File path:
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/mergetree/MergeTreeWriter.java
##
@@ -134,26 +139,36 @@ public void sync() throws Exception {
}
private Increment drainIncrement() {
+// drain files to create Increment
Review comment:
No need for this comment.
##
File path:
flink-table-store-core/src/test/java/org/apache/flink/table/store/file/mergetree/MergeTreeTest.java
##
@@ -61,7 +61,12 @@
import static java.util.Collections.singletonList;
import static org.assertj.core.api.Assertions.assertThat;
-/** Test for {@link MergeTree}. */
+/**
+ * Test for {@link MergeTree}.
+ *
+ * Manual test: please adjust TARGET_FILE_SIZE to 1, so that a large number
of upgrade files will
+ * be generated.
Review comment:
No need for manual test. For `testCloseUpgrade` set write buffer size to
4kb, page size to 1kb and target file size to 1kb. With 1000 records and
`@RepeatedTest(10)` to run 10 times it has quite some probability to fail
without this fix.
##
File path:
flink-table-store-core/src/test/java/org/apache/flink/table/store/file/operation/TestCommitThread.java
##
@@ -173,7 +173,11 @@ public void run() {
} catch (Exception e) {
throw new RuntimeException(e);
}
-writer.close();
+try {
+writer.close();
+} catch (Exception e) {
+throw new RuntimeException(e);
+}
Review comment:
Remove `try {.writer.sync() }` above this code fragment because it is
included in `writer.close()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-table-store] tsreaper commented on a change in pull request #13: [FLINK-25770] Delete file is not correct in MergeTreeWriter
tsreaper commented on a change in pull request #13:
URL: https://github.com/apache/flink-table-store/pull/13#discussion_r790514373
##
File path:
flink-table-store-core/src/test/java/org/apache/flink/table/store/file/operation/TestCommitThread.java
##
@@ -173,7 +173,11 @@ public void run() {
} catch (Exception e) {
throw new RuntimeException(e);
}
-writer.close();
+try {
+writer.close();
+} catch (Exception e) {
+throw new RuntimeException(e);
+}
Review comment:
Remove `try { writer.sync() }` above this code fragment because it is
included in `writer.close()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #18452: [FLINK-25758][flink-gs-fs-hadoop] Fix licensing issues
tillrohrmann commented on a change in pull request #18452: URL: https://github.com/apache/flink/pull/18452#discussion_r790521866 ## File path: flink-filesystems/flink-gs-fs-hadoop/pom.xml ## @@ -214,6 +214,8 @@ under the License. META-INF/*.SF META-INF/*.DSA META-INF/*.RSA + + javax/annotation/** Review comment: Instead of filtering it out when the shading happens, can this dependency be excluded when depending on the dependency which pulls it it (e.g. in the `` section)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25307) Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure
[
https://issues.apache.org/jira/browse/FLINK-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480923#comment-17480923
]
Till Rohrmann commented on FLINK-25307:
---
Any progress on this issue [~gaoyunhaii]?
> Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test
> timeout on azure
> ---
>
> Key: FLINK-25307
> URL: https://issues.apache.org/jira/browse/FLINK-25307
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Runtime / Coordination
>Affects Versions: 1.13.3, 1.15.0
>Reporter: Yun Gao
>Assignee: Yun Gao
>Priority: Blocker
> Labels: pull-request-available, stale-critical, test-stability
> Fix For: 1.15.0
>
>
> {code:java}
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/common.sh: line 860:
> kill: (93166) - No such process
> Dec 14 10:30:13 Stopping job timeout watchdog (with pid=93166)
> Dec 14 10:30:13 [FAIL] Test script contains errors.
> Dec 14 10:30:13 Checking for errors...
> Dec 14 10:30:14 No errors in log files.
> Dec 14 10:30:14 Checking for exceptions...
> Dec 14 10:30:14 No exceptions in log files.
> Dec 14 10:30:14 Checking for non-empty .out files...
> Dec 14 10:30:14 No non-empty .out files.
> Dec 14 10:30:14
> Dec 14 10:30:14 [FAIL] 'Resuming Savepoint (hashmap, async, no parallelism
> change) end-to-end test' failed after 15 minutes and 0 seconds! Test exited
> with exit code 1
> Dec 14 10:30:14
> 10:30:14 ##[group]Environment Information
> Dec 14 10:30:15 Searching for .dump, .dumpstream and related files in
> '/home/vsts/work/1/s'
> dmesg: read kernel buffer failed: Operation not permitted
> Dec 14 10:30:16 Stopping taskexecutor daemon (pid: 93751) on host fv-az43-70.
> Dec 14 10:30:17 Stopping standalonesession daemon (pid: 93500) on host
> fv-az43-70.
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/usr/bin/bash'. This may indicate a child process inherited the
> STDIO streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Run e2e tests
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28088&view=logs&j=bea52777-eaf8-5663-8482-18fbc3630e81&t=b2642e3a-5b86-574d-4c8a-f7e2842bfb14&l=79112
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25764) Docker sets JobManager's rpc address to same host by default
[ https://issues.apache.org/jira/browse/FLINK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-25764: - Priority: Minor (was: Major) > Docker sets JobManager's rpc address to same host by default > > > Key: FLINK-25764 > URL: https://issues.apache.org/jira/browse/FLINK-25764 > Project: Flink > Issue Type: Bug > Components: Deployment / Kubernetes, flink-docker >Affects Versions: 1.15.0 >Reporter: Niklas Semmler >Priority: Minor > Labels: usability > > In the [docker > entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh], > the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25). > This environment variable overrides the value set for jobmanager.rpc.address > in the flink config (line 78, 71). For the TaskManager, this means that it > tries to find the JobManager on the same host. When this is not the case, the > TaskManager will retry and ultimately terminate. Hence, for cluster > deployments, the variable has to be defined when starting the docker. > For Kubernetes deployments, the TaskManager cannot connect to the > jobmanager.rpc.address even when it is defined by the flink configmap. > However, we don't see this problem pop up, because for now the configmap is > mounted read-only into the containers (see FLINK-21383 for more details). > To simplify this configuration, I propose to (a) never set a default setting > for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any > non-JobManager container. The only down-side is that all docker deployments > will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and > JobManager run on the same node. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (FLINK-25749) YARNSessionFIFOSecuredITCase.testDetachedMode fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-25749.
-
Resolution: Fixed
> YARNSessionFIFOSecuredITCase.testDetachedMode fails on AZP
> --
>
> Key: FLINK-25749
> URL: https://issues.apache.org/jira/browse/FLINK-25749
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test {{YARNSessionFIFOSecuredITCase.testDetachedMode}} fails on AZP:
> {code}
> 2022-01-21T03:28:18.3712993Z Jan 21 03:28:18 java.lang.AssertionError:
> 2022-01-21T03:28:18.3715115Z Jan 21 03:28:18 Found a file
> /__w/2/s/flink-yarn-tests/target/flink-yarn-tests-fifo-secured/flink-yarn-tests-fifo-secured-logDir-nm-0_0/application_1642735639007_0002/container_1642735639007_0002_01_01/jobmanager.log
> with a prohibited string (one of [Exception, Started
> SelectChannelConnector@0.0.0.0:8081]). Excerpts:
> 2022-01-21T03:28:18.3716389Z Jan 21 03:28:18 [
> 2022-01-21T03:28:18.3717531Z Jan 21 03:28:18 2022-01-21 03:27:56,921 INFO
> org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl [] -
> Resource manager service is not running. Ignore revoking leadership.
> 2022-01-21T03:28:18.3720496Z Jan 21 03:28:18 2022-01-21 03:27:56,922 INFO
> org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped
> dispatcher akka.tcp://flink@11c5f741db81:37697/user/rpc/dispatcher_0.
> 2022-01-21T03:28:18.3722401Z Jan 21 03:28:18 2022-01-21 03:27:56,922 INFO
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] -
> Interrupted while waiting for queue
> 2022-01-21T03:28:18.3723661Z Jan 21 03:28:18 java.lang.InterruptedException:
> null
> 2022-01-21T03:28:18.3724529Z Jan 21 03:28:18 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
> ~[?:1.8.0_292]
> 2022-01-21T03:28:18.3725450Z Jan 21 03:28:18 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
> ~[?:1.8.0_292]
> 2022-01-21T03:28:18.3726239Z Jan 21 03:28:18 at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> ~[?:1.8.0_292]
> 2022-01-21T03:28:18.3727618Z Jan 21 03:28:18 at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:323)
> [hadoop-yarn-client-2.8.5.jar:?]
> 2022-01-21T03:28:18.3729147Z Jan 21 03:28:18 2022-01-21 03:27:56,927 WARN
> org.apache.hadoop.ipc.Client [] - Failed to
> connect to server: 11c5f741db81/172.25.0.2:39121: retries get failed due to
> exceeded maximum allowed retries number: 0
> 2022-01-21T03:28:18.3730293Z Jan 21 03:28:18
> java.nio.channels.ClosedByInterruptException: null
> 2022-01-21T03:28:18.3730834Z Jan 21 03:28:18
> java.nio.channels.ClosedByInterruptException: null
> 2022-01-21T03:28:18.3731499Z Jan 21 03:28:18 at
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> ~[?:1.8.0_292]
> 2022-01-21T03:28:18.3732203Z Jan 21 03:28:18 at
> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:658)
> ~[?:1.8.0_292]
> 2022-01-21T03:28:18.3733478Z Jan 21 03:28:18 at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
> ~[hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3734470Z Jan 21 03:28:18 at
> org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
> ~[hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3735432Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3736414Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3737734Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:410)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3738853Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client.getConnection(Client.java:1550)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3739752Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client.call(Client.java:1381)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3740638Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.Client.call(Client.java:1345)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3741589Z Jan 21 03:28:18 at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> [hadoop-common-2.8.5.jar:?]
> 2022-01-21T03:28:18.3742621Z Jan 21 03:28:18 at
> org.apache.hadoop.
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) * a4acd8e2b6d14724243eddfdd22caa8bf11890e5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25764) Docker sets JobManager's rpc address to same host by default
[ https://issues.apache.org/jira/browse/FLINK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480927#comment-17480927 ] Chesnay Schepler commented on FLINK-25764: -- The docker image having a default for JOB_MANAGER_RPC_ADDRESS is not a problem in that sense. If we were to remove that you'd still have a default "localhost" setting through the Flink config. So as I understand it all it does is improve the out-of-the-box experience if you're just experimenting with docker. ??(b) never set a default setting for any non-JobManager container?? We can't, or rather shouldn't, make this distinction. The "jobmanager"/"taskmanager"/etc branches you see in the docker image are essentially legacy code that we'd ideally get rid of, so that users use the distribution in the docker image as if it weren't a docker image (i.e., they use it as if they'd just use it without docker). The core issue is that the docker scripts expect all changes to the config to happen _after_ it sets the defaults (e.g., by setting them via FLINK_PROPERTIES or as dynamic properties when starting the JM/TM.) What we need is a way to detect whether the flink config that is in the container is actually the default config, or already modified by user. If it is the latter, we wouldn't change anything. I'm not sure what the best option is for that, but maybe a simple checksum would work. > Docker sets JobManager's rpc address to same host by default > > > Key: FLINK-25764 > URL: https://issues.apache.org/jira/browse/FLINK-25764 > Project: Flink > Issue Type: Bug > Components: Deployment / Kubernetes, flink-docker >Affects Versions: 1.15.0 >Reporter: Niklas Semmler >Priority: Minor > Labels: usability > > In the [docker > entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh], > the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25). > This environment variable overrides the value set for jobmanager.rpc.address > in the flink config (line 78, 71). For the TaskManager, this means that it > tries to find the JobManager on the same host. When this is not the case, the > TaskManager will retry and ultimately terminate. Hence, for cluster > deployments, the variable has to be defined when starting the docker. > For Kubernetes deployments, the TaskManager cannot connect to the > jobmanager.rpc.address even when it is defined by the flink configmap. > However, we don't see this problem pop up, because for now the configmap is > mounted read-only into the containers (see FLINK-21383 for more details). > To simplify this configuration, I propose to (a) never set a default setting > for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any > non-JobManager container. The only down-side is that all docker deployments > will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and > JobManager run on the same node. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18459: fix syntax errors in joins.md
flinkbot edited a comment on pull request #18459: URL: https://github.com/apache/flink/pull/18459#issuecomment-1019802598 ## CI report: * 0c3df235f0c927490dd0d4917d39092f175e7fd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29994) * c664a06bc46a706892eb58a86701a477f35d6b77 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30003) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25765) Kubernetes: flink's configmap and flink's actual config are out of sync
[ https://issues.apache.org/jira/browse/FLINK-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480929#comment-17480929 ] Chesnay Schepler commented on FLINK-25765: -- ??manage user's expectations regarding the configmap?? I'd rather do this. As is we expect users to set the configmap once and to then never touch it again. > Kubernetes: flink's configmap and flink's actual config are out of sync > --- > > Key: FLINK-25765 > URL: https://issues.apache.org/jira/browse/FLINK-25765 > Project: Flink > Issue Type: Bug > Components: Deployment / Kubernetes >Affects Versions: 1.15.0 >Reporter: Niklas Semmler >Priority: Major > Labels: usability > > For kubernetes setups, Flink's configmap does not reflect the actual config. > Causes > # Config values are overridden by the environment variables in the docker > image (see FLINK-25764) > # Flink reads the config on start-up, but does not subscribe to changes > # Changes to the config map do not lead to restarts of the flink cluster > Effects > # Users cannot expect to understand Flink's config from the configmap > # TaskManager/JobManager started at different times may start with different > configs, if the user edits the configmap > Related to FLINK-21383. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] Myasuka commented on a change in pull request #18460: [FLINK-25767][doc] Totally translated state.md into Chinese
Myasuka commented on a change in pull request #18460:
URL: https://github.com/apache/flink/pull/18460#discussion_r790516189
##
File path: docs/content.zh/docs/dev/datastream/fault-tolerance/state.md
##
@@ -25,32 +25,23 @@ specific language governing permissions and limitations
under the License.
-->
-# Working with State
+# 使用状态
-In this section you will learn about the APIs that Flink provides for writing
-stateful programs. Please take a look at [Stateful Stream
-Processing]({{< ref "docs/concepts/stateful-stream-processing" >}})
-to learn about the concepts behind stateful stream processing.
+本章节您将学到 Flink 用于编写有状态程序的 API。要学习有状态流处理背后的概念，请参阅[Stateful Stream
+Processing]({{< ref "docs/concepts/stateful-stream-processing" >}})。
## Keyed DataStream
-If you want to use keyed state, you first need to specify a key on a
-`DataStream` that should be used to partition the state (and also the records
-in the stream themselves). You can specify a key using `keyBy(KeySelector)`
-in Java/Scala API or `key_by(KeySelector)` in Python API on a `DataStream`.
-This will yield a `KeyedStream`, which then allows operations that use keyed
state.
+如果你希望使用 keyed state，首先需要为`DataStream`指定 key。这个 key 用于状态分区（也会给数据流中的记录本身分区）。
+你能够使用 `DataStream` 中 Java/Scala API 的 `keyBy(KeySelector)` 或者是 Python API 的
`key_by(KeySelector)` 来指定 key。
+它将返回 `KeyedStream`，从而允许使用 keyed state 操作。
-A key selector function takes a single record as input and returns the key for
-that record. The key can be of any type and **must** be derived from
-deterministic computations.
+Key selector 函数接收单个记录作为输入，返回这条记录的 key。该 key 可以为任何类型，它**必须**能够被推算出来。
Review comment:
`deterministic` 是想强调 这个selector的计算是确定的、恒定的。
##
File path: docs/content.zh/docs/dev/datastream/fault-tolerance/state.md
##
@@ -25,32 +25,23 @@ specific language governing permissions and limitations
under the License.
-->
-# Working with State
+# 使用状态
-In this section you will learn about the APIs that Flink provides for writing
-stateful programs. Please take a look at [Stateful Stream
-Processing]({{< ref "docs/concepts/stateful-stream-processing" >}})
-to learn about the concepts behind stateful stream processing.
+本章节您将学到 Flink 用于编写有状态程序的 API。要学习有状态流处理背后的概念，请参阅[Stateful Stream
+Processing]({{< ref "docs/concepts/stateful-stream-processing" >}})。
## Keyed DataStream
-If you want to use keyed state, you first need to specify a key on a
-`DataStream` that should be used to partition the state (and also the records
-in the stream themselves). You can specify a key using `keyBy(KeySelector)`
-in Java/Scala API or `key_by(KeySelector)` in Python API on a `DataStream`.
-This will yield a `KeyedStream`, which then allows operations that use keyed
state.
+如果你希望使用 keyed state，首先需要为`DataStream`指定 key。这个 key 用于状态分区（也会给数据流中的记录本身分区）。
+你能够使用 `DataStream` 中 Java/Scala API 的 `keyBy(KeySelector)` 或者是 Python API 的
`key_by(KeySelector)` 来指定 key。
Review comment:
```suggestion
你可以使用 `DataStream` 中 Java/Scala API 的 `keyBy(KeySelector)` 或者是 Python API 的
`key_by(KeySelector)` 来指定 key。
```
##
File path: docs/content.zh/docs/dev/datastream/fault-tolerance/state.md
##
@@ -605,41 +593,27 @@ val counts: DataStream[(String, Int)] = stream
})
```
-## Operator State
+## 操作符状态 (Operator State)
-*Operator State* (or *non-keyed state*) is state that is bound to one
-parallel operator instance. The [Kafka Connector]({{< ref
"docs/connectors/datastream/kafka" >}}) is a good motivating example for the
use of
-Operator State in Flink. Each parallel instance of the Kafka consumer maintains
-a map of topic partitions and offsets as its Operator State.
+*操作符状态*（或者*非键控状态*）是绑定到一个并行操作符实例的状态。在 Flink 中使用操作符状态，[Kafka Connector]({{< ref
"docs/connectors/datastream/kafka" >}})是一个很具有启发性的例子。Kafka 消费者每个并发实例维护了 topic
partitions 和偏移量的 map 作为它的操作符状态。
Review comment:
这里的翻译很生硬，愿意是想用Kafka的source connector是如何实现的来阐述如何用operator state。
##
File path: docs/content.zh/docs/dev/datastream/fault-tolerance/state.md
##
@@ -85,15 +76,12 @@ keyed = words.key_by(lambda row: row[0])
{{< /tab >}}
{{< /tabs >}}
- Tuple Keys and Expression Keys
+ 元组 Keys 和表达式 Keys
Review comment:
Tuple keys 说的是废弃的 `#keyBy(int... fields)` API
Expression Keys 说的是 废弃的 `keyBy(String... fields)` API，这里的翻译会让人难以理解。
##
File path: docs/content.zh/docs/dev/datastream/fault-tolerance/state.md
##
@@ -85,15 +76,12 @@ keyed = words.key_by(lambda row: row[0])
{{< /tab >}}
{{< /tabs >}}
- Tuple Keys and Expression Keys
+ 元组 Keys 和表达式 Keys
-Flink also has two alternative ways of defining keys: tuple keys and expression
-keys in the Java/Scala API(still not supported in the Python API). With this
you can
-specify keys using tuple field indices or expressions
-for selecting fields of objects. We don't recommend using these today but you
-can refer to the Javadoc of DataStream to learn about them. Using a KeySelecto
[GitHub] [flink] dannycranmer merged pull request #18348: [FLINK-24905][connectors/kinesis] Adding support for Kinesis DataStream sink as kinesis table connector sink.
dannycranmer merged pull request #18348: URL: https://github.com/apache/flink/pull/18348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18119: [FLINK-24947] Support hostNetwork for native K8s integration on session mode
flinkbot edited a comment on pull request #18119: URL: https://github.com/apache/flink/pull/18119#issuecomment-994734000 ## CI report: * 237027aabd1c103dbdd722e67f1f2a8475149c48 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29978) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18409: [hotfix][dist] Fix that flink-gs-fs-hadoop not included in flink-dist
flinkbot edited a comment on pull request #18409: URL: https://github.com/apache/flink/pull/18409#issuecomment-1017132626 ## CI report: * a44383b3a2c7a6dc5a9c96ad225ad8baaf03fee4 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29986) * 74c02f4b21ad4488b0466a748114e26f15495c4e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29991) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18360: [FLINK-25329][runtime] Support memory execution graph store in session cluster
flinkbot edited a comment on pull request #18360: URL: https://github.com/apache/flink/pull/18360#issuecomment-1012981724 ## CI report: * 36e5148ffb71691d7c9798b51fb52e88f1927748 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29514) * a4acd8e2b6d14724243eddfdd22caa8bf11890e5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30004) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18394: [FLINK-25520][Table SQL/API] Implement "ALTER TABLE ... COMPACT" SQL
flinkbot edited a comment on pull request #18394: URL: https://github.com/apache/flink/pull/18394#issuecomment-1015323011 ## CI report: * 7da11c60c656bfab79cf3ae76bc56cc729ae24a6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29857) * 154b36442660ab9898943a3c3d34a2591023f312 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29997) * a638e63713a9453e027d5c45aa6db05a9bec9230 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18428: [FLINK-25575] Add Sink V2 operators and translation
flinkbot edited a comment on pull request #18428: URL: https://github.com/apache/flink/pull/18428#issuecomment-1017677769 ## CI report: * f4fdde6fd2c0cf28cf44d63bd8dfc34da24690d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29898) * 7e814ab74613f30066a09b7b7e7c406493607c58 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29998) * b9b29ad0b3697de70c570ee08d4ccf45f1c3e0bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] alpreu commented on a change in pull request #18145: [FLINK-25368][connectors/kafka] Substitute KafkaConsumer with AdminClient when getting offsets
alpreu commented on a change in pull request #18145:
URL: https://github.com/apache/flink/pull/18145#discussion_r790522381
##
File path:
flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.java
##
@@ -281,27 +283,50 @@ protected AbstractPartitionDiscoverer
createPartitionDiscoverer(
protected Map fetchOffsetsWithTimestamp(
Collection partitions, long timestamp) {
-Map partitionOffsetsRequest = new
HashMap<>(partitions.size());
+Map partitionOffsetsRequest = new
HashMap<>(partitions.size());
for (KafkaTopicPartition partition : partitions) {
partitionOffsetsRequest.put(
-new TopicPartition(partition.getTopic(),
partition.getPartition()), timestamp);
+new TopicPartition(partition.getTopic(),
partition.getPartition()),
+OffsetSpec.forTimestamp(timestamp));
}
final Map result = new
HashMap<>(partitions.size());
// use a short-lived consumer to fetch the offsets;
// this is ok because this is a one-time operation that happens only
on startup
-try (KafkaConsumer consumer = new KafkaConsumer(properties)) {
-for (Map.Entry
partitionToOffset :
-
consumer.offsetsForTimes(partitionOffsetsRequest).entrySet()) {
-
+try (Admin adminClient = Admin.create(properties)) {
+Map topicPartitionOffsets =
+adminClient
+.listOffsets(partitionOffsetsRequest)
+.all()
+.thenApply(
+info -> {
+Map offsets =
new HashMap<>();
+info.forEach(
+(tp, listOffsetsResultInfo) ->
{
+if (listOffsetsResultInfo
!= null) {
+offsets.put(
+tp,
listOffsetsResultInfo.offset());
+}
+});
+return offsets;
+})
+.get();
+for (Map.Entry partitionToOffset :
+topicPartitionOffsets.entrySet()) {
result.put(
new KafkaTopicPartition(
partitionToOffset.getKey().topic(),
partitionToOffset.getKey().partition()),
(partitionToOffset.getValue() == null)
? null
-: partitionToOffset.getValue().offset());
+: partitionToOffset.getValue());
Review comment:
I believe the code can be easier to read if this for loop was moved into
the above one (L304), WDYT?
##
File path:
flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/enumerator/KafkaSourceEnumerator.java
##
@@ -558,25 +533,100 @@ public PartitionOffsetsRetrieverImpl(
}
}
+/**
+ * List offset for the specified partitions and OffsetSpec. This
operation enables to find
Review comment:
```suggestion
* List offsets for the specified partitions and OffsetSpec. This
operation enables to find
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wsry commented on a change in pull request #18130: [FLINK-25035][runtime] Shuffle service supports consuming subpartition range
wsry commented on a change in pull request #18130: URL: https://github.com/apache/flink/pull/18130#discussion_r790532234 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/InputChannel.java ## @@ -94,6 +97,7 @@ protected InputChannel( this.inputGate = checkNotNull(inputGate); this.channelInfo = new InputChannelInfo(inputGate.getGateIndex(), channelIndex); this.partitionId = checkNotNull(partitionId); +this.consumedSubpartitionIndex = consumedSubpartitionIndex; Review comment: nit: maybe add an ```>=0``` argument check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25486) Perjob can not recover from checkpoint when zookeeper leader changes
[ https://issues.apache.org/jira/browse/FLINK-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-25486: -- Priority: Blocker (was: Critical) > Perjob can not recover from checkpoint when zookeeper leader changes > > > Key: FLINK-25486 > URL: https://issues.apache.org/jira/browse/FLINK-25486 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.15.0, 1.13.5, 1.14.2 >Reporter: Liu >Assignee: Liu >Priority: Blocker > Labels: pull-request-available > Fix For: 1.15.0, 1.13.6, 1.14.4 > > > When the config > high-availability.zookeeper.client.tolerate-suspended-connections is default > false, the appMaster will failover once zk leader changes. In this case, the > old appMaster will clean up all the zk info and the new appMaster will not > recover from the latest checkpoint. > The process is as following: > # Start a perJob application. > # kill zk's leade node which cause the perJob to suspend. > # In MiniDispatcher's function jobReachedTerminalState, shutDownFuture is > set to UNKNOWN . > # The future is transferred to ClusterEntrypoint, the method is called with > cleanupHaData true. > # Clean up zk data and exit. > # The new appMaster will not find any checkpoints to start and the state is > lost. > Since the job can recover automatically when the zk leader changes, it is > reasonable to keep zk info for the coming recovery. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] zhuzhurk closed pull request #18126: [FLINK-25036][runtime] Introduce vertex wise scheduling strategy
zhuzhurk closed pull request #18126: URL: https://github.com/apache/flink/pull/18126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
