[jira] [Created] (FLINK-21998) Copy more code from hive and move them to a dedicated package
Rui Li created FLINK-21998: -- Summary: Copy more code from hive and move them to a dedicated package Key: FLINK-21998 URL: https://issues.apache.org/jira/browse/FLINK-21998 Project: Flink Issue Type: Sub-task Components: Connectors / Hive Reporter: Rui Li Fix For: 1.13.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21999) The logic about whether Checkpoint is enabled.
ZhangWei created FLINK-21999: Summary: The logic about whether Checkpoint is enabled. Key: FLINK-21999 URL: https://issues.apache.org/jira/browse/FLINK-21999 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Reporter: ZhangWei org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder#isCheckpointingEnabled assumes checkpoint enabled when JobCheckpointingSettings is not null. While this is not enough, we must also guarantee the checkpoint interval is between [MINIMAL_CHECKPOINT_TIME, Long.MaxValue). That is like the JobGraph#isCheckpointingEnabled does. In current implement, when we do not set checkpoint interval, leaving it the default value -1, the interval will be changed to Long.MaxValue. Thus DefaultExecutionGraphBuilder#isCheckpointingEnabled will return true. That is not correct. in addition, there are different classes assume checkpoint enabled with different interval range. 1. CheckpointConfig -> (0,Long.MaxValue*]*. 2. JobGraph -> (0,Long.MaxValue) This is not consistent. And the correct range is [MINIMAL_CHECKPOINT_TIME, Long.MaxValue). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22000) Set a default character set in InputStreamReader to solve potential garbled problems
dbgp2021 created FLINK-22000: Summary: Set a default character set in InputStreamReader to solve potential garbled problems Key: FLINK-22000 URL: https://issues.apache.org/jira/browse/FLINK-22000 Project: Flink Issue Type: Improvement Reporter: dbgp2021 When a InputStreamReader is used, the parameter setting of a default character set is recommended to solve potential garbled problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22001) Exceptions from JobMaster initialization are not forwarded to the user
Robert Metzger created FLINK-22001:
--
Summary: Exceptions from JobMaster initialization are not
forwarded to the user
Key: FLINK-22001
URL: https://issues.apache.org/jira/browse/FLINK-22001
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Robert Metzger
Steps to reproduce:
Set up a streaming job with an invalid parallelism configuration, for example:
{code}
.setParallelism(15).setMaxParallelism(1);
{code}
This should report the following exception to the user:
{code}
Caused by: org.apache.flink.runtime.JobException: Vertex
Window(GlobalWindows(), DeltaTrigger, TimeEvictor, ComparableAggregator,
PassThroughWindowFunction)'s parallelism (15) is higher than the max
parallelism (1). Please lower the parallelism or increase the max parallelism.
at
org.apache.flink.runtime.executiongraph.ExecutionJobVertex.(ExecutionJobVertex.java:160)
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:781)
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:193)
at
org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:106)
at
org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:252)
at
org.apache.flink.runtime.scheduler.SchedulerBase.(SchedulerBase.java:185)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.(DefaultScheduler.java:119)
at
org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:132)
at
org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:110)
at
org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:340)
at
org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:317)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:94)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
at
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.startJobMasterServiceSafely(JobManagerRunnerImpl.java:363)
... 13 more
{code}
However, what the user sees is
{code}
2021-03-28 20:32:33,935 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job
419f60eac551619fc1081c670ced3649 reached globally terminal state FAILED.
...
2021-03-28 20:32:33,974 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped
dispatcher akka://flink/user/rpc/dispatcher_2.
2021-03-28 20:32:33,977 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Stopping Akka RPC service.
Exception in thread "main" org.apache.flink.util.FlinkException: Failed to
execute job 'CarTopSpeedWindowingExample'.
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1975)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1853)
at
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:69)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1839)
at
org.apache.flink.streaming.examples.windowing.TopSpeedWindowing.main(TopSpeedWindowing.java:101)
Caused by: java.lang.RuntimeException: Error while waiting for job to be
initialized
at
org.apache.flink.client.ClientUtils.waitUntilJobInitializationFinished(ClientUtils.java:160)
at
org.apache.flink.client.program.PerJobMiniClusterFactory.lambda$submitJob$2(PerJobMiniClusterFactory.java:83)
at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:73)
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
at
java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorke
[jira] [Created] (FLINK-22002) AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail because of submitting task time-out.
Guowei Ma created FLINK-22002:
-
Summary:
AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail
because of submitting task time-out.
Key: FLINK-22002
URL: https://issues.apache.org/jira/browse/FLINK-22002
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.12.2
Reporter: Guowei Ma
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15634&view=logs&j=955770d3-1fed-5a0a-3db6-0c7554c910cb&t=14447d61-56b4-5000-80c1-daa459247f6a&l=6424
{code:java}
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase
2021-03-29T00:27:25.3406344Z [ERROR]
testSingleAggOnTable_HashAgg_WithLocalAgg(org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase)
Time elapsed: 21.908 s <<< ERROR!
2021-03-29T00:27:25.3407190Z java.lang.RuntimeException: Failed to fetch next
result
2021-03-29T00:27:25.3407792Zat
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
2021-03-29T00:27:25.3408502Zat
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
2021-03-29T00:27:25.3409188Zat
org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:117)
2021-03-29T00:27:25.3416724Zat
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350)
2021-03-29T00:27:25.3417510Zat
java.util.Iterator.forEachRemaining(Iterator.java:115)
2021-03-29T00:27:25.3418416Zat
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:108)
2021-03-29T00:27:25.3419031Zat
org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:298)
2021-03-29T00:27:25.3419657Zat
org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:138)
2021-03-29T00:27:25.3420638Zat
org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:104)
2021-03-29T00:27:25.3421384Zat
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:182)
2021-03-29T00:27:25.3422284Zat
org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg(AggregateReduceGroupingITCase.scala:135)
2021-03-29T00:27:25.3422975Zat
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2021-03-29T00:27:25.3423504Zat
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2021-03-29T00:27:25.3424298Zat
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-03-29T00:27:25.3425229Zat
java.lang.reflect.Method.invoke(Method.java:498)
2021-03-29T00:27:25.3426107Zat
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
2021-03-29T00:27:25.3426756Zat
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2021-03-29T00:27:25.3427743Zat
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
2021-03-29T00:27:25.3428520Zat
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2021-03-29T00:27:25.3429128Zat
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2021-03-29T00:27:25.3429715Zat
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
2021-03-29T00:27:25.3433435Zat
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
2021-03-29T00:27:25.3433977Zat
org.junit.rules.RunRules.evaluate(RunRules.java:20)
2021-03-29T00:27:25.3434476Zat
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
2021-03-29T00:27:25.3435607Zat
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
2021-03-29T00:27:25.3436460Zat
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
2021-03-29T00:27:25.3437054Zat
org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
2021-03-29T00:27:25.3437673Zat
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
2021-03-29T00:27:25.3438765Zat
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
2021-03-29T00:27:25.3439362Zat
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
2021-03-29T00:27:25.3440504Zat
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
2021-03-29T00:27:25.3441100Zat
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
2021-03-29T00:27:25.3441673Zat
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
2021-03-29T00:27:25.3442205Zat
[jira] [Created] (FLINK-22003) UnalignedCheckpointITCase fail
Guowei Ma created FLINK-22003:
-
Summary: UnalignedCheckpointITCase fail
Key: FLINK-22003
URL: https://issues.apache.org/jira/browse/FLINK-22003
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing
Affects Versions: 1.13.0
Reporter: Guowei Ma
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15601&view=logs&j=119bbba7-f5e3-5e08-e72d-09f1529665de&t=7dc1f5a9-54e1-502e-8b02-c7df69073cfc&l=4142
{code:java}
[ERROR] execute[parallel pipeline with remote channels, p =
5](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time
elapsed: 60.018 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 6
milliseconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
at
java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1859)
at
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:69)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1839)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1822)
at
org.apache.flink.test.checkpointing.UnalignedCheckpointTestBase.execute(UnalignedCheckpointTestBase.java:138)
at
org.apache.flink.test.checkpointing.UnalignedCheckpointITCase.execute(UnalignedCheckpointITCase.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-22004) Translate Flink Roadmap to Chinese.
Yuan Mei created FLINK-22004: Summary: Translate Flink Roadmap to Chinese. Key: FLINK-22004 URL: https://issues.apache.org/jira/browse/FLINK-22004 Project: Flink Issue Type: Task Components: Documentation Reporter: Yuan Mei https://flink.apache.org/roadmap.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [BULK]Re: [SURVEY] Remove Mesos support
+1 It's already a matter of fact for a while that we no longer port new features to the Mesos deployment. Thank you~ Xintong Song On Fri, Mar 26, 2021 at 10:37 PM Till Rohrmann wrote: > +1 for officially deprecating this component for the 1.13 release. > > Cheers, > Till > > On Thu, Mar 25, 2021 at 1:49 PM Konstantin Knauf > wrote: > >> Hi Matthias, >> >> Thank you for following up on this. +1 to officially deprecate Mesos in >> the code and documentation, too. It will be confusing for users if this >> diverges from the roadmap. >> >> Cheers, >> >> Konstantin >> >> On Thu, Mar 25, 2021 at 12:23 PM Matthias Pohl >> wrote: >> >>> Hi everyone, >>> considering the upcoming release of Flink 1.13, I wanted to revive the >>> discussion about the Mesos support ones more. Mesos is also already >>> listed >>> as deprecated in Flink's overall roadmap [1]. Maybe, it's time to align >>> the >>> documentation accordingly to make it more explicit? >>> >>> What do you think? >>> >>> Best, >>> Matthias >>> >>> [1] https://flink.apache.org/roadmap.html#feature-radar >>> >>> On Wed, Oct 28, 2020 at 9:40 AM Till Rohrmann >>> wrote: >>> >>> > Hi Oleksandr, >>> > >>> > yes you are right. The biggest problem is at the moment the lack of >>> test >>> > coverage and thereby confidence to make changes. We have some e2e tests >>> > which you can find here [1]. These tests are, however, quite coarse >>> grained >>> > and are missing a lot of cases. One idea would be to add a Mesos e2e >>> test >>> > based on Flink's end-to-end test framework [2]. I think what needs to >>> be >>> > done there is to add a Mesos resource and a way to submit jobs to a >>> Mesos >>> > cluster to write e2e tests. >>> > >>> > [1] https://github.com/apache/flink/tree/master/flink-jepsen >>> > [2] >>> > >>> https://github.com/apache/flink/tree/master/flink-end-to-end-tests/flink-end-to-end-tests-common >>> > >>> > Cheers, >>> > Till >>> > >>> > On Tue, Oct 27, 2020 at 12:29 PM Oleksandr Nitavskyi < >>> > o.nitavs...@criteo.com> wrote: >>> > >>> >> Hello Xintong, >>> >> >>> >> Thanks for the insights and support. >>> >> >>> >> Browsing the Mesos backlog and didn't identify anything critical, >>> which >>> >> is left there. >>> >> >>> >> I see that there are were quite a lot of contributions to the Flink >>> Mesos >>> >> in the recent version: >>> >> https://github.com/apache/flink/commits/master/flink-mesos. >>> >> We plan to validate the current Flink master (or release 1.12 branch) >>> our >>> >> Mesos setup. In case of any issues, we will try to propose changes. >>> >> My feeling is that our test results shouldn't affect the Flink 1.12 >>> >> release cycle. And if any potential commits will land into the 1.12.1 >>> it >>> >> should be totally fine. >>> >> >>> >> In the future, we would be glad to help you guys with any >>> >> maintenance-related questions. One of the highest priorities around >>> this >>> >> component seems to be the development of the full e2e test. >>> >> >>> >> Kind Regards >>> >> Oleksandr Nitavskyi >>> >> >>> >> From: Xintong Song >>> >> Sent: Tuesday, October 27, 2020 7:14 AM >>> >> To: dev ; user >>> >> Cc: Piyush Narang >>> >> Subject: [BULK]Re: [SURVEY] Remove Mesos support >>> >> >>> >> Hi Piyush, >>> >> >>> >> Thanks a lot for sharing the information. It would be a great relief >>> that >>> >> you are good with Flink on Mesos as is. >>> >> >>> >> As for the jira issues, I believe the most essential ones should have >>> >> already been resolved. You may find some remaining open issues here >>> [1], >>> >> but not all of them are necessary if we decide to keep Flink on Mesos >>> as is. >>> >> >>> >> At the moment and in the short future, I think helps are mostly >>> needed on >>> >> testing the upcoming release 1.12 with Mesos use cases. The community >>> is >>> >> currently actively preparing the new release, and hopefully we could >>> come >>> >> up with a release candidate early next month. It would be greatly >>> >> appreciated if you fork as experienced Flink on Mesos users can help >>> with >>> >> verifying the release candidates. >>> >> >>> >> >>> >> Thank you~ >>> >> >>> >> Xintong Song >>> >> >>> >> [1] >>> >> >>> https://issues.apache.org/jira/browse/FLINK-17402?jql=project%20%3D%20FLINK%20AND%20component%20%3D%20%22Deployment%20%2F%20Mesos%22%20AND%20status%20%3D%20Open >>> >> < >>> >> >>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-17402%3Fjql%3Dproject%2520%253D%2520FLINK%2520AND%2520component%2520%253D%2520%2522Deployment%2520%252F%2520Mesos%2522%2520AND%2520status%2520%253D%2520Open&data=04%7C01%7Co.nitavskyi%40criteo.com%7C3585e1f25bdf4e091af808d87a3f92db%7C2a35d8fd574d48e3927c8c398e225a01%7C1%7C0%7C637393760750820881%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hytJFQE0MCPzMLiQTQTdbg3GVckX5M3r1NPRGrRV8j4%3D&reserved=0 >>> >> > >>> >> >>> >> On Tue,
[jira] [Created] (FLINK-22005) SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1)
Guowei Ma created FLINK-22005: - Summary: SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1) Key: FLINK-22005 URL: https://issues.apache.org/jira/browse/FLINK-22005 Project: Flink Issue Type: Bug Components: Table SQL / Client Affects Versions: 1.13.0 Reporter: Guowei Ma The test fail because of Waiting for Elasticsearch records indefinitely. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15583&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529&l=19826 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [BULK]Re: [SURVEY] Remove Mesos support
+1 Best, Yangze Guo On Mon, Mar 29, 2021 at 11:31 AM Xintong Song wrote: > > +1 > It's already a matter of fact for a while that we no longer port new features > to the Mesos deployment. > > Thank you~ > > Xintong Song > > > > On Fri, Mar 26, 2021 at 10:37 PM Till Rohrmann wrote: >> >> +1 for officially deprecating this component for the 1.13 release. >> >> Cheers, >> Till >> >> On Thu, Mar 25, 2021 at 1:49 PM Konstantin Knauf wrote: >>> >>> Hi Matthias, >>> >>> Thank you for following up on this. +1 to officially deprecate Mesos in the >>> code and documentation, too. It will be confusing for users if this >>> diverges from the roadmap. >>> >>> Cheers, >>> >>> Konstantin >>> >>> On Thu, Mar 25, 2021 at 12:23 PM Matthias Pohl >>> wrote: Hi everyone, considering the upcoming release of Flink 1.13, I wanted to revive the discussion about the Mesos support ones more. Mesos is also already listed as deprecated in Flink's overall roadmap [1]. Maybe, it's time to align the documentation accordingly to make it more explicit? What do you think? Best, Matthias [1] https://flink.apache.org/roadmap.html#feature-radar On Wed, Oct 28, 2020 at 9:40 AM Till Rohrmann wrote: > Hi Oleksandr, > > yes you are right. The biggest problem is at the moment the lack of test > coverage and thereby confidence to make changes. We have some e2e tests > which you can find here [1]. These tests are, however, quite coarse > grained > and are missing a lot of cases. One idea would be to add a Mesos e2e test > based on Flink's end-to-end test framework [2]. I think what needs to be > done there is to add a Mesos resource and a way to submit jobs to a Mesos > cluster to write e2e tests. > > [1] https://github.com/apache/flink/tree/master/flink-jepsen > [2] > https://github.com/apache/flink/tree/master/flink-end-to-end-tests/flink-end-to-end-tests-common > > Cheers, > Till > > On Tue, Oct 27, 2020 at 12:29 PM Oleksandr Nitavskyi < > o.nitavs...@criteo.com> wrote: > >> Hello Xintong, >> >> Thanks for the insights and support. >> >> Browsing the Mesos backlog and didn't identify anything critical, which >> is left there. >> >> I see that there are were quite a lot of contributions to the Flink >> Mesos >> in the recent version: >> https://github.com/apache/flink/commits/master/flink-mesos. >> We plan to validate the current Flink master (or release 1.12 branch) >> our >> Mesos setup. In case of any issues, we will try to propose changes. >> My feeling is that our test results shouldn't affect the Flink 1.12 >> release cycle. And if any potential commits will land into the 1.12.1 it >> should be totally fine. >> >> In the future, we would be glad to help you guys with any >> maintenance-related questions. One of the highest priorities around this >> component seems to be the development of the full e2e test. >> >> Kind Regards >> Oleksandr Nitavskyi >> >> From: Xintong Song >> Sent: Tuesday, October 27, 2020 7:14 AM >> To: dev ; user >> Cc: Piyush Narang >> Subject: [BULK]Re: [SURVEY] Remove Mesos support >> >> Hi Piyush, >> >> Thanks a lot for sharing the information. It would be a great relief >> that >> you are good with Flink on Mesos as is. >> >> As for the jira issues, I believe the most essential ones should have >> already been resolved. You may find some remaining open issues here [1], >> but not all of them are necessary if we decide to keep Flink on Mesos >> as is. >> >> At the moment and in the short future, I think helps are mostly needed >> on >> testing the upcoming release 1.12 with Mesos use cases. The community is >> currently actively preparing the new release, and hopefully we could >> come >> up with a release candidate early next month. It would be greatly >> appreciated if you fork as experienced Flink on Mesos users can help >> with >> verifying the release candidates. >> >> >> Thank you~ >> >> Xintong Song >> >> [1] >> https://issues.apache.org/jira/browse/FLINK-17402?jql=project%20%3D%20FLINK%20AND%20component%20%3D%20%22Deployment%20%2F%20Mesos%22%20AND%20status%20%3D%20Open >> < >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-17402%3Fjql%3Dproject%2520%253D%2520FLINK%2520AND%2520component%2520%253D%2520%2522Deployment%2520%252F%2520Mesos%2522%2520AND%2520status%2520%253D%2520Open&data=04%7C01%7Co.nitavskyi%40criteo.com%7C3585e1f25bdf4e091af808d87a3f92db%7C2a35d8fd574d48e3927c8c398e225a01%7C1%7C0%7C637393760750820881%7CUn
[jira] [Created] (FLINK-22006) Could not run more than 20 jobs in a native K8s session with K8s HA enabled
Yang Wang created FLINK-22006: - Summary: Could not run more than 20 jobs in a native K8s session with K8s HA enabled Key: FLINK-22006 URL: https://issues.apache.org/jira/browse/FLINK-22006 Project: Flink Issue Type: Bug Affects Versions: 1.12.2, 1.13.0 Reporter: Yang Wang Attachments: image-2021-03-24-18-08-42-116.png Currently, if we start a native K8s session cluster with K8s HA enabled, we could not run more than 20 streaming jobs. The latest job is always initializing, and the previous one is created and waiting to be assigned. It seems that some internal resources have been exhausted, e.g. okhttp thread pool , tcp connections or something else. !image-2021-03-24-18-08-42-116.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22007) PartitionReleaseInBatchJobBenchmarkExecutor seems to be failing
Piotr Nowojski created FLINK-22007: -- Summary: PartitionReleaseInBatchJobBenchmarkExecutor seems to be failing Key: FLINK-22007 URL: https://issues.apache.org/jira/browse/FLINK-22007 Project: Flink Issue Type: Bug Components: Benchmarks, Runtime / Coordination Affects Versions: 1.13.0 Reporter: Piotr Nowojski Fix For: 1.13.0 Travis CI is failing: https://travis-ci.com/github/apache/flink-benchmarks/builds/221290042 While there is also some problem with the Jenkins builds for the same benchmark. http://codespeed.dak8s.net:8080/job/flink-scheduler-benchmarks/232 It would be also interesting for the future to understand why the Jenkins build is green and try to fix it (ideally, if some benchmarks fail, partial results should be still uploaded but the Jenkins build should be marked as failed). Otherwise issues like that can remain unnoticed for quite a bit of time. CC [~Thesharing] [~zhuzh] -- This message was sent by Atlassian Jira (v8.3.4#803005)
