[jira] [Created] (FLINK-7724) add extra metrics to MetricStoreTest.setupStore

2017-09-27 Thread Bowen Li (JIRA)
Bowen Li created FLINK-7724: --- Summary: add extra metrics to MetricStoreTest.setupStore Key: FLINK-7724 URL: https://issues.apache.org/jira/browse/FLINK-7724 Project: Flink Issue Type: Test

What can Mnemonic do with Flink ?

2017-09-27 Thread Gang Wang
Hi, I'm a contributor from Mnemonic community, I'm looking for an entry point to leverage some features. e.g. durable object/computing models for Flink. Mnemonic has supported some use cases of Hadoop MR and Spark e.g. https://mnemonic.apache.org/docs/sparkusecases.html https://mnemonic.apache.

[jira] [Created] (FLINK-7723) DispatcherTest instable

2017-09-27 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-7723: Summary: DispatcherTest instable Key: FLINK-7723 URL: https://issues.apache.org/jira/browse/FLINK-7723 Project: Flink Issue Type: Bug Components: T

[jira] [Created] (FLINK-7722) MiniCluster does not appear to honor Log4j settings

2017-09-27 Thread Elias Levy (JIRA)
Elias Levy created FLINK-7722: - Summary: MiniCluster does not appear to honor Log4j settings Key: FLINK-7722 URL: https://issues.apache.org/jira/browse/FLINK-7722 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7721) StatusWatermarkValve should output a new min watermark only if it was aggregated from aligned chhanels

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7721: -- Summary: StatusWatermarkValve should output a new min watermark only if it was aggregated from aligned chhanels Key: FLINK-7721 URL: https://issues.apache.org/jira/bro

Re: JIRA contributor rights for FLINK

2017-09-27 Thread Till Rohrmann
Welcome to the Flink community Malcolm :-) I've given you contributor permissions. On Wed, Sep 27, 2017 at 5:07 PM, Malcolm Taylor wrote: > Hi, > I am a new contributor to FLINK. Can someone please assign me contributor > rights in JIRA, so that I can assign an issue ? > My user name is malcolmt

[jira] [Created] (FLINK-7720) Centralize creation of backends and state related resources

2017-09-27 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-7720: - Summary: Centralize creation of backends and state related resources Key: FLINK-7720 URL: https://issues.apache.org/jira/browse/FLINK-7720 Project: Flink

[jira] [Created] (FLINK-7719) Send checkpoint id to task as part of deployment descriptor when resuming

2017-09-27 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-7719: - Summary: Send checkpoint id to task as part of deployment descriptor when resuming Key: FLINK-7719 URL: https://issues.apache.org/jira/browse/FLINK-7719 Project: Fl

[jira] [Created] (FLINK-7718) Port JobVertixMetricsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7718: -- Summary: Port JobVertixMetricsHandler to new REST endpoint Key: FLINK-7718 URL: https://issues.apache.org/jira/browse/FLINK-7718 Project: Flink I

[jira] [Created] (FLINK-7717) Port TaskManagerMetricsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7717: -- Summary: Port TaskManagerMetricsHandler to new REST endpoint Key: FLINK-7717 URL: https://issues.apache.org/jira/browse/FLINK-7717 Project: Flink

[jira] [Created] (FLINK-7714) Port JarPlanHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7714: -- Summary: Port JarPlanHandler to new REST endpoint Key: FLINK-7714 URL: https://issues.apache.org/jira/browse/FLINK-7714 Project: Flink Issue Type

[jira] [Created] (FLINK-7716) Port JobManagerMetricsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7716: -- Summary: Port JobManagerMetricsHandler to new REST endpoint Key: FLINK-7716 URL: https://issues.apache.org/jira/browse/FLINK-7716 Project: Flink

[jira] [Created] (FLINK-7713) Port JarUploadHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7713: -- Summary: Port JarUploadHandler to new REST endpoint Key: FLINK-7713 URL: https://issues.apache.org/jira/browse/FLINK-7713 Project: Flink Issue Ty

JIRA contributor rights for FLINK

2017-09-27 Thread Malcolm Taylor
Hi, I am a new contributor to FLINK. Can someone please assign me contributor rights in JIRA, so that I can assign an issue ? My user name is malcolmt thanks, Malcolm

[jira] [Created] (FLINK-7715) Port JarRunHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7715: -- Summary: Port JarRunHandler to new REST endpoint Key: FLINK-7715 URL: https://issues.apache.org/jira/browse/FLINK-7715 Project: Flink Issue Type:

[jira] [Created] (FLINK-7711) Port JarListHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7711: -- Summary: Port JarListHandler to new REST endpoint Key: FLINK-7711 URL: https://issues.apache.org/jira/browse/FLINK-7711 Project: Flink Issue Type

[jira] [Created] (FLINK-7712) Port JarDeleteHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7712: -- Summary: Port JarDeleteHandler to new REST endpoint Key: FLINK-7712 URL: https://issues.apache.org/jira/browse/FLINK-7712 Project: Flink Issue Ty

[jira] [Created] (FLINK-7709) Port CheckpointStatsDetailsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7709: -- Summary: Port CheckpointStatsDetailsHandler to new REST endpoint Key: FLINK-7709 URL: https://issues.apache.org/jira/browse/FLINK-7709 Project: Flink

[jira] [Created] (FLINK-7710) Port CheckpointStatsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7710: -- Summary: Port CheckpointStatsHandler to new REST endpoint Key: FLINK-7710 URL: https://issues.apache.org/jira/browse/FLINK-7710 Project: Flink Is

[jira] [Created] (FLINK-7707) Port CheckpointStatsDetailsSubtasksHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7707: -- Summary: Port CheckpointStatsDetailsSubtasksHandler to new REST endpoint Key: FLINK-7707 URL: https://issues.apache.org/jira/browse/FLINK-7707 Project: Fl

[jira] [Created] (FLINK-7708) Port CheckpointConfigHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7708: -- Summary: Port CheckpointConfigHandler to new REST endpoint Key: FLINK-7708 URL: https://issues.apache.org/jira/browse/FLINK-7708 Project: Flink I

[jira] [Created] (FLINK-7706) Port JobAccumulatorsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7706: -- Summary: Port JobAccumulatorsHandler to new REST endpoint Key: FLINK-7706 URL: https://issues.apache.org/jira/browse/FLINK-7706 Project: Flink Is

[jira] [Created] (FLINK-7705) Port JobDetailsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7705: -- Summary: Port JobDetailsHandler to new REST endpoint Key: FLINK-7705 URL: https://issues.apache.org/jira/browse/FLINK-7705 Project: Flink Issue T

[jira] [Created] (FLINK-7704) Port JobPlanHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7704: -- Summary: Port JobPlanHandler to new REST endpoint Key: FLINK-7704 URL: https://issues.apache.org/jira/browse/FLINK-7704 Project: Flink Issue Type

[jira] [Created] (FLINK-7703) Port JobExceptionsHandler to new REST endpoint

2017-09-27 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7703: -- Summary: Port JobExceptionsHandler to new REST endpoint Key: FLINK-7703 URL: https://issues.apache.org/jira/browse/FLINK-7703 Project: Flink Issu

[jira] [Created] (FLINK-7702) Javadocs link broken

2017-09-27 Thread Gabor Gevay (JIRA)
Gabor Gevay created FLINK-7702: -- Summary: Javadocs link broken Key: FLINK-7702 URL: https://issues.apache.org/jira/browse/FLINK-7702 Project: Flink Issue Type: Bug Components: Document

[jira] [Created] (FLINK-7701) IllegalArgumentException in Netty bootstrap with small memory state segment size

2017-09-27 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-7701: -- Summary: IllegalArgumentException in Netty bootstrap with small memory state segment size Key: FLINK-7701 URL: https://issues.apache.org/jira/browse/FLINK-7701 Project: F

[jira] [Created] (FLINK-7700) State merging in RocksDB backend leaves old state

2017-09-27 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-7700: --- Summary: State merging in RocksDB backend leaves old state Key: FLINK-7700 URL: https://issues.apache.org/jira/browse/FLINK-7700 Project: Flink Issue T

Re: Zookeeper failure handling

2017-09-27 Thread Gyula Fóra
Thanks Till for pointing these issues out, you are right this is exactly the same problem! Gyula Till Rohrmann ezt írta (időpont: 2017. szept. 27., Sze, 11:18): > I agree that this is not very nice and can put a lot of stress on your > cluster. > > There is actually an open issue for exactly th

[jira] [Created] (FLINK-7699) Define the BufferListener interface to replace EventlListener in BufferProvider

2017-09-27 Thread zhijiang (JIRA)
zhijiang created FLINK-7699: --- Summary: Define the BufferListener interface to replace EventlListener in BufferProvider Key: FLINK-7699 URL: https://issues.apache.org/jira/browse/FLINK-7699 Project: Flink

Re: got Warn message - "the expected leader session ID did not equal the received leader session ID " when using LocalFlinkMiniCluster to interpret scala code

2017-09-27 Thread XiangWei Huang
Hi Till, I’ve found that a StandaloneMiniCluster doesn’t startup web fronted when it is running.so,how can i cancel a running job on it with restful method. Cheers, Till > 在 2017年9月20日,15:43,Till Rohrmann 写道: > > Hi XiangWei, > > programmatically there is no nice tooling yet

[jira] [Created] (FLINK-7698) Join with null literals leads to NPE

2017-09-27 Thread Timo Walther (JIRA)
Timo Walther created FLINK-7698: --- Summary: Join with null literals leads to NPE Key: FLINK-7698 URL: https://issues.apache.org/jira/browse/FLINK-7698 Project: Flink Issue Type: Bug Co

Re: Zookeeper failure handling

2017-09-27 Thread Till Rohrmann
I agree that this is not very nice and can put a lot of stress on your cluster. There is actually an open issue for exactly this [1] and also a PR [2]. The problem is that in the general case it will allow for split-brain situations and therefore it has not been merged yet. I'm actually not quite

Re: Question about job canceling in Flink

2017-09-27 Thread Till Rohrmann
Hi Wangsan, what you are observing is the intended behaviour of Flink in the case of cancelling a job. The assumption is that if a job is cancelled, then there should be no clean shut down. This entails that the Tasks are interrupted in order to terminate them as fast as possible. Thus, the probl

Re: Zookeeper failure handling

2017-09-27 Thread Gyula Fóra
Hi Till, Thanks for the explanation, yes this sounds like a hard problem but it just seems wrong that whenever the ZK leader is restarted all the Flink jobs fail on a cluster. This might be within the overall guarantees of the system but can lead to some cascading failures if every job recovers at

Re: [E] Re: Elasticsearch connector

2017-09-27 Thread Till Rohrmann
The problem are the different signatures. The old version seem to take an `ActionRequest` whereas the old version requires a `IndexRequest`. Cheers, Till On Tue, Sep 26, 2017 at 10:09 PM, Premkumar, Ramprabhu < ramprabhu.premku...@verizon.com> wrote: > Thank you for the response! > > It turns ou

Re: Zookeeper failure handling

2017-09-27 Thread Till Rohrmann
Hi Gyula, if we don't listen to the LeaderLatch#notLeader call but instead wait until we see (via the NodeCache) a new leader information being written to the leader path in order to revoke leadership, then we potentially end up running the same job twice. Even though this can theoretically alread

Re: Zookeeper failure handling

2017-09-27 Thread Gyula Fóra
On a second iteration, the whole problem seems to stem from the fact that we revoke leadership from the JM when the notLeader method is called before waiting for a new leader to be elected. Ideally we should wait until isLeader is called again to check who was the previous leader but I can see how