Re: Kinesis connector shard discovery mechanism

2018-09-13 Thread Ying Xu
+1 on separating out the logic relevant to Kinesalite. Kinesalite is likely used in testing environment a lot. On Thu, Sep 13, 2018 at 1:52 AM, Евгений Юшин wrote: > Hi there > > Flink Kinesis consumer checks shards id for a particular pattern: > "^shardId-\\d{12}" > > https://github.com/apache

[jira] [Created] (FLINK-10344) Rethink SubmittedJobGraphListener

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10344: - Summary: Rethink SubmittedJobGraphListener Key: FLINK-10344 URL: https://issues.apache.org/jira/browse/FLINK-10344 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10345) Rethink SubmittedJobGraphListener

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10345: - Summary: Rethink SubmittedJobGraphListener Key: FLINK-10345 URL: https://issues.apache.org/jira/browse/FLINK-10345 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10343) Expose setCurrentKey method to streamRuntimeContext

2018-09-13 Thread aitozi (JIRA)
aitozi created FLINK-10343: -- Summary: Expose setCurrentKey method to streamRuntimeContext Key: FLINK-10343 URL: https://issues.apache.org/jira/browse/FLINK-10343 Project: Flink Issue Type: Improveme

[jira] [Created] (FLINK-10342) Kafka duplicate topic consumption when topic name is changed

2018-09-13 Thread Oleksandr Nitavskyi (JIRA)
Oleksandr Nitavskyi created FLINK-10342: --- Summary: Kafka duplicate topic consumption when topic name is changed Key: FLINK-10342 URL: https://issues.apache.org/jira/browse/FLINK-10342 Project: F

[jira] [Created] (FLINK-10341) Add option to print flink command when running bin/flink

2018-09-13 Thread Jeff Zhang (JIRA)
Jeff Zhang created FLINK-10341: -- Summary: Add option to print flink command when running bin/flink Key: FLINK-10341 URL: https://issues.apache.org/jira/browse/FLINK-10341 Project: Flink Issue Ty

[jira] [Created] (FLINK-10340) Implement Cosh udf

2018-09-13 Thread Sergey Tsvetkov (JIRA)
Sergey Tsvetkov created FLINK-10340: --- Summary: Implement Cosh udf Key: FLINK-10340 URL: https://issues.apache.org/jira/browse/FLINK-10340 Project: Flink Issue Type: Task Reporte

[jira] [Created] (FLINK-10339) SpillReadBufferPool cannot use off-heap memory

2018-09-13 Thread zhijiang (JIRA)
zhijiang created FLINK-10339: Summary: SpillReadBufferPool cannot use off-heap memory Key: FLINK-10339 URL: https://issues.apache.org/jira/browse/FLINK-10339 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-10338) Use ZooKeeperStateStore in ZooKeeperMesosWorkerStore

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10338: - Summary: Use ZooKeeperStateStore in ZooKeeperMesosWorkerStore Key: FLINK-10338 URL: https://issues.apache.org/jira/browse/FLINK-10338 Project: Flink Issue

[jira] [Created] (FLINK-10337) Use ZooKeeperStateStore in ZooKeeperCompletedCheckpointStore

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10337: - Summary: Use ZooKeeperStateStore in ZooKeeperCompletedCheckpointStore Key: FLINK-10337 URL: https://issues.apache.org/jira/browse/FLINK-10337 Project: Flink

[jira] [Created] (FLINK-10336) Use ZooKeeperStateStore in ZooKeeperSubmittedJobGraphStore

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10336: - Summary: Use ZooKeeperStateStore in ZooKeeperSubmittedJobGraphStore Key: FLINK-10336 URL: https://issues.apache.org/jira/browse/FLINK-10336 Project: Flink

[jira] [Created] (FLINK-10335) Create common ZooKeeperStateStore based on ZooKeeperStateHandleStore and RetrievableStateStorageHelper

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10335: - Summary: Create common ZooKeeperStateStore based on ZooKeeperStateHandleStore and RetrievableStateStorageHelper Key: FLINK-10335 URL: https://issues.apache.org/jira/browse/FLINK

Kinesis connector shard discovery mechanism

2018-09-13 Thread Евгений Юшин
Hi there Flink Kinesis consumer checks shards id for a particular pattern: "^shardId-\\d{12}" https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/model/StreamShardHandle.java#L132 While this inlines with

[jira] [Created] (FLINK-10334) Move RetrievableStateStorageHelper out of ZooKeeperStateHandleStore

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10334: - Summary: Move RetrievableStateStorageHelper out of ZooKeeperStateHandleStore Key: FLINK-10334 URL: https://issues.apache.org/jira/browse/FLINK-10334 Project: Flink

[jira] [Created] (FLINK-10333) Rethink ZooKeeper based stores (SubmittedJobGraph, MesosWorker, CompletedCheckpoints)

2018-09-13 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10333: - Summary: Rethink ZooKeeper based stores (SubmittedJobGraph, MesosWorker, CompletedCheckpoints) Key: FLINK-10333 URL: https://issues.apache.org/jira/browse/FLINK-10333

[jira] [Created] (FLINK-10332) Move data available notification in PipelinedSubpartition out of the synchronized block

2018-09-13 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-10332: --- Summary: Move data available notification in PipelinedSubpartition out of the synchronized block Key: FLINK-10332 URL: https://issues.apache.org/jira/browse/FLINK-10332

[jira] [Created] (FLINK-10331) Fix unnecessary flush requests to the network stack

2018-09-13 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-10331: --- Summary: Fix unnecessary flush requests to the network stack Key: FLINK-10331 URL: https://issues.apache.org/jira/browse/FLINK-10331 Project: Flink Issue Type:

[jira] [Created] (FLINK-10330) Code Compilation failed in local mode

2018-09-13 Thread Shimin Yang (JIRA)
Shimin Yang created FLINK-10330: --- Summary: Code Compilation failed in local mode Key: FLINK-10330 URL: https://issues.apache.org/jira/browse/FLINK-10330 Project: Flink Issue Type: Bug