[jira] [Created] (FLINK-8857) HBase connector read example throws exception at the end.

2018-03-05 Thread Xu Zhang (JIRA)
Xu Zhang created FLINK-8857: --- Summary: HBase connector read example throws exception at the end. Key: FLINK-8857 URL: https://issues.apache.org/jira/browse/FLINK-8857 Project: Flink Issue Type: Bug

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-05 Thread Tzu-Li (Gordon) Tai
Btw, could you also open a JIRA to track this improvement? Thanks! On 5 March 2018 at 5:47:09 PM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) wrote: Hi, +1, I think the new ListShards API should provide all the information we need. In the past, we were restricted to update the SDK version becaus

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-05 Thread Tzu-Li (Gordon) Tai
Hi, +1, I think the new ListShards API should provide all the information we need. In the past, we were restricted to update the SDK version because it would break state compatibility (the consumer used to write AWS classes inside state). I think this was fixed since Flink 1.3, so it should be s

[jira] [Created] (FLINK-8858) SQL Client to submit long running query in file

2018-03-05 Thread Renjie Liu (JIRA)
Renjie Liu created FLINK-8858: - Summary: SQL Client to submit long running query in file Key: FLINK-8858 URL: https://issues.apache.org/jira/browse/FLINK-8858 Project: Flink Issue Type: New Featu

[jira] [Created] (FLINK-8859) RocksDB backend should pass WriteOption to Rocks.put() when restoring

2018-03-05 Thread Sihua Zhou (JIRA)
Sihua Zhou created FLINK-8859: - Summary: RocksDB backend should pass WriteOption to Rocks.put() when restoring Key: FLINK-8859 URL: https://issues.apache.org/jira/browse/FLINK-8859 Project: Flink

[jira] [Created] (FLINK-8860) SlotManager spamming log files

2018-03-05 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8860: -- Summary: SlotManager spamming log files Key: FLINK-8860 URL: https://issues.apache.org/jira/browse/FLINK-8860 Project: Flink Issue Type: Bug Components

Re: StreamSQL queriable state

2018-03-05 Thread Timo Walther
Hi Renjie, it would be great if we can reuse my QueryableStateTableSink implementation. As far as I know, the queryable state interfaces changed a bit in the last months and a new queryable state client as been added. So my branch needs to be rebased to the current APIs. We don't need a big

Re: [DISCUSS] Convert main Table API classes into traits

2018-03-05 Thread Timo Walther
Hi everyone, @Rong: Yes, that's what I meant. Package private and protected modifiers and fields will be moved to internal implementation. The hierachy would look like: trait TableEnvironment {   def fromTableSource(source: TableSource[_]): Table   def registerExternalCatalog(name: String, ex

[jira] [Created] (FLINK-8861) Add support for batch queries in SQL Client

2018-03-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8861: --- Summary: Add support for batch queries in SQL Client Key: FLINK-8861 URL: https://issues.apache.org/jira/browse/FLINK-8861 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8862) Support HBase snapshot read

2018-03-05 Thread Xu Zhang (JIRA)
Xu Zhang created FLINK-8862: --- Summary: Support HBase snapshot read Key: FLINK-8862 URL: https://issues.apache.org/jira/browse/FLINK-8862 Project: Flink Issue Type: Improvement Components:

[jira] [Created] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8863: --- Summary: Add user-defined function support in SQL Client Key: FLINK-8863 URL: https://issues.apache.org/jira/browse/FLINK-8863 Project: Flink Issue Type: Sub-t

[jira] [Created] (FLINK-8864) Add CLI query history

2018-03-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8864: --- Summary: Add CLI query history Key: FLINK-8864 URL: https://issues.apache.org/jira/browse/FLINK-8864 Project: Flink Issue Type: Sub-task Components:

[jira] [Created] (FLINK-8865) Add CLI query code completion in SQL Client

2018-03-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8865: --- Summary: Add CLI query code completion in SQL Client Key: FLINK-8865 URL: https://issues.apache.org/jira/browse/FLINK-8865 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8866) Create unified interfaces to configure and instatiate TableSinks

2018-03-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-8866: --- Summary: Create unified interfaces to configure and instatiate TableSinks Key: FLINK-8866 URL: https://issues.apache.org/jira/browse/FLINK-8866 Project: Flink

[jira] [Created] (FLINK-8867) Rocksdb checkpointing failing with fs.default-scheme: hdfs:// config

2018-03-05 Thread Shashank Agarwal (JIRA)
Shashank Agarwal created FLINK-8867: --- Summary: Rocksdb checkpointing failing with fs.default-scheme: hdfs:// config Key: FLINK-8867 URL: https://issues.apache.org/jira/browse/FLINK-8867 Project: Fli

[jira] [Created] (FLINK-8868) Support Table Function as Table

2018-03-05 Thread Ruidong Li (JIRA)
Ruidong Li created FLINK-8868: - Summary: Support Table Function as Table Key: FLINK-8868 URL: https://issues.apache.org/jira/browse/FLINK-8868 Project: Flink Issue Type: Improvement Com

[jira] [Created] (FLINK-8869) Kafka restore from checkpoint without react to the new add partition to kafka server

2018-03-05 Thread aitozi (JIRA)
aitozi created FLINK-8869: - Summary: Kafka restore from checkpoint without react to the new add partition to kafka server Key: FLINK-8869 URL: https://issues.apache.org/jira/browse/FLINK-8869 Project: Flink

[jira] [Created] (FLINK-8870) End-to-end tests wrongly pass if md5sum is not installed

2018-03-05 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-8870: -- Summary: End-to-end tests wrongly pass if md5sum is not installed Key: FLINK-8870 URL: https://issues.apache.org/jira/browse/FLINK-8870 Project: Flink Is

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-05 Thread Thomas Weise
On Sun, Mar 4, 2018 at 11:38 PM, Bowen Li wrote: > If ListShards() gives all the info that Flink needs, +1 on switching. > DescribeStreams() has a limitation of 5 requests/sec, which is pretty > bad > > But, I believe the goal of switching APIs should be *making Flink jobs that > read from Ki

[jira] [Created] (FLINK-8872) Yarn detached mode via -yd does not detach

2018-03-05 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8872: -- Summary: Yarn detached mode via -yd does not detach Key: FLINK-8872 URL: https://issues.apache.org/jira/browse/FLINK-8872 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

2018-03-05 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-8871: - Summary: Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager Key: FLINK-8871 URL: https://issues.apache.org/jira/browse/FLINK-8871

[jira] [Created] (FLINK-8873) move unit tests of KeyedStream.scala from DataStreamTest.scala to KeyedStreamTest.scala

2018-03-05 Thread Bowen Li (JIRA)
Bowen Li created FLINK-8873: --- Summary: move unit tests of KeyedStream.scala from DataStreamTest.scala to KeyedStreamTest.scala Key: FLINK-8873 URL: https://issues.apache.org/jira/browse/FLINK-8873 Project:

[jira] [Created] (FLINK-8874) rewrite Flink docs/dev/stream/operators/process_function.md to recommend using KeyedProcessFunction

2018-03-05 Thread Bowen Li (JIRA)
Bowen Li created FLINK-8874: --- Summary: rewrite Flink docs/dev/stream/operators/process_function.md to recommend using KeyedProcessFunction Key: FLINK-8874 URL: https://issues.apache.org/jira/browse/FLINK-8874

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-05 Thread Bowen Li
Good for AWS has increased the limit from 5txn/s last year to 10txn/s On Mon, Mar 5, 2018 at 8:32 AM, Thomas Weise wrote: > On Sun, Mar 4, 2018 at 11:38 PM, Bowen Li wrote: > > > If ListShards() gives all the info that Flink needs, +1 on switching. > > DescribeStreams() has a limitation of 5 re

Re: StreamSQL queriable state

2018-03-05 Thread Renjie Liu
Hi, Timo: https://docs.google.com/document/d/18XbrFuwzzRzHR84a48j17BIGpTndSS1q-shSgCBEd8I/edit?usp=sharing is a small design doc for the implementation. I agree with you to put SQL client integration in another PR and focus on the sink implementation in this PR. On Mon, Mar 5, 2018 at 9:33 PM Timo