Re: Questions of "State Processing API in Scala"

2020-01-20 Thread Tzu-Li (Gordon) Tai
Hi Izual, Thanks for reporting this! I'm also forwarding this to the user mailing list, as that is the more suitable place for this question. I think the usability of the State Processor API in Scala is indeed something that hasn’t been looked at closely yet. On Tue, Jan 21, 2020 at 8:12 AM izua

[jira] [Created] (FLINK-15704) Kafka09ITCase.testBrokerFailure fails due to time out on travis

2020-01-20 Thread Yu Li (Jira)
Yu Li created FLINK-15704: - Summary: Kafka09ITCase.testBrokerFailure fails due to time out on travis Key: FLINK-15704 URL: https://issues.apache.org/jira/browse/FLINK-15704 Project: Flink Issue Type

Questions of "State Processing API in Scala"

2020-01-20 Thread izual
Hi community, When I use state in Scala, something makes confused, I followed these steps to generate and read states: a. implements the example[1] `CountWindowAverage` in Scala(exactly same), and run jobA => that makes good. b. execute `flink cancel -s ${JobID}` => savepoints was generated as

[jira] [Created] (FLINK-15703) FlinkKafkaConsumer should enable partition discovery by default.

2020-01-20 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-15703: Summary: FlinkKafkaConsumer should enable partition discovery by default. Key: FLINK-15703 URL: https://issues.apache.org/jira/browse/FLINK-15703 Project: Flink

Re: [VOTE] FLIP-27 - Refactor Source Interface

2020-01-20 Thread Becket Qin
Hi Folks, I'd like to resume the voting thread for FlIP-27. Please note that the FLIP wiki has been updated to reflect the latest discussions in the discussion thread. To avoid confusion, I'll only count the votes casted after this point. FLIP wiki: https://cwiki.apache.org/confluence/display/F

[jira] [Created] (FLINK-15702) Make sqlClient classloader aligned with other components

2020-01-20 Thread liupengcheng (Jira)
liupengcheng created FLINK-15702: Summary: Make sqlClient classloader aligned with other components Key: FLINK-15702 URL: https://issues.apache.org/jira/browse/FLINK-15702 Project: Flink Issu

[jira] [Created] (FLINK-15701) Allows transfer.sh to retry when fails to uploads logs.

2020-01-20 Thread Yun Gao (Jira)
Yun Gao created FLINK-15701: --- Summary: Allows transfer.sh to retry when fails to uploads logs. Key: FLINK-15701 URL: https://issues.apache.org/jira/browse/FLINK-15701 Project: Flink Issue Type: Imp

[jira] [Created] (FLINK-15700) Improve Python API Tutorial doc

2020-01-20 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-15700: Summary: Improve Python API Tutorial doc Key: FLINK-15700 URL: https://issues.apache.org/jira/browse/FLINK-15700 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-15699) FirstValueAggFunctionWithoutOrderTest failed on Traivs

2020-01-20 Thread Congxian Qiu(klion26) (Jira)
Congxian Qiu(klion26) created FLINK-15699: - Summary: FirstValueAggFunctionWithoutOrderTest failed on Traivs Key: FLINK-15699 URL: https://issues.apache.org/jira/browse/FLINK-15699 Project: Flin

[jira] [Created] (FLINK-15698) Restructure Configuration Docs to separate "Getting Started", "Common Operations" and "Tuning/Debugging"

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15698: Summary: Restructure Configuration Docs to separate "Getting Started", "Common Operations" and "Tuning/Debugging" Key: FLINK-15698 URL: https://issues.apache.org/jira/browse/FLINK

[jira] [Created] (FLINK-15697) Remove Options that are not configured in "flink-conf.yaml" from Configuration Docs

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15697: Summary: Remove Options that are not configured in "flink-conf.yaml" from Configuration Docs Key: FLINK-15697 URL: https://issues.apache.org/jira/browse/FLINK-15697 P

[jira] [Created] (FLINK-15696) Update description of "RocksDB Configurable Options" for new options

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15696: Summary: Update description of "RocksDB Configurable Options" for new options Key: FLINK-15696 URL: https://issues.apache.org/jira/browse/FLINK-15696 Project: Flink

[jira] [Created] (FLINK-15695) Remove outdated section of Configuration Page

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15695: Summary: Remove outdated section of Configuration Page Key: FLINK-15695 URL: https://issues.apache.org/jira/browse/FLINK-15695 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-15694) Remove HDFS Section of Configuration Page

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15694: Summary: Remove HDFS Section of Configuration Page Key: FLINK-15694 URL: https://issues.apache.org/jira/browse/FLINK-15694 Project: Flink Issue Type: Sub-tas

Join of three streams with different frequency

2020-01-20 Thread Dominik Wosiński
Hello, I wanted to ask whether the idea and the general concept that I have is correct or if there is anything better in Flink to use. Say, I that I have 3 different streams : - currency_codes, which has the name of the currency. It has the following fields (*currency_iso_code, tst, curren

[jira] [Created] (FLINK-15693) Stop receiving incoming RPC messages when RpcEndpoint is closing

2020-01-20 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-15693: - Summary: Stop receiving incoming RPC messages when RpcEndpoint is closing Key: FLINK-15693 URL: https://issues.apache.org/jira/browse/FLINK-15693 Project: Flink

[jira] [Created] (FLINK-15692) Enable limiting RocksDB memory consumption by default

2020-01-20 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-15692: - Summary: Enable limiting RocksDB memory consumption by default Key: FLINK-15692 URL: https://issues.apache.org/jira/browse/FLINK-15692 Project: Flink Issue

[jira] [Created] (FLINK-15691) TaskExecutorPartitionLifecycleTest#runInTaskExecutorThreadAndWait does not run in main thread

2020-01-20 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-15691: - Summary: TaskExecutorPartitionLifecycleTest#runInTaskExecutorThreadAndWait does not run in main thread Key: FLINK-15691 URL: https://issues.apache.org/jira/browse/FLINK-15691

[jira] [Created] (FLINK-15690) Enable configuring ExecutionConfig & CheckpointConfig from flink-conf.yaml

2020-01-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-15690: Summary: Enable configuring ExecutionConfig & CheckpointConfig from flink-conf.yaml Key: FLINK-15690 URL: https://issues.apache.org/jira/browse/FLINK-15690 Pr

[jira] [Created] (FLINK-15689) Pass execution configuration from BatchTableEnvironment to ExecutionEnvironment

2020-01-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-15689: Summary: Pass execution configuration from BatchTableEnvironment to ExecutionEnvironment Key: FLINK-15689 URL: https://issues.apache.org/jira/browse/FLINK-15689

Re: [VOTE] FLIP-92: Add N-Ary Stream Operator in Flink

2020-01-20 Thread Piotr Nowojski
Thank you all for the votes. So far we have received 7 approving votes, 2 of which are binding and there is no -1 votes: * Nico (binding) * Zhijiang (binding) * Zhenghua (non-binding) * Yun (non-binding) * Haibo (non-binding) * Jingsong (non-binding) * Arvid (non-binding) Therefore, I'm happy to

[jira] [Created] (FLINK-15688) Add N-Ary Stream Operator in Flink

2020-01-20 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-15688: -- Summary: Add N-Ary Stream Operator in Flink Key: FLINK-15688 URL: https://issues.apache.org/jira/browse/FLINK-15688 Project: Flink Issue Type: New Featur

Re: [VOTE] FLIP-92: Add N-Ary Stream Operator in Flink

2020-01-20 Thread Nico Kruber
+1 (binding) Aside the mentioned use cases in SQL/Batch, I also see advantages in streaming where joining multiple streams together seems to be quite common, too. For that, however, we would need the addition in the DataStream API... but agreed it could be done in a second step Nico On 14/01/20

[jira] [Created] (FLINK-15687) Potential test instabilities due to concurrent access to TaskSlotTable.

2020-01-20 Thread Kostas Kloudas (Jira)
Kostas Kloudas created FLINK-15687: -- Summary: Potential test instabilities due to concurrent access to TaskSlotTable. Key: FLINK-15687 URL: https://issues.apache.org/jira/browse/FLINK-15687 Project:

Re: BroadcastState enforce processing

2020-01-20 Thread Yun Gao
Hi Dominik, If I understand the scenario correctly, I think one possible solution is to implement one specified TwoInputStreamOperator directly, and also implements the InputSelectable interface. Then this operator could control the priority its two inputs by return proper InputSele

BroadcastState enforce processing

2020-01-20 Thread Dominik Wosiński
Hey, I have a question since I have observed the following situation: We have two streams A and B that will be read from Kafka. Let's say we have a set of rules for processing that is stored in A and B is the stream that we will process. Since there is no guarantee that elements from A will be pr

[jira] [Created] (FLINK-15686) SELECT 'ABC'; does not work in sql-client

2020-01-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-15686: Summary: SELECT 'ABC'; does not work in sql-client Key: FLINK-15686 URL: https://issues.apache.org/jira/browse/FLINK-15686 Project: Flink Issue Type:

[jira] [Created] (FLINK-15685) org.apache.flink.tests.util.kafka.SQLClientKafkaITCase failed on traivs

2020-01-20 Thread Congxian Qiu(klion26) (Jira)
Congxian Qiu(klion26) created FLINK-15685: - Summary: org.apache.flink.tests.util.kafka.SQLClientKafkaITCase failed on traivs Key: FLINK-15685 URL: https://issues.apache.org/jira/browse/FLINK-15685

[jira] [Created] (FLINK-15684) Add "taskmanager.memory.flink.size" to the common options

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15684: Summary: Add "taskmanager.memory.flink.size" to the common options Key: FLINK-15684 URL: https://issues.apache.org/jira/browse/FLINK-15684 Project: Flink Iss

[jira] [Created] (FLINK-15683) Restructure Documentation Docs

2020-01-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-15683: Summary: Restructure Documentation Docs Key: FLINK-15683 URL: https://issues.apache.org/jira/browse/FLINK-15683 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-15682) Flink native kubernetes Could not run

2020-01-20 Thread yuzhaojing (Jira)
yuzhaojing created FLINK-15682: -- Summary: Flink native kubernetes Could not run Key: FLINK-15682 URL: https://issues.apache.org/jira/browse/FLINK-15682 Project: Flink Issue Type: Bug C

[jira] [Created] (FLINK-15681) Remove legacy ExecutionAndAllocationFuture class

2020-01-20 Thread vinoyang (Jira)
vinoyang created FLINK-15681: Summary: Remove legacy ExecutionAndAllocationFuture class Key: FLINK-15681 URL: https://issues.apache.org/jira/browse/FLINK-15681 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-15680) testIsAvailableOrNotAfterRequestAndRecycleMultiSegments timed out

2020-01-20 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-15680: -- Summary: testIsAvailableOrNotAfterRequestAndRecycleMultiSegments timed out Key: FLINK-15680 URL: https://issues.apache.org/jira/browse/FLINK-15680 Project: Flink