[jira] [Created] (FLINK-14310) Get ExecutionVertexID from ExecutionVertex rather than creating new instances

2019-10-01 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-14310: --- Summary: Get ExecutionVertexID from ExecutionVertex rather than creating new instances Key: FLINK-14310 URL: https://issues.apache.org/jira/browse/FLINK-14310 Project: Flink

[jira] [Created] (FLINK-14309) Kafka09ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink fails on Travis

2019-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-14309: - Summary: Kafka09ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink fails on Travis Key: FLINK-14309 URL: https://issues.apache.org/jira/browse/FLINK-14309

[jira] [Created] (FLINK-14308) Kafka09ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator fails on Travis

2019-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-14308: - Summary: Kafka09ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator fails on Travis Key: FLINK-14308 URL: https://issues.apache.org/jira/browse/FLINK-143

Re: [ANNOUNCE] Progress of Apache Flink 1.10 #1

2019-10-01 Thread Bowen Li
Thanks Yu and Gary for the detailed summary and update! On Fri, Sep 27, 2019 at 6:54 AM Yu Li wrote: > Hi community, > > Since we are now more than one month into the Flink 1.10 release cycle, we > thought it would be adequate to give a progress update. Below we have > included a list of the ong

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-01 Thread Xuefu Z
Here are some of my thoughts on the minor debates above: 1. +1 for 4 categories of functions. They are categorized along two dimensions of binary values: X: *temporary* vs non-temporary (persistent); Y: *system* vs non-system (so said catalog). 2. In my opinion, class functionIdentifier doesn't re

[CODE STYLE] Parameters of method are always final

2019-10-01 Thread Zili Chen
Hi devs, Coming from this discussion[1] I'd like to propose that in Flink codebase we suggest a code style that parameters of method are always final. Almost everywhere parameters of method are final already and if we have such consensus we can prevent redundant final modifier in method declaratio

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-01 Thread Bowen Li
If something like the yaml file is the way to go and achieve such motivation, we would cover that with current design. On Tue, Oct 1, 2019 at 12:05 Bowen Li wrote: > Hi Timo, Dawid, > > I've added the suggested SQL and related changes to TableEnvironment API > and other classes to the google doc

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-01 Thread Bowen Li
Hi Timo, Dawid, I've added the suggested SQL and related changes to TableEnvironment API and other classes to the google doc. Also removed "USE MODULE" and its APIs. Will update FLIP wiki once we have a consensus. W.r.t. descriptor approach, my gut feeling is similar to Dawid's. Besides, I feel y

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-01 Thread Bowen Li
Hi Dawid, Thanks for bringing the suggestions up. I was prototyping yesterday and found out those places exactly as what you suggested. For CallExpression and UnresolvedCallExpression, I've added them to FLIP-57. We will replace ObjectIdentifier with FunctionIdentifier and mark that as a breaking

ApacheCon North America 2020, project participation

2019-10-01 Thread Rich Bowen
Hi, folks, (Note: You're receiving this email because you're on the dev@ list for one or more Apache Software Foundation projects.) For ApacheCon North America 2019, we asked projects to participate in the creation of project/topic specific tracks. This was very successful, with about 15 projects

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2019-10-01 Thread Thomas Weise
Thanks for putting together this proposal! I see that the "Per Split Event Time" and "Event Time Alignment" sections are still TBD. It would probably be good to flesh those out a bit before proceeding too far as the event time alignment will probably influence the interaction with the split reade

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Konstantinos Kallas
Hi Piotrek, Thank you very much for your feedback. I am mostly interested in bugs in Flink applications. Especially ones that are hard to notice -- either because they don't always occur, or because they don't crash the program, but instead they subtly affect its output, or lead to deadlocks.

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-01 Thread Dawid Wysakowicz
Hi Timo, Bowen, Unfortunately I did not have enough time to go through all the suggestions in details so I can not comment on the whole FLIP. I just wanted to give my opinion on the "descriptor approach in loadModule" part. I am not sure if we need it here. We might be overthinking this a bit. It

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-01 Thread Fabian Hueske
Thanks for the summary Bowen! Looks good to me. Cheers, Fabian Am Mo., 30. Sept. 2019 um 23:24 Uhr schrieb Bowen Li : > Hi all, > > I've updated the FLIP wiki with the following changes: > > - Lifespan of temp functions are not tied to those of catalogs and > databases. Users can create temp fu

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-01 Thread Dawid Wysakowicz
Hi, I think we're really close to a very good design for the functions. Thank you to all involved in the discussion and especially Bowen for driving the FLIP. The last part I'm missing in the FLIP is a bit more elaborate description of the FunctionIdentifier. I gave it a bit more thought and disc

[jira] [Created] (FLINK-14307) Extract JobGraphWriter from JobGraphStore

2019-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-14307: - Summary: Extract JobGraphWriter from JobGraphStore Key: FLINK-14307 URL: https://issues.apache.org/jira/browse/FLINK-14307 Project: Flink Issue Type: Impro

[jira] [Created] (FLINK-14306) flink-python build fails with No module named pkg_resources

2019-10-01 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-14306: -- Summary: flink-python build fails with No module named pkg_resources Key: FLINK-14306 URL: https://issues.apache.org/jira/browse/FLINK-14306 Project: Flink

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Jan Lukavský
Hi, I'd add another one regarding Java hashCode() and its practical usability for distributed systems [1], although practically all (Java based) data processing systems rely on it. One bug directly related to this I once saw was, that using an Enum inside other object used as partitioning ke

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Piotr Nowojski
Hi, Are you asking about bugs in Flink, in libraries that Flink is using or bugs in applications that were using Flink? From my perspective/what I have seen: The most problematic bugs while developing features for Flink: Dead locks & data losses caused by concurrency issues in network s

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-01 Thread Timo Walther
Hi Bowen, thanks for your response. Re 2) I also don't have a better approach for this issue. It is similar to changing the general TableConfig between two statements. It would be good to add your explanation to the design document. Re 3) It would be interesting to know about which "core" fu

[jira] [Created] (FLINK-14305) Move ownership of JobManagerMetricGroup to Dispatcher

2019-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-14305: - Summary: Move ownership of JobManagerMetricGroup to Dispatcher Key: FLINK-14305 URL: https://issues.apache.org/jira/browse/FLINK-14305 Project: Flink Issue

[jira] [Created] (FLINK-14304) Avoid task starvation with mailbox

2019-10-01 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-14304: --- Summary: Avoid task starvation with mailbox Key: FLINK-14304 URL: https://issues.apache.org/jira/browse/FLINK-14304 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14303) Factor resource manager related metrics out of JobManagerMetricGroup

2019-10-01 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-14303: - Summary: Factor resource manager related metrics out of JobManagerMetricGroup Key: FLINK-14303 URL: https://issues.apache.org/jira/browse/FLINK-14303 Project: Flink

[jira] [Created] (FLINK-14302) FlinkKafkaInternalProducer should not send `ADD_PARTITIONS_TO_TXN` request if `newPartitionsInTransaction` is empty when enable EoS

2019-10-01 Thread Wei-Che Wei (Jira)
Wei-Che Wei created FLINK-14302: --- Summary: FlinkKafkaInternalProducer should not send `ADD_PARTITIONS_TO_TXN` request if `newPartitionsInTransaction` is empty when enable EoS Key: FLINK-14302 URL: https://issues.ap