Re: Apache Flink and Kudu integration

2016-10-27 Thread Márton Balassi
Hi Ruben, I am currently not aware of such an effort, but I definitely do agree that it is an interesting pattern to investigate. As a motivation you could have a look at the Spark connector implementations to see the Kudu APIs in use. For that I would recommend the DataSource API implementation t

[jira] [Created] (FLINK-4953) Allow querying time in window functions

2016-10-27 Thread Manu Zhang (JIRA)
Manu Zhang created FLINK-4953: - Summary: Allow querying time in window functions Key: FLINK-4953 URL: https://issues.apache.org/jira/browse/FLINK-4953 Project: Flink Issue Type: Improvement

Apache Flink and Kudu integration

2016-10-27 Thread ruben.casado.tejedor
Hi all, Is there any PoC about reading/writing from/to Kudu? I think the flow kafka-flink-kudu is an interesting pattern. I would like to evaluate it so please let me know if there is any existing attempt to avoid starting from scratch. Advices are welcomed :) Best --

[jira] [Created] (FLINK-4952) Add Scala API for KeyedStream.flatMap

2016-10-27 Thread Manu Zhang (JIRA)
Manu Zhang created FLINK-4952: - Summary: Add Scala API for KeyedStream.flatMap Key: FLINK-4952 URL: https://issues.apache.org/jira/browse/FLINK-4952 Project: Flink Issue Type: Improvement

Re: DataStream#explain

2016-10-27 Thread Jark Wu
Hi Anton, I think we can ignore the id, do not print operator’s id in the table explain. IMO, the id information is not necessary. - Jark Wu > 在 2016年10月27日,下午8:16,Anton Solovev 写道: > > How can I execute several tests of table explain separately? > While StreamExecutionEnvironment is up,

[jira] [Created] (FLINK-4951) Better Javadocs for KeyedStream.flatMap

2016-10-27 Thread Manu Zhang (JIRA)
Manu Zhang created FLINK-4951: - Summary: Better Javadocs for KeyedStream.flatMap Key: FLINK-4951 URL: https://issues.apache.org/jira/browse/FLINK-4951 Project: Flink Issue Type: Improvement

Re: [HEADS UP] Introducing ConfigOptions

2016-10-27 Thread Jark Wu
+1 The automatic configuration page generating sounds really cool. - Jark Wu > 在 2016年10月28日,上午12:23,Stephan Ewen 写道: > > I think adding descriptions to the options and auto-generating the config > docs is a very good followup. > > How about we do that after we migrated all options? Then we

[jira] [Created] (FLINK-4950) Add support to include multiple Yarn application entries in Yarn properties file

2016-10-27 Thread Vijay Srinivasaraghavan (JIRA)
Vijay Srinivasaraghavan created FLINK-4950: -- Summary: Add support to include multiple Yarn application entries in Yarn properties file Key: FLINK-4950 URL: https://issues.apache.org/jira/browse/FLINK-4950

[jira] [Created] (FLINK-4949) Refactor Gelly driver inputs

2016-10-27 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4949: - Summary: Refactor Gelly driver inputs Key: FLINK-4949 URL: https://issues.apache.org/jira/browse/FLINK-4949 Project: Flink Issue Type: Improvement Compon

[jira] [Created] (FLINK-4948) Consider using checksums or similar to detect bad checkpoints

2016-10-27 Thread Jamie Grier (JIRA)
Jamie Grier created FLINK-4948: -- Summary: Consider using checksums or similar to detect bad checkpoints Key: FLINK-4948 URL: https://issues.apache.org/jira/browse/FLINK-4948 Project: Flink Issu

[jira] [Created] (FLINK-4947) Make all configuration possible via flink-conf.yaml and CLI.

2016-10-27 Thread Jamie Grier (JIRA)
Jamie Grier created FLINK-4947: -- Summary: Make all configuration possible via flink-conf.yaml and CLI. Key: FLINK-4947 URL: https://issues.apache.org/jira/browse/FLINK-4947 Project: Flink Issue

[jira] [Created] (FLINK-4946) Load jar files from subdirectories of lib

2016-10-27 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4946: - Summary: Load jar files from subdirectories of lib Key: FLINK-4946 URL: https://issues.apache.org/jira/browse/FLINK-4946 Project: Flink Issue Type: Improvement

Contributing to flink-web

2016-10-27 Thread Greg Hogan
Should we align the process for contributing to apache/flink-web to mirror that for apache/flink? Flink's JIRA has an existing component for "Project Website". Commits to flink-web are sent to the commits mailing list. Does Jira require further integeration? https://flink.apache.org/improve-websi

[jira] [Created] (FLINK-4945) KafkaConsumer logs wrong warning about confirmation for unknown checkpoint

2016-10-27 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4945: - Summary: KafkaConsumer logs wrong warning about confirmation for unknown checkpoint Key: FLINK-4945 URL: https://issues.apache.org/jira/browse/FLINK-4945 Project: F

Re: [HEADS UP] Introducing ConfigOptions

2016-10-27 Thread Stephan Ewen
I think adding descriptions to the options and auto-generating the config docs is a very good followup. How about we do that after we migrated all options? Then we can add the docs generation logic in that step as well. On Thu, Oct 27, 2016 at 5:22 PM, Robert Metzger wrote: > +1 good idea. > >

Re: [HEADS UP] Introducing ConfigOptions

2016-10-27 Thread Robert Metzger
+1 good idea. I'm wondering whether we can extend this a bit to include a description of the configuration options as well. This way, we can generate the configuration page for the documentation from the code. This makes maintenance for developers really easy. This is also the approach Apache Kafk

[jira] [Created] (FLINK-4944) Replace Akka's deatch watch with own heartbeat on the TM side

2016-10-27 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4944: Summary: Replace Akka's deatch watch with own heartbeat on the TM side Key: FLINK-4944 URL: https://issues.apache.org/jira/browse/FLINK-4944 Project: Flink

[jira] [Created] (FLINK-4943) flink-mesos/ConfigConstants: Typo: YYARN -> YARN

2016-10-27 Thread JIRA
Mischa Krüger created FLINK-4943: Summary: flink-mesos/ConfigConstants: Typo: YYARN -> YARN Key: FLINK-4943 URL: https://issues.apache.org/jira/browse/FLINK-4943 Project: Flink Issue Type: Im

Re: [DISCUSS] Make FieldAccessor logic consistent with remaining API

2016-10-27 Thread Gábor Gévay
Hello, Thanks Fabian for starting this discussion. I would just like to add a few thougths about why does the FieldAccessors even exist. One could say that we should instead just re-use ExpressionKeys for the aggregations, and we are done. As far as I can see, the way ExpressionKeys is often used

Re: [HEADS UP] Introducing ConfigOptions

2016-10-27 Thread Stephan Ewen
Hey! Here is the parent issue (with per component sub-issues) for the option migration. https://issues.apache.org/jira/browse/FLINK-4765 Stephan On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels wrote: > +1 I like it a lot. Much clearer and better maintainable! > > Do we have a plan to mi

[jira] [Created] (FLINK-4942) Improve processing performance of HeapInternalTimerService with key groups

2016-10-27 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4942: - Summary: Improve processing performance of HeapInternalTimerService with key groups Key: FLINK-4942 URL: https://issues.apache.org/jira/browse/FLINK-4942 Project: F

[jira] [Created] (FLINK-4941) Show ship strategy in web interface

2016-10-27 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-4941: - Summary: Show ship strategy in web interface Key: FLINK-4941 URL: https://issues.apache.org/jira/browse/FLINK-4941 Project: Flink Issue Type: Bug

RE: DataStream#explain

2016-10-27 Thread Anton Solovev
How can I execute several tests of table explain separately? While StreamExecutionEnvironment is up, an idCounter in StreamTransformation keeps incrementing and so each test depends on previous ones. -Original Message- From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Thursday, Oct

[jira] [Created] (FLINK-4940) Add Support for Broadcast/Global State

2016-10-27 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4940: --- Summary: Add Support for Broadcast/Global State Key: FLINK-4940 URL: https://issues.apache.org/jira/browse/FLINK-4940 Project: Flink Issue Type: Sub-ta

Re: [HEADS UP] Introducing ConfigOptions

2016-10-27 Thread Maximilian Michels
+1 I like it a lot. Much clearer and better maintainable! Do we have a plan to migrate ConfigConstants to the new ConfigOptions? Where do we maintain a list of all config options? -Max On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen wrote: > Hi all! > > A few weeks back we introduced a new way t

Re: [DISCUSS] FLIP-13: Consistent Processing with Loops

2016-10-27 Thread Stephan Ewen
How about we break this up into two FLIPs? There are after all two orthogonal problems (termination, fault tolerance) with quite different discussion states. Concerning fault tolerance, I like the ideas. For the termination proposal, I would like to iterate a bit more. *Termination algorithm:* M

[jira] [Created] (FLINK-4939) GenericWriteAheadSink: Decouple the subtask that created a pending checkpoint from the one that commits it.

2016-10-27 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-4939: - Summary: GenericWriteAheadSink: Decouple the subtask that created a pending checkpoint from the one that commits it. Key: FLINK-4939 URL: https://issues.apache.org/jira/browse/F

Re: [DISCUSS] FLIP-13: Consistent Processing with Loops

2016-10-27 Thread Stephan Ewen
Hi! I am still scanning it and compiling some comments. Give me a bit ;-) Stephan On Wed, Oct 26, 2016 at 6:13 PM, Paris Carbone wrote: > Hey all, > > Now that many of you have already scanned the document (judging from the > views) maybe it is time to give back some feedback! > Did you like

Re: [Discuss] FLIP-13 Side Outputs in Flink

2016-10-27 Thread Fabian Hueske
Hi CPC, I agree, support for side outputs would be nice for DataSet as well. However, this is not easily possible because it would require an extensive rewrite of the DataSet optimizer. IMO, that's out of scope for this proposal. Cheers, Fabian 2016-10-27 0:29 GMT+02:00 CPC : > Is it just relat