Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Jark Wu
Hi Fabian, You are right, the main thing we need to change for removing STREAM keyword is the table registration. If you would like, I can do a prototype. Hi Timo, I’m glad to contribute our work back to Flink. I will look into it and create JIRAs next days. - Jark Wu > 在 2016年8月24日,上午12:

[DISCUSS][FLINK-3414] Scala API for CEP's pattern definition

2016-08-23 Thread Ivan Mushketyk
Hi, Currently CEP supports API for pattern definition, but it is very similar to the Java one. Using Scala we could achieve much more concise API. In JIRA comment for this task I've described a proposal for how this API could look like: [1] Could you please review it and write your comments? Be

[DISCUSS] [FLINK-3950] Meter interface

2016-08-23 Thread Ivan Mushketyk
Hi, I was working on the creating new Meter metric type, but I need some input to define a proper interface for it. Initial idea developed in this pull request [1] was to follow Dropwizard's meter interface, but zentol suggested that we may need a different interface for Flink use-cases. The issue

[jira] [Created] (FLINK-4460) Expose Late Arriving Events

2016-08-23 Thread Chen Qin (JIRA)
Chen Qin created FLINK-4460: --- Summary: Expose Late Arriving Events Key: FLINK-4460 URL: https://issues.apache.org/jira/browse/FLINK-4460 Project: Flink Issue Type: Improvement Components:

[jira] [Created] (FLINK-4459) Introduce SlotProvider to Scheduler

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4459: Summary: Introduce SlotProvider to Scheduler Key: FLINK-4459 URL: https://issues.apache.org/jira/browse/FLINK-4459 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4458) Remove ForkableFlinkMiniCluster

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4458: Summary: Remove ForkableFlinkMiniCluster Key: FLINK-4458 URL: https://issues.apache.org/jira/browse/FLINK-4458 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4457) Make the ExecutionGraph independent of Akka

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4457: Summary: Make the ExecutionGraph independent of Akka Key: FLINK-4457 URL: https://issues.apache.org/jira/browse/FLINK-4457 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-4456) Replace ActorGateway in RuntimeEnvironment by interface

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4456: Summary: Replace ActorGateway in RuntimeEnvironment by interface Key: FLINK-4456 URL: https://issues.apache.org/jira/browse/FLINK-4456 Project: Flink Issue T

[jira] [Created] (FLINK-4455) Replace ActorGateways in NetworkEnvironment by interfaces

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4455: Summary: Replace ActorGateways in NetworkEnvironment by interfaces Key: FLINK-4455 URL: https://issues.apache.org/jira/browse/FLINK-4455 Project: Flink Issue

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Fabian Hueske
Hi Jark, We can think about removing the STREAM keyword or not. In principle, Calcite should allow the same windowing syntax on streaming and static tables (this is one of the main goals of Calcite). The Table API can also distinguish stream and batch without the STREAM keyword by looking at the E

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Timo Walther
Hi Jark, your design document looks very promising. It would be great if you could contribute parts of your implementation back. E.g. UDTFs, UDAFs or even your CROSS APPLY operator. We don't need a FLIP for every little new feature. So feel free to create a Jira issue, discuss it a little bit

[jira] [Created] (FLINK-4454) Lookups for JobManager address in config

2016-08-23 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-4454: - Summary: Lookups for JobManager address in config Key: FLINK-4454 URL: https://issues.apache.org/jira/browse/FLINK-4454 Project: Flink Issue Type:

[jira] [Created] (FLINK-4453) Scala code example in Window documentation shows Java

2016-08-23 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-4453: Summary: Scala code example in Window documentation shows Java Key: FLINK-4453 URL: https://issues.apache.org/jira/browse/FLINK-4453 Project: Flink Issue Typ

[jira] [Created] (FLINK-4452) TaskManager network buffer guages

2016-08-23 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4452: - Summary: TaskManager network buffer guages Key: FLINK-4452 URL: https://issues.apache.org/jira/browse/FLINK-4452 Project: Flink Issue Type: New Feature C

[jira] [Created] (FLINK-4451) Throw exception when remote connection cannot be resolved

2016-08-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4451: Summary: Throw exception when remote connection cannot be resolved Key: FLINK-4451 URL: https://issues.apache.org/jira/browse/FLINK-4451 Project: Flink Issue

[jira] [Created] (FLINK-4450) update storm version to 1.0.0

2016-08-23 Thread yuzhongliu (JIRA)
yuzhongliu created FLINK-4450: - Summary: update storm version to 1.0.0 Key: FLINK-4450 URL: https://issues.apache.org/jira/browse/FLINK-4450 Project: Flink Issue Type: Improvement Compo

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Jark Wu
Hi Fabian, Timo, Sorry for the late response. Regarding Calcite’s StreamSQL syntax, what I concern is only the STREAM keyword and no agg-without-window. Which makes different syntax for streaming and static tables. I don’t think Flink should have a custom SQL syntax, but it’s better to have a

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Ufuk Celebi
On Tue, Aug 23, 2016 at 9:47 AM, Fabian Hueske wrote: > I drafted an API proposal a few months ago [2] and could convert this into > a FLIP to discuss the API and break it down into subtasks. > > What do you think? Sounds very reasonable :) +1

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Fabian Hueske
Hi, I did a bit of prototyping yesterday to check to what extend Calcite supports window operations on streams if we would implement them for the Table API. For the Table API we do not go through Calcite's SQL parser and validator, but generate the logical plan (tree of RelNodes) ourselves mostly

[jira] [Created] (FLINK-4449) Heartbeat Manager between ResourceManager and TaskExecutor

2016-08-23 Thread zhangjing (JIRA)
zhangjing created FLINK-4449: Summary: Heartbeat Manager between ResourceManager and TaskExecutor Key: FLINK-4449 URL: https://issues.apache.org/jira/browse/FLINK-4449 Project: Flink Issue Type: