[Discussion] Query Regarding Operator chaining

2016-07-01 Thread Vinay Patil
Hi, According to the documentation : *"**Each task is executed by one thread ,**Chaining operators together into tasks is a useful optimization: it reduces the overhead of thread-to-thread handover and buffering, and increases overall throughput while decreasing latency"* So does it mean that the

Re: [DISCUSS] Releasing Flink 1.1.0

2016-07-01 Thread Maximilian Michels
Yet another one for the release: FLINK-4144 https://github.com/apache/flink/pull/2191 On Fri, Jul 1, 2016 at 5:18 PM, Maximilian Michels wrote: > There is also FLINK-4141. We just found this during testing. PR is > waiting to be merged here: https://github.com/apache/flink/pull/2190 > > On Fri, J

[jira] [Created] (FLINK-4144) Yarn properties file: replace hostname/port with Yarn application id

2016-07-01 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-4144: - Summary: Yarn properties file: replace hostname/port with Yarn application id Key: FLINK-4144 URL: https://issues.apache.org/jira/browse/FLINK-4144 Project:

Re: Web dashboard binaries and licensing

2016-07-01 Thread Aljoscha Krettek
Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our binary releases. On Fri, 1 Jul 2016 at 18:06 Greg Hogan wrote: > These external libraries will still be shipped in the binaries, which > complicates the issue since source and binary licenses must be tracked > separately.

Re: Web dashboard binaries and licensing

2016-07-01 Thread Greg Hogan
These external libraries will still be shipped in the binaries, which complicates the issue since source and binary licenses must be tracked separately. http://www.apache.org/dev/licensing-howto.html#binary I do agree that this would be a nice improvement to the build system. On Fri, Jul 1, 201

Re: [Discussion] Query regarding Join and Windows

2016-07-01 Thread Aljoscha Krettek
Hi, yes, the window operator is stateful, which means that it will pick up where it left in case of a failure and restore. You're right about the graph, chained operators are shown as one box. Cheers, Aljoscha On Fri, 1 Jul 2016 at 04:52 Vinay Patil wrote: > Hi, > > Just watched the video on R

Re: [Discuss] Why different job's tasks can run in the single process.

2016-07-01 Thread Aljoscha Krettek
I'm not aware of any plans to change this. It would be possible though, to add a mode where a standalone cluster only accepts a single job. On Fri, 1 Jul 2016 at 04:50 Jark Wu wrote: > As standalone mode has the disadvantage that the TaskManager JVM can’t > isolate different jobs. Does we have a

Re: [DISCUSS] Releasing Flink 1.1.0

2016-07-01 Thread Maximilian Michels
There is also FLINK-4141. We just found this during testing. PR is waiting to be merged here: https://github.com/apache/flink/pull/2190 On Fri, Jul 1, 2016 at 3:26 PM, Maximilian Michels wrote: > FLINK-3904 is not Yarn related. Not pressing to fix for this release > and actually a bit tricky to f

Re: Web dashboard binaries and licensing

2016-07-01 Thread Maximilian Michels
Ah, now I see that a compile-time dependency resolution would make sense. Then we don't have to check license compatibility for web dependencies which are downloaded during compile time and are not part of the source distribution. +1 Would be worth the effort to integrate this in our build system,

[jira] [Created] (FLINK-4143) Configurable delimiter for metric identifier

2016-07-01 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-4143: --- Summary: Configurable delimiter for metric identifier Key: FLINK-4143 URL: https://issues.apache.org/jira/browse/FLINK-4143 Project: Flink Issue Type:

[jira] [Created] (FLINK-4142) Recovery problem in HA on Hadoop Yarn 2.5.0

2016-07-01 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4142: - Summary: Recovery problem in HA on Hadoop Yarn 2.5.0 Key: FLINK-4142 URL: https://issues.apache.org/jira/browse/FLINK-4142 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4141) TaskManager failures not always recover when killed during an ApplicationMaster failure in HA mode on Yarn

2016-07-01 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4141: - Summary: TaskManager failures not always recover when killed during an ApplicationMaster failure in HA mode on Yarn Key: FLINK-4141 URL: https://issues.apache.org/jira/browse/FL

[jira] [Created] (FLINK-4140) CheckpointCoordinator fails to discard completed checkpoint

2016-07-01 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4140: - Summary: CheckpointCoordinator fails to discard completed checkpoint Key: FLINK-4140 URL: https://issues.apache.org/jira/browse/FLINK-4140 Project: Flink

Re: [DISCUSS] Releasing Flink 1.1.0

2016-07-01 Thread Maximilian Michels
FLINK-3904 is not Yarn related. Not pressing to fix for this release and actually a bit tricky to fix. I've addressed the other issues and merged all pending pull requests. Good to go from my side. On Fri, Jul 1, 2016 at 11:29 AM, Chesnay Schepler wrote: > There are also 2 fixes for Cassandra tha

[jira] [Created] (FLINK-4139) Yarn: Adjust parallelism and task slots correctly

2016-07-01 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-4139: - Summary: Yarn: Adjust parallelism and task slots correctly Key: FLINK-4139 URL: https://issues.apache.org/jira/browse/FLINK-4139 Project: Flink Iss

Re: [DISCUSS] Releasing Flink 1.1.0

2016-07-01 Thread Chesnay Schepler
There are also 2 fixes for Cassandra that should be included: https://github.com/apache/flink/pull/2167 https://github.com/apache/flink/pull/2183 We should also include the documentation for the metrics stuff (hopefully merged today) https://github.com/apache/flink/pull/2158 In regards to met