[jira] [Created] (FLINK-2827) Potential resource leak in TwitterSource#loadAuthenticationProperties()

2015-10-06 Thread Ted Yu (JIRA)
Ted Yu created FLINK-2827: - Summary: Potential resource leak in TwitterSource#loadAuthenticationProperties() Key: FLINK-2827 URL: https://issues.apache.org/jira/browse/FLINK-2827 Project: Flink Issu

[jira] [Created] (FLINK-2826) transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking

2015-10-06 Thread Ted Yu (JIRA)
Ted Yu created FLINK-2826: - Summary: transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking Key: FLINK-2826 URL: https://issues.apache.org/jira/browse/FLINK-2826

[jira] [Created] (FLINK-2825) FlinkClient.killTopology fails due to missing leader session ID

2015-10-06 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2825: -- Summary: FlinkClient.killTopology fails due to missing leader session ID Key: FLINK-2825 URL: https://issues.apache.org/jira/browse/FLINK-2825 Project: Flink

[jira] [Created] (FLINK-2824) Iteration feedback partitioning does not work as expected

2015-10-06 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-2824: - Summary: Iteration feedback partitioning does not work as expected Key: FLINK-2824 URL: https://issues.apache.org/jira/browse/FLINK-2824 Project: Flink Issue Type:

[jira] [Created] (FLINK-2823) YARN client should report a proper exception if Hadoop Env variables are not set

2015-10-06 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2823: --- Summary: YARN client should report a proper exception if Hadoop Env variables are not set Key: FLINK-2823 URL: https://issues.apache.org/jira/browse/FLINK-2823 Project:

Re: Failing test

2015-10-06 Thread Till Rohrmann
If there is none yet, then we do. Label it with "test-stability". I think the consensus was also to mark it as critical. Otherwise, just add the log to the JIRA. On Tue, Oct 6, 2015 at 2:57 PM, Matthias J. Sax wrote: > Hi, > > One test just failed on current master: > https://travis-ci.org/apac

Failing test

2015-10-06 Thread Matthias J. Sax
Hi, One test just failed on current master: https://travis-ci.org/apache/flink/jobs/83871008 Do we need a JIRA? > LeaderChangeStateCleanupTest.testReelectionOfSameJobManager:245 » Timeout > Futu... -Matthias signature.asc Description: OpenPGP digital signature

[jira] [Created] (FLINK-2822) Windowing classes incorrectly import scala.Serializable

2015-10-06 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-2822: - Summary: Windowing classes incorrectly import scala.Serializable Key: FLINK-2822 URL: https://issues.apache.org/jira/browse/FLINK-2822 Project: Flink Issue Type: B

Re: Towards Flink 0.10

2015-10-06 Thread Maximilian Michels
@Stephan: There is an issue for implementing a detached job submission: https://issues.apache.org/jira/browse/FLINK-2797. That has the advantage that the execution mode and control flow is not hard-coded in the user job. I think it makes more sense to integrate methods like (cancel/getAccumulators/

Re: Towards Flink 0.10

2015-10-06 Thread Alexey Sapozhnikov
Hello everyone. Stephan, will 0.10 include the cache issues for createRemoteEnvironment? That not every addressing of environment will retransmit all the jars? On Tue, Oct 6, 2015 at 1:16 PM, Stephan Ewen wrote: > How about making a quick effort for this: > https://issues.apache.org/jira/browse

Re: Towards Flink 0.10

2015-10-06 Thread Stephan Ewen
How about making a quick effort for this: https://issues.apache.org/jira/browse/FLINK-2313 It introduces an execution control handle (returned by StreamExecutionEnvironment.execute()) for operations like cancel(), getAccumulators(), scaleIn/Out(), ... It would be big time API breaking, so would be

Re: Iteration feedback partitioning does not work properly

2015-10-06 Thread Gyula Fóra
Hi, This is just a workaround, which actually breaks input order from my source. I think the iteration construction should be reworked to set the parallelism of the source/sink to the parallelism of the head operator (and validate that all heads have the same parallelism). I thought this was the

Re: streaming GroupBy + Fold

2015-10-06 Thread Martin Neumann
The window is actually part of the workaround we currently using (should have commented it out) where we use a window and a MapFunction instead of a Fold. Original I was running fold without a window facing the same problems. The workaround works for now so there is no urgency on that one. I just

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-06 Thread Theodore Vasiloudis
One problem that we are seeing with FlinkML PRs is that there are simply not enough commiters to "shepherd" all of them. While I think this process would help generally, I don't think it would solve this kind of problem. Regards, Theodore On Mon, Oct 5, 2015 at 3:28 PM, Matthias J. Sax wrote:

Re: Iteration feedback partitioning does not work properly

2015-10-06 Thread Aljoscha Krettek
Hi, I think what you would like to to can be achieved by: IterativeStream it = in.map(IdentityMap).setParallelism(2).iterate() DataStream mapped = it.map(...) it.closeWith(mapped.partitionByHash(someField)) The input is rebalanced to the map inside the iteration as in your example and the feedba