Re: Question About "Preserve Partitioning" in Stream Iteration

2015-08-03 Thread Aljoscha Krettek
Yes, that's what I was proposing in my second mail: I thought about having some tighter restrictions here. My idea was to enforce that the feedback edges must have the same parallelism as the original input stream, otherwise shipping strategies such as "keyBy", "shuffle", "rebalance" don't seem to

[jira] [Created] (FLINK-2465) SocketClientSink closeConnection function has memory leak

2015-08-03 Thread fangfengbin (JIRA)
fangfengbin created FLINK-2465: -- Summary: SocketClientSink closeConnection function has memory leak Key: FLINK-2465 URL: https://issues.apache.org/jira/browse/FLINK-2465 Project: Flink Issue Typ

Re: Question About "Preserve Partitioning" in Stream Iteration

2015-08-03 Thread Gyula Fóra
It is critical for many applications (such as SAMOA or Storm compatibility) to build arbitrary cyclic flows. If your suggestion covers all cases (for instance nested iterations) then I am not against it. The current implementation is just one way to do it, but it allows arbitrary cycles. From the

Re: Question About "Preserve Partitioning" in Stream Iteration

2015-08-03 Thread Stephan Ewen
I don't think there is a fundamental limitation to the simpler approach. The only real difference is that DOPs are adjusted before the tail, so only one head/tail pair is needed. Nested iterations should still be possible... On Mon, Aug 3, 2015 at 10:21 AM, Gyula Fóra wrote: > It is critical fo

Re: Question About "Preserve Partitioning" in Stream Iteration

2015-08-03 Thread Gyula Fóra
Okay, sounds reasonable :) Stephan Ewen ezt írta (időpont: 2015. aug. 3., H, 10:24): > I don't think there is a fundamental limitation to the simpler approach. > The only real difference is that DOPs are adjusted before the tail, so only > one head/tail pair is needed. > > Nested iterations shou

[jira] [Created] (FLINK-2466) Travis build failure

2015-08-03 Thread Sachin Goel (JIRA)
Sachin Goel created FLINK-2466: -- Summary: Travis build failure Key: FLINK-2466 URL: https://issues.apache.org/jira/browse/FLINK-2466 Project: Flink Issue Type: Bug Reporter: Sachin G

Renaming log file?

2015-08-03 Thread Matthias J. Sax
Hi, I think that the log files are not named consistently (and it annoys me every single time). -> flink-mjsax-jobmanager-log -> flink-mjsax-jobmanager-out -> flink-mjsax-taskmanager-log -> flink-mjsax-taskmanager-out and -> flink-mjsax-flink-client-log -> flink-mjsax-flink-

Re: Renaming log file?

2015-08-03 Thread Stephan Ewen
Sounds fair to rename them... On Mon, Aug 3, 2015 at 11:46 AM, Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote: > Hi, > > I think that the log files are not named consistently (and it annoys me > every single time). > > -> flink-mjsax-jobmanager-log > -> flink-mjsax-jobmanager-out

[jira] [Created] (FLINK-2467) Example WordCountStorm.jar in not packaged correclty

2015-08-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2467: -- Summary: Example WordCountStorm.jar in not packaged correclty Key: FLINK-2467 URL: https://issues.apache.org/jira/browse/FLINK-2467 Project: Flink Issue

Wiki page for 0.10 release

2015-08-03 Thread Fabian Hueske
Hi everybody, I created a wiki page [1] to track issues related to the 0.10 release such as API-breaking changes. Please feel free to extend. Cheers, Fabian [1] https://cwiki.apache.org/confluence/display/FLINK/0.10+Release

[jira] [Created] (FLINK-2468) StreamCheckpointingITCase failure

2015-08-03 Thread Sachin Goel (JIRA)
Sachin Goel created FLINK-2468: -- Summary: StreamCheckpointingITCase failure Key: FLINK-2468 URL: https://issues.apache.org/jira/browse/FLINK-2468 Project: Flink Issue Type: Bug Repor

[jira] [Created] (FLINK-2469) JobManager crashes on Cancel

2015-08-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2469: -- Summary: JobManager crashes on Cancel Key: FLINK-2469 URL: https://issues.apache.org/jira/browse/FLINK-2469 Project: Flink Issue Type: Bug Re

[jira] [Created] (FLINK-2470) Stream Iteration can Hang after some Data

2015-08-03 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2470: --- Summary: Stream Iteration can Hang after some Data Key: FLINK-2470 URL: https://issues.apache.org/jira/browse/FLINK-2470 Project: Flink Issue Type: Bug

Re: Renaming log file?

2015-08-03 Thread Maximilian Michels
This could potentially break external software but +1 for renaming it now. On Mon, Aug 3, 2015 at 11:54 AM, Stephan Ewen wrote: > Sounds fair to rename them... > > On Mon, Aug 3, 2015 at 11:46 AM, Matthias J. Sax < > mj...@informatik.hu-berlin.de> wrote: > > > Hi, > > > > I think that the log fi

[jira] [Created] (FLINK-2471) FileSinkFunction invoke performance optimize

2015-08-03 Thread fangfengbin (JIRA)
fangfengbin created FLINK-2471: -- Summary: FileSinkFunction invoke performance optimize Key: FLINK-2471 URL: https://issues.apache.org/jira/browse/FLINK-2471 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-2472) Make the JobClientActor check periodically if the submitted Job is still running and if the JobManager is still alive

2015-08-03 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-2472: Summary: Make the JobClientActor check periodically if the submitted Job is still running and if the JobManager is still alive Key: FLINK-2472 URL: https://issues.apache.org/jira/

[jira] [Created] (FLINK-2473) Add a timeout to ActorSystem shutdown in the Client

2015-08-03 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2473: --- Summary: Add a timeout to ActorSystem shutdown in the Client Key: FLINK-2473 URL: https://issues.apache.org/jira/browse/FLINK-2473 Project: Flink Issue Type: B

Failing Test

2015-08-03 Thread Matthias J. Sax
Hi, I just hit a failing test (https://travis-ci.org/apache/flink/jobs/73899795). It is know or new? > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 86.929 sec > <<< FAILURE! - in > org.apache.flink.test.checkpointing.StreamCheckpointingITCase > runCheckpointedProgram(org.apac

Re: Failing Test

2015-08-03 Thread Stephan Ewen
Seen this a few times as well. May be something with the latest "partitioned state" changes... On Mon, Aug 3, 2015 at 5:48 PM, Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote: > Hi, > > I just hit a failing test > (https://travis-ci.org/apache/flink/jobs/73899795). It is know or new? > >

Re: A soft reminder

2015-08-03 Thread Stephan Ewen
I think I have found two problems in the hash table. One may be the loss of records, the other one the EOF. Give me a bit to confirm, test, and prepare a patch. On Thu, Jul 30, 2015 at 7:46 PM, Gábor Gévay wrote: > It is working with setSolutionSetUnmanagedMemory(true), thanks! > > Gabor > > >

[jira] [Created] (FLINK-2474) Occasional failures in PartitionedStateCheckpointingITCase

2015-08-03 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2474: --- Summary: Occasional failures in PartitionedStateCheckpointingITCase Key: FLINK-2474 URL: https://issues.apache.org/jira/browse/FLINK-2474 Project: Flink Issue

Re: Off-heap memory in Flink?

2015-08-03 Thread Stephan Ewen
Hey! There is the wiki on Flink's memory management: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741525 Greetings, Stephan On Fri, Jul 31, 2015 at 4:06 PM, Radu Tudoran wrote: > Hi, > > Is there some info, description about how this off-heap memory is managed > and its

Failing Test again

2015-08-03 Thread Matthias J. Sax
Today, not a single built was successful completely. Please see here: Flink Streaming Core: https://travis-ci.org/mjsax/flink/jobs/73938109 https://travis-ci.org/mjsax/flink/jobs/73951362 https://travis-ci.org/apache/flink/jobs/73938124 https://travis-ci.org/apache/flink/jobs/73899795 https://trav

[jira] [Created] (FLINK-2475) Rename Flink Client log and out file

2015-08-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2475: -- Summary: Rename Flink Client log and out file Key: FLINK-2475 URL: https://issues.apache.org/jira/browse/FLINK-2475 Project: Flink Issue Type: Improvemen

Re: Failing Test again

2015-08-03 Thread Henry Saputra
Thanks for reporting it , Matthias. Will try to run Travis for latest Flink. Tachyon test is a bit flaky. Maybe updating to latest release could help. - Henry On Mon, Aug 3, 2015 at 2:18 PM, Matthias J. Sax wrote: > Today, not a single built was successful completely. Please see here: > > Flink

Re: Failing Test again

2015-08-03 Thread Matthias J. Sax
I only report failing tests after a rebase. ;) -Matthias On 08/03/2015 11:23 PM, Henry Saputra wrote: > Thanks for reporting it , Matthias. Will try to run Travis for latest Flink. > > Tachyon test is a bit flaky. Maybe updating to latest release could help. > > - Henry > > On Mon, Aug 3, 2015

[jira] [Created] (FLINK-2476) Remove unwanted check null of input1 in ConnectedDataStream

2015-08-03 Thread fangfengbin (JIRA)
fangfengbin created FLINK-2476: -- Summary: Remove unwanted check null of input1 in ConnectedDataStream Key: FLINK-2476 URL: https://issues.apache.org/jira/browse/FLINK-2476 Project: Flink Issue

[jira] [Created] (FLINK-2477) Add test for SocketClientSink

2015-08-03 Thread Huang Wei (JIRA)
Huang Wei created FLINK-2477: Summary: Add test for SocketClientSink Key: FLINK-2477 URL: https://issues.apache.org/jira/browse/FLINK-2477 Project: Flink Issue Type: Test Components: St

[jira] [Created] (FLINK-2478) The page “FlinkML - Machine Learning for Flink“ https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/ contains a dead link

2015-08-03 Thread Slim Baltagi (JIRA)
Slim Baltagi created FLINK-2478: --- Summary: The page “FlinkML - Machine Learning for Flink“ https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/ contains a dead link Key: FLINK-2478 URL: https://issues.a

Re: Failing Test again

2015-08-03 Thread Aljoscha Krettek
What are the commits that you rebased on? Could you maybe narrow down what caused the regression? On Mon, 3 Aug 2015 at 23:31 Matthias J. Sax wrote: > I only report failing tests after a rebase. ;) > > -Matthias > > On 08/03/2015 11:23 PM, Henry Saputra wrote: > > Thanks for reporting it , Matth