Re: Unit tests failing, losing stream contents

2016-08-15 Thread Ciar, David B.
rsableLike.scala:234) From: Stephan Ewen Sent: 12 August 2016 10:39 To: user@flink.apache.org Subject: Re: Unit tests failing, losing stream contents Hi David! I would guess that the first exception happens once in a while, as part of a rare race condition. As Max

Re: Unit tests failing, losing stream contents

2016-08-15 Thread Ciar, David B.
le. Thanks, David From: Stephan Ewen Sent: 12 August 2016 10:39:16 To: user@flink.apache.org Subject: Re: Unit tests failing, losing stream contents Hi David! I would guess that the first exception happens once in a while, as part of a rare race condition. As Max said, two executions happe

Re: Unit tests failing, losing stream contents

2016-08-12 Thread Stephan Ewen
Hi David! I would guess that the first exception happens once in a while, as part of a rare race condition. As Max said, two executions happen simultaneously. We should fix that race condition, though. The second exception looks like it is purely part of your application code. Greetings, Stephan

Re: Unit tests failing, losing stream contents

2016-08-12 Thread Maximilian Michels
Hi David, You're starting two executions at the same time (in different threads). Here's why: Execution No 1 DataStreamUtils.collect(..) starts a Thread which executes your job and collects stream elements. It runs asynchronously. The collect(..) method returns after starting the thread. Executi

Unit tests failing, losing stream contents

2016-08-11 Thread Ciar, David B.
Hi everyone, I've been trying to write unit tests for my data stream bolts (map, flatMap, apply etc.), however the results I've been getting are strange. The code for testing is here (running with scalatest and sbt): https://gist.github.com/dbciar/7469adfea9e6442cdc9568aed07095ff It runs t