Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Vinay Patil
Yeah understood. Thank you for helping guys. Regards, Vinay Patil *+91-800-728-4749* On Thu, May 26, 2016 at 5:40 PM, Stephan Ewen wrote: > The SuccessException does not really have a dependency. > > It is just a special Exception class that you throw in your code when you > want to stop. > Th

Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Stephan Ewen
The SuccessException does not really have a dependency. It is just a special Exception class that you throw in your code when you want to stop. The code that calls "env.execute()" catches the exception and checks whether the failure cause is that special exceptions. Flink propagates the exceptions

Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Vinay Patil
Hi Stephan, Yes using DeserializationSchema solution will definitely work. I am not able to get the dependency for SuccessException. Any help on this Regards, Vinay Patil *+91-800-728-4749* On Thu, May 26, 2016 at 3:32 PM, Stephan Ewen wrote: > Hi! > > On Flink 1.0, there is the "flink-test-u

Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Stephan Ewen
Hi! On Flink 1.0, there is the "flink-test-utils_2.10" dependency that has a some useful things. The "SuccessException" seems a quite common thing - I have seen that in other infinite program tests as well (Google Dataflow / Beam) Another way you can architect tests is to have an element in the

Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Vinay Patil
Hi Aljoscha, Thank you for answering. Throwing SuccessException is a good idea , however when I am adding following dependency, no classes are getting added to the jar: org.apache.flink flink-tests_2.10 1.0.3 Is there any other dependency that I have to add ? I have also added t

Re: Junit Issue while testing Kafka Source

2016-05-26 Thread Aljoscha Krettek
Hi, what we are doing in most internal tests is to verify in a sink whether the data is correct and then throw a SuccessException. This brings down the job and we check whether we catch a SuccessException to verify that the test was successful. Look, for example, at the ValidatingSink in EventTimeW

Re: Junit Issue while testing Kafka Source

2016-05-25 Thread Nick Dimiduk
I'm also curious for a solution here. My test code executes the flow from a separate thread. Once i've joined on all my producer threads and I've verified the output, I simply interrupt the flow thread. This spews exceptions, but it all appears to be harmless. Maybe there's a better way? I think y