Hi Stephan, Yes using DeserializationSchema solution will definitely work. I am not able to get the dependency for SuccessException. Any help on this
Regards, Vinay Patil *+91-800-728-4749* On Thu, May 26, 2016 at 3:32 PM, Stephan Ewen <se...@apache.org> wrote: > Hi! > > On Flink 1.0, there is the "flink-test-utils_2.10" dependency that has a > some useful things. > > The "SuccessException" seems a quite common thing - I have seen that in > other infinite program tests as well (Google Dataflow / Beam) > > Another way you can architect tests is to have an element in the stream > that signals end-of-stream. The DeserializationSchema can check for that > and return "end of stream". > > Greetings, > Stephan > > > > On Thu, May 26, 2016 at 11:55 AM, Vinay Patil <vinay18.pa...@gmail.com> > wrote: > > > Hi Aljoscha, > > > > Thank you for answering. > > Throwing SuccessException is a good idea , however when I am adding > > following dependency, no classes are getting added to the jar: > > > > <dependency> > > <groupId>org.apache.flink</groupId> > > <artifactId>flink-tests_2.10</artifactId> > > <version>1.0.3</version> > > </dependency> > > > > Is there any other dependency that I have to add ? I have also added > > test-utils dependency. > > > > I am trying the following in my test case : > > 1) Consuming data from Kafka using FlinkKafkaConsumer and passing it to > map > > as Tuple2 > > 2) In the map function I am just checking if Tuple2 contains data, if > yes, > > throw the exception("success") > > 3) This way I am verifying that the configuration is correct and that we > > are able to read from kafka. > > > > Am I doing it right, is there any better approach ? > > > > Regards, > > Vinay Patil > > > > *+91-800-728-4749* > > > > On Thu, May 26, 2016 at 1:01 PM, Aljoscha Krettek <aljos...@apache.org> > > wrote: > > > > > Hi, > > > what we are doing in most internal tests is to verify in a sink whether > > the > > > data is correct and then throw a SuccessException. This brings down the > > job > > > and we check whether we catch a SuccessException to verify that the > test > > > was successful. Look, for example, at the ValidatingSink in > > > EventTimeWindowCheckpointingITCase in the Flink source. > > > > > > Cheers, > > > Aljoscha > > > > > > On Thu, 26 May 2016 at 01:58 Nick Dimiduk <ndimi...@gmail.com> wrote: > > > > > > > I'm also curious for a solution here. My test code executes the flow > > > from a > > > > separate thread. Once i've joined on all my producer threads and I've > > > > verified the output, I simply interrupt the flow thread. This spews > > > > exceptions, but it all appears to be harmless. > > > > > > > > Maybe there's a better way? I think you'd need some "death pill" to > > send > > > > into the stream that signals its termination. > > > > > > > > On Tue, May 24, 2016 at 7:29 PM, Vinay Patil < > vinay18.pa...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I am able to read from a topic using FlinkKafkaConsumer and return > > the > > > > > result, however when I am testing this scenario in Junit the > result > > is > > > > > getting printed(kafkaStream.print()) but I am not able to exit the > > > Job, > > > > > env.execute keeps running, > > > > > I tried to return env.execute from method but that did not work > > either. > > > > > > > > > > 1) Is there any way to end the execution of job forcefully. > > > > > 2) How do I test if the data has come from topic > > > > > > > > > > - One way I think of is to get the output of stream.print() in a > > > > > PrintStream and check the result.(but not able to test this > since > > > job > > > > is > > > > > not getting exited) > > > > > > > > > > Please help with these issues > > > > > > > > > > Regards, > > > > > Vinay Patil > > > > > > > > > > > > > > >