Re: Testing with ProcessingTime and FromElementsFunction with TestEnvironment

2019-08-13 Thread Till Rohrmann
Hi Michal, you need to implement a source which does not terminate. Take a look at the InifiteSource [1] which does exactly this. That way there won't be a Long.MAX_VALUE being sent when closing the source operator. [1] https://github.com/apache/flink/blob/master/flink-streaming-java/src/test/jav

Re: Testing with ProcessingTime and FromElementsFunction with TestEnvironment

2019-08-09 Thread Michal Klempa
Hi guys, I have opposite issue :-) I would like to unit test negative behavior - that the Event Time timer is not fired when no further event arrives (which would advance the watermarks). But due to StreamSource firing Long.MAX_VALUE watermark after enclosed finite FromElementsFunction run method d

Re: Testing with ProcessingTime and FromElementsFunction with TestEnvironment

2019-05-09 Thread Till Rohrmann
Hi Steve, afaik there is no such thing in Flink. I agree that Flink's testing utilities should be improved. If you implement such a source, then you might be able to contribute it back to the community. That would be super helpful. Cheers, Till On Wed, May 8, 2019 at 6:40 PM Steven Nelson wrote

Re: Testing with ProcessingTime and FromElementsFunction with TestEnvironment

2019-05-08 Thread Steven Nelson
That’s what I figured was happening :( Your explanation is a lot better than what I gave to my team, so that will help a lot, thank you! Is there a testing source already created that does this sort of thing? The Flink-testing library seems a bit sparse. -Steve Sent from my iPhone > On May 8

Re: Testing with ProcessingTime and FromElementsFunction with TestEnvironment

2019-05-08 Thread Till Rohrmann
Hi Steve, I think the reason for the different behaviour is due to the way event time and processing time are implemented. When you are using event time, watermarks need to travel through the topology denoting the current event time. When you source terminates, the system will send a watermark wi