Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-10-08 Thread Matthieu Bonneviot
Hi I am a new flink developer and I also feel we need to improve the testing side of flink. >From Tony list: 1-stream logic control: we may consider reactiveX as a reference for stream testing: http://reactivex.io/. They have a test scheduler and use marble diagram to write unit test: for example

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-10-07 Thread Tony Wei
Hi all, Does anyone want to share something about how to provide more better testing tools or good documentations? @Till, @Aljoscha Do you have further suggestions about the improvements of testing tools and what is the next step we can do? Best, Tony Wei Tony Wei 於 2018年9月29日 週六 上午12:20寫道:

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-09-28 Thread Tony Wei
Hi Till, Thanks for your feedback. I didn't think about this before. It will be better if we can provide such tools instead of let user deal with testing in operator level directly. Because test harness was introduced for contributors, who may have more knowledge about internal design, to test the

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-09-28 Thread Till Rohrmann
Hi Tony, I think this is a long sought-after feature to provide better testing tools for our users. Thus, I'm strongly in favour of adding something like this. If I remember correctly Aljoscha already spend some brain cycles on this and he also gave a training about the current state at FF SF 2018

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-09-28 Thread Tony Wei
Hi all, @Ken, Thanks for your positive feedback. I have a similar experience with test harness and that's why I want to provide more contents on testing doc to prevent from this kind of problems. Does anyone have any feedback and advice? I would like to collect more opinions from every developers

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator

2018-09-25 Thread Ken Krugler
Hi Tony, I think this would be great - we’ve been building out tests using AbstractStreamOperator, and the lack of documentation has made it challenging. For example, there was this exchange I had with Piotr about a month ago: > You made a small mistake when restoring from state using test harn