I agree with Ufuk. Reworking tests might cause a lot of problems! We should do this with great care only!
The most important thing is to pay attention to open PRs and ask to write unit tests. I would also suggest, to have at least one new unit test per fixed bug (that of course test exactly the bug that was fixed ;)) Furthermore, I expect some people to have problems to mock correctly. Is a little tricky sometimes. Do we have a few nice mock examples that we could add to the contributor guide? -Matthias On 09/19/2015 06:39 PM, Chiwan Park wrote: > I just created a JIRA issue [1]. > > Regards, > Chiwan Park > > [1] https://issues.apache.org/jira/browse/FLINK-2712 > >> On Sep 20, 2015, at 1:33 AM, Chiwan Park <chiwanp...@apache.org> wrote: >> >> Okay, I’ll create a JIRA issue and send a pull request for it. :) >> >> Regards, >> Chiwan Park >> >>> On Sep 19, 2015, at 7:35 PM, Ufuk Celebi <u...@apache.org> wrote: >>> >>> Thanks Stephan for pointing this out. I agree with you. +1 >>> >>> @Chiwan: Good idea with the Wiki. Actually maybe even better to add it to >>> the contribution guide? Do you have time to open a PR with Stephan’s >>> suggestions? >>> >>> @Martin: I agree that it does not suffice to just point this out as a new >>> guideline. I think the main problem is that it is very time consuming and >>> error prone. We have seen some “minor refactorings” lately, which looked >>> harmless, but actually introduced bugs. This is a danger as well with >>> refactoring tests (we refactor them, but don’t have the same amount of test >>> coverage, which results in bugs at some point in time in the future). >>> >>> Are there any known “heavy hitters”, which take a lot of time, but which >>> could be tested in unit tests instead? I would start with those if you want >>> to do it. But in general I would do this incrementally instead of aiming >>> for a complete rewrite. >>> >>> – Ufuk >>> >>>> On 19 Sep 2015, at 10:53, Martin Liesenberg <martin.liesenb...@gmail.com> >>>> wrote: >>>> >>>> Should there be a concerted effort to reduce the amount of unnecessary >>>> integration tests and cover those cases by unit tests? >>>> >>>> We could collect the cases in a ticket and work through the list one by >>>> one, no? >>>> >>>> Best regards, >>>> Martin >>>> >>>> Chiwan Park <chiwanp...@apache.org> schrieb am Fr., 18. Sep. 2015 um >>>> 12:33 Uhr: >>>> >>>>> Hi Stephan, >>>>> >>>>> Thanks for nice guide! I think we can upload this to the wiki or how to >>>>> contribute documentation. >>>>> This guide would be helpful for newcomers. >>>>> >>>>> Regards, >>>>> Chiwan Park >>>>> >>>>>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <se...@apache.org> wrote: >>>>>> >>>>>> Hi all! >>>>>> >>>>>> The build time of Flink with all tests is nearing 1h on Travis for the >>>>>> shortest run. >>>>>> It is good that we do excessive testing, there are many mechanisms that >>>>>> need that. >>>>>> >>>>>> I have also seen that a lot of fixes that could be tested in a UnitTest >>>>>> style are actually tested as a full Flink program (Integration test >>>>> style) >>>>>> >>>>>> While these tests are always easier to write, they have two problems: >>>>>> - The bring up the build time by about 5 secs per test >>>>>> - They are often not as targeted to the problem as a UnitTest >>>>>> >>>>>> I would like to encourage everyone to keep this in mind and do Unit tests >>>>>> in the cases where they are the preferred choice. Please also keep that >>>>> in >>>>>> mind when reviewing pull requests. >>>>>> >>>>>> For Example: >>>>>> - API / TypeInformation changes can be very well tested without running >>>>>> the program. Simply create the program and test the operator's type info. >>>>>> - Custom functions can be very well tested in isolation >>>>>> - Input/Output formats actually test well in UnitTests. >>>>>> >>>>>> Integration tests need to be used when verifying behavior across >>>>> components >>>>>> / layers, so keep using them when they need to be used. >>>>>> >>>>>> >>>>>> Greetings, >>>>>> Stephan >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>> >> >> >> >>
signature.asc
Description: OpenPGP digital signature
