Hello Jinkui Shi, Due to the nature of most of the Yarn tests, we need them to be in a separate module. More concretely, these tests have a dependency on 'flink-dist' because they need to deploy the Flink fat jar to the Yarn tests cluster. The fat jar also contains the 'flink-yarn' code. Thus, 'flink-yarn' needs to be a separate module and built before 'flink-yarn-tests'.
That being said, some of the tests don't need the fat jar, so we could move some of the tests to 'flink-yarn'. However, that is mostly a cosmetic change and not important for the testing coverage. Best, Max On Thu, Sep 22, 2016 at 12:26 PM, Stephan Ewen <se...@apache.org> wrote: > > "flink-test-utils" contains, as the name says, utils for testing. Intended > to be used by users in writing their own tests. > "flink-tests" contains cross module tests, no user should ever need to have > a dependency on that. > > They are different because users explicitly asked for test utils to be > factored into a separate project. > > As an honest reply here: Setting up a project as huge as Flink need to take > many things into account > > - Multiple languages (Java / Scala), with limitations of IDEs in mind > - Dependency conflicts and much shading magic > - Dependency matrices (multiple hadoop and scala versions) > - Supporting earlier Java versions > - clean scope differentiation, so users can reuse utils and testing code > > > That simply requires some extra modules once in a while. Flink people have > worked hard on coming up with a structure that serves the need of the > production users and automated build/testing systems. These production user > requests are most important to us, and sometimes, we need to take cuts in > "beauty of directory structure" to help them. > > Constantly accusing the community of creating bad structures before even > trying to understand the reasoning behind that does not come across as very > friendly. Constantly accusing the community of sloppy work just because > your laptop settings are incompatible with the default configuration > likewise. > > I hope you understand that. > > > On Thu, Sep 22, 2016 at 2:58 AM, shijinkui <shijin...@huawei.com> wrote: > > > Hi, Stephan > > > > Thanks for your reply. > > > > In my mind, Maven-shade-plugin and sbt-assembly both default exclude test > > code for the fat jar. > > > > In fact, unit tests are use to test the main code, ensure our code logic > > fit our expect . This is general convention. I think. Flink has be a top > > apache project. We shouldn't be special. We're programmer, should be > > professional. > > > > Even more, there are `flink-tes-utils-parent` and `flink-tests` module, > > what's the relation between them. > > > > I have to ask why they are exist? Where is the start of such confusion > > modules? > > > > I think we shouldn't do nothing for this. Code and design should be > > comfortable. > > > > Thanks > > > > From Jinkui Shi > > > > -----邮件原件----- > > 发件人: Stephan Ewen [mailto:se...@apache.org] > > 发送时间: 2016年9月21日 22:19 > > 收件人: dev@flink.apache.org > > 主题: Re: [discuss] merge module flink-yarn and flink-yarn-test > > > > I would like Robert to comment on this. > > > > I think there was a reason to have different modules, which had again > > something to do with the Maven Shade Plugin Dependencies and shading really > > seem the trickiest thing in bigger Java/Scala projects ;-) > > > > On Wed, Sep 21, 2016 at 11:04 AM, shijinkui <shijin...@huawei.com> wrote: > > > > > Hi, All > > > > > > There too much module in the root. There are no necessary to separate > > > the test code from sub-module. > > > > > > I never see such design: two modules, one is main code, the other is > > > test code. > > > > > > Is there some special reason? > > > > > > From Jinkui Shi > > > > >
