If we can get a incremental builds to work, that would actually be the preferred solution in my opinion.
Many companies have invested heavily in making a "single repository" code base work, because it has the advantage of not having to update/publish several repositories first. However, the strong prerequisite for that is an incremental build system that builds only (fine grained) what it has to build. I am not sure how we could make that work with Maven and Travis... On Wed, Feb 22, 2017 at 10:42 PM, Greg Hogan <c...@greghogan.com> wrote: > An additional option for reducing time to build and test is parallel > execution. This would help users more than on TravisCI since we're > generally running on multi-core machines rather than VM slices. > > Is the idea that each user would only check out the modules that he or she > is developing with? For example, if a developer is not working on > flink-mesos or flink-yarn then the "flink-deploy" module would not be clone > to their filesystem? > > We can run a TravisCI nightly build on each repo to validate against API > changes. > > Greg > > On Wed, Feb 22, 2017 at 12:24 PM, Fabian Hueske <fhue...@gmail.com> wrote: > > > Hi everybody, > > > > I think this should be a discussion about the benefits and drawbacks of > > separating the code into distinct repositories from a development point > of > > view. > > So I agree with Stephan that we should not divide the community by > creating > > separate groups of committers. > > Also the discussion about independent releases is not be strictly related > > to the decision, IMO. > > > > I see a few pros and cons for splitting the code base into separate > > repositories which (I think) haven't been mentioned before: > > pros: > > - IDE setup will be leaner. It is not necessary to compile the whole code > > base to run a test after switching a branch. > > cons: > > - developing libraries features that require changes in the core / APIs > > become more time consuming due to back-and-forth between code bases. > > However, I think this is not very often the case. > > > > Aljoscha has good points as well. Many of the build issues could be > solved > > by different build profiles and configurations. > > > > Best, Fabian > > > > 2017-02-22 14:59 GMT+01:00 Gábor Hermann <m...@gaborhermann.com>: > > > > > @Stephan: > > > > > > Although I tried to raise some issues about splitting committers, I'm > > > still strongly in favor of some kind of restructuring. We just have to > be > > > conscious about the disadvantages. > > > > > > Not splitting the committers could leave the libraries in the same > > > stalling status, described by Till. Of course, dedicating current > > > committers as shepherds of the libraries could easily resolve the > issue. > > > But that requires time from current committers. It seems like > trade-offs > > > between code quality, speed of development, and committer efforts. > > > > > > From what I see in the discussion about ML, there are many people > willing > > > to contribute as well as production use-cases. This means we could and > > > should move forward. However, the development speed is significantly > > slowed > > > down by stalling PRs. The proposal for contributors helping the review > > > process did not really work out so far. In my opinion, either code > > quality > > > (by more easily accepting new committers) or some committer time > > > (reviewing/merging) should be sacrificed to move forward. As Till has > > > indicated, it would be shameful if we let this contribution effort die. > > > > > > Cheers, > > > Gabor > > > > > > > > >
