Hi there, just a shot in a dark: Have you tried any of the existing stuff, like Takari Lifecycle before modding Maven itself? ( http://takari.io/book/40-lifecycle.html)
Thanks, T On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov < maximilian.novi...@db.com> wrote: > Hi All, > > > > *We want to create upstream change to Maven* to support true incremental > build for big-sized projects. > > To raise a pull request we have to pass long chain of Deutsche Bank’s > internal procedures. So, *before starting the process we would like to > get your feedback regarding this feature*. > > > > *Motivation:* > > > > Our project is hosted in mono-repo and contains ~600 modules. All modules > has the same SNAPSHOT version. > > There are lot of test automation around this, everything is tested before > merge into release branch. > > > > Current setup helps us to simplify build/release/dependency management for > 10+ teams those contribute into codebase. We can release everything in > 1-click. > > The major drawback of such approach is build time: *full local build took > 45-60 min (*-T8)*, CI build ~25min(*-T16*)*. > > > > To speed-up our build we needed 2 features: incremental build and shared > cache. > > Initially we started to think about migration to Gradle or Bazel. As > migration costs for the mentioned tools were too high, we decided to add > similar functionality into Maven. > > > > Current results we get: *1-2 mins for local build(*-T8*)* if build was > cached by CI*, CI build ~5 mins (*-T16*).* > > > > *Feature description:* > > > > The idea is to calculate checksum for inputs and save outputs in cache. > > [image: image2019-8-27_20-0-14.png] > > Each node checksum calculated with: > > > > · Effective POM hash > > · Sources hash > > · Dependencies hash (dependencies within multi-module project) > > > > Project sources inputs are searched inside project + all paths from > plugins configuration: > > [image: image2019-8-30_10-28-56.png] > > How does it work in practice: > > > > 1. CI: runs builds and stores outputs in shared cache > > 2. CI: reuse outputs for same inputs, so time is decreasing > > 3. Locally: when I checkout branch and run ‘install’ for whole > project, I get all actual snapshots from remote cache for this branch > > 4. Locally: if I change multiple modules in tree, only changed > subtree is rebuilt > > > > Impact on current Maven codebase is very localized (MojoExecutor, where we > injected cache controller). > > Caching can be activated/deactivated by property, so current maven flow > will work as is. > > > > And the big plus is that you don’t need to re-work your current project. > Caching should work out of box, just need to add config in .mvn folder. > > > > Please let us know what do you think. We are ready to invest in this > feature and address any further feedback. > > > > Kind regards, > > Max > > > > > --- > This e-mail may contain confidential and/or privileged information. If you > are not the intended recipient (or have received this e-mail in error) > please notify the sender immediately and delete this e-mail. Any > unauthorized copying, disclosure or distribution of the material in this > e-mail is strictly forbidden. > > Please refer to https://www.db.com/disclosures for additional EU > corporate and regulatory disclosures and to > http://www.db.com/unitedkingdom/content/privacy.htm for information about > privacy. >
