This might be useful to do. BTW, based on my experience with different build systems in the past few years (extensively SBT/Maven/Bazel, and to a less extent Gradle/Cargo), I think the longer term solution is to move to Bazel. It is so much easier to understand and use, and also much more feature rich with great support for multiple languages. It also supports distributed/local build cache so builds can be much faster.
On Mon, Jan 28, 2019 at 1:28 AM, Gabor Somogyi < gabor.g.somo...@gmail.com > wrote: > > Do you have some numbers how much is this faster? I'm asking it because > previously I've evaluated another plugin and found the following: > - Incremental build didn't bring too much even in bigger than spark > projects > - Incremental test was buggy and sometimes the required tests were not > executed which caused several issues > All in all a single tiny little bug in the incremental test could cause > horror for developers so it must be rock solid. > Is this project used somewhere in production? > > On Sat, Jan 26, 2019 at 4:03 PM Sean Owen < srowen@ gmail. com ( > sro...@gmail.com ) > wrote: > > >> Sounds interesting; would it be able to handle R and Python modules built >> by this project ? The home grown solution here does I think and that is >> helpful. >> >> On Sat, Jan 26, 2019, 6:57 AM vaclavkosar < admin@ vaclavkosar. com ( >> ad...@vaclavkosar.com ) wrote: >> >> >>> >>> >>> I think it would be good idea to use gitflow-incremental-builder maven >>> plugin for Spark builds. It saves resources by building only modules that >>> are impacted by changes compared to git master branch via >>> gitflow-incremental-builder maven plugin. For example if there is only a >>> change introduced into on of files of spark-avro_2.11 then only that maven >>> module and its maven dependencies and dependents would be build or tested. >>> If there are no disagreements, I can submit a pull request for that. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Project URL: https:/ / github. com/ vackosar/ gitflow-incremental-builder ( >>> https://github.com/vackosar/gitflow-incremental-builder ) >>> >>> >>> >>> Disclaimer: I originally created the project. But most of recent >>> improvements and maintenance were deved by Falko. >>> >>> >>> >> >> > >
