GitHub user shanthoosh opened a pull request:
https://github.com/apache/samza/pull/640
[WIP][Not ready for review]SAMZA-1885: Improve samza build time.
Samza codebase currently uses gradle 2.8 (which was released in 2014) and
builds the codebase serially executing each of the gradle tasks one at a time.
Not taking advantage of recent gradle features has caused slow builds (taking
around 15 minutes to build samza codebase locally). This hampers the overall
developer productivity within the team. Proposal is to have a fasten builds by
a series of changes.
**Changes:**
1. Upgrade to gradle version: 4.7(changes in gradle.properties,
gradle-wrapper.jar).
2. Enable parallel gradle builds.
3. Set maximumParallelForks to 15(The maximum number of forked test
processes to execute in parallel. The default value is 1 (no parallel test
execution). It cannot exceed the value of max-workers for the current build.
Reference:
https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html)
4. Fix the tests that cannot be run in parallel(TestsJobsResource,
TestTaskResource).
**Result:**
Reduces the build time by a factor of 2.
Before these changes, it took around 13 minutes 40 seconds to build the
open source samza codebase locally.
After these changes, it took around around 6 minutes 14 seconds to build
samza codebase locally.
**Pending:**
Ran the build locally multiple(50) times. After these changes, only the
TestRunner API tests fail consistently due to static state in it to maintain
inemory(Will fix it in subsequent patches).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shanthoosh/samza improve_build_time
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/640.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #640
----
commit 347f8fef986ffaf87b6275d72370aea0f6ac9daf
Author: Shanthoosh Venkataraman <spvenkat@...>
Date: 2018-09-14T17:43:20Z
Changes:
1. Switch to newer version of gradle.
2. Turn on gradle parallel builds.
----
---
