FYI put up a PR for jenkins following this prototype: https://github.com/apache/bookkeeper/pull/1481
On Mon, Jun 4, 2018 at 10:05 PM, Sijie Guo <guosi...@gmail.com> wrote: > Hi all, > > I have been looking into the bookkeeper build time for a while. It usually > takes about 1hr ~ 2hrs to completely run a bookkeeper build with full tests. > > Most of the time is spent on running bookkeeper-server module. Building > bookkeeper-server module usually takes about 50 mins to 1 hr. Sometimes it > also causes time out. > > E.g. https://builds.apache.org/job/bookkeeper_precommit_ > pullrequest_java8/1178/org.apache.bookkeeper$bookkeeper-server/ > > ``` > *Started 3 hr 29 min ago* > *Took 57 min > <https://builds.apache.org/job/bookkeeper_precommit_pullrequest_java8/org.apache.bookkeeper$bookkeeper-server/buildTimeTrend> > on H23 > <https://builds.apache.org/computer/H23>* > > ``` > > Looking into the detailed build time, following 4 packages used most of > the time. > > https://builds.apache.org/job/bookkeeper_precommit_ > pullrequest_java8/1178/org.apache.bookkeeper$bookkeeper-server/testReport/ > > - `org.apache.bookkeeper.client` : 19min > - `org.apache.bookkeeper.bookie` : 7min > - `org.apache.bookkeeper.replication`: 7min > - `org.apache.bookkeeper.tls`: 7min > - all the remaining tests: 13min > > > In order to improve the building time, we can split the build job into > multiple smaller jobs that only run a subset of tests. so we can reduce the > build time to 1/3 or even 1/4. > > > Here is one quick prototype using travis: https://travis-ci.org/ > sijie/bookkeeper/builds/388083586 > > using `mvn -pl bookkeeper-server test -Dtest=${BK_TEST_GROUP}` to test a > group of tests. Those test groups can be: > > - client: "org.apache.bookkeeper.client.**" > - bookie: "org.apache.bookkeeper.bookie.**" > - replication: "org.apache.bookkeeper.replication.**" > - tls: "org.apache.bookkeeper.tls.**" > - all others: "!org.apache.bookkeeper.client.**,!org. > apache.bookkeeper.bookie.**,!org.apache.bookkeeper.replication.**,! > org.apache.bookkeeper.tls.**" > > > Thoughts? If this looks good, I will submit a PR for this, but do it in > apache jenkins. > > - Sijie > >