Il gio 7 giu 2018, 02:49 Sijie Guo <guosi...@gmail.com> ha scritto: > with splitting the mvn build into multiple smaller jobs, we can quickly > identify what tests are failing and only rerun the failed precommit check > via "rurun ..." comments. > > however sometimes some of the PRs don't actually require running fullset of > precommit checks (e.g. documentation, configuration changes, CI changes). > in order to reduce the time and allow fast iterations on those changes, I > setup a template in PULL_REQUEST_TEMPLATE to allow people choose to skip > some precommit checks when filing the pull requests. > here is the change - https://github.com/apache/bookkeeper/pull/1494 . This > change itself also use those skip phrases to skip some precommit checks > since that PR is not a code change PR. >
Great Enrico > This also allow faster iterations on PRs that doesn't touch > bookkeeper-server module, because the build time in bookkeeper-server > module unfortunately is really bad. we should discuss splitting > bookkeeper-server module soon, but that would be a different thread. > Sure, I would have wanted to split client module in 4.8 but I did not have time. Hope we can do for 4.9. Yes...it is a different story Enrico > > - Sijie > > On Wed, Jun 6, 2018 at 3:42 PM, Jia Zhai <zhaiji...@gmail.com> wrote: > > > 👍 > > > > On Wed, Jun 6, 2018 at 5:05 AM, Sijie Guo <guosi...@gmail.com> wrote: > > > > > Put up a fix to remove postcommit phrase triggers - > > > https://github.com/apache/bookkeeper/pull/1482 > > > > > > - Sijie > > > > > > On Tue, Jun 5, 2018 at 12:32 PM, Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > I think enabling phrase trigger on precommits job will result in > > > > triggering postcommit jobs when using "retest this please". > > > > > > > > Coming up a fix to remove postcommit phrase triggers because they > don't > > > > make sense. > > > > > > > > - Sijie > > > > > > > > On Tue, Jun 5, 2018 at 1:12 AM, Enrico Olivelli <eolive...@gmail.com > > > > > > wrote: > > > > > > > >> +1 > > > >> Awesome ! > > > >> > > > >> We will need to add builds for JDK10, so we have to be stronger in > > > >> testing. > > > >> These days my email box is full of "retest this please" > > > >> > > > >> Thank you Sijie > > > >> > > > >> Enrico > > > >> > > > >> > > > >> > > > >> Il giorno mar 5 giu 2018 alle ore 08:47 Sijie Guo < > guosi...@gmail.com > > > > > > ha > > > >> scritto: > > > >> > > > >> > FYI put up a PR for jenkins following this prototype: > > > >> > https://github.com/apache/bookkeeper/pull/1481 > > > >> > > > > >> > On Mon, Jun 4, 2018 at 10:05 PM, Sijie Guo <guosi...@gmail.com> > > > wrote: > > > >> > > > > >> > > Hi all, > > > >> > > > > > >> > > I have been looking into the bookkeeper build time for a while. > It > > > >> > usually > > > >> > > takes about 1hr ~ 2hrs to completely run a bookkeeper build with > > > full > > > >> > tests. > > > >> > > > > > >> > > Most of the time is spent on running bookkeeper-server module. > > > >> Building > > > >> > > bookkeeper-server module usually takes about 50 mins to 1 hr. > > > >> Sometimes > > > >> > it > > > >> > > also causes time out. > > > >> > > > > > >> > > E.g. https://builds.apache.org/job/bookkeeper_precommit_ > > > >> > > pullrequest_java8/1178/org.apache.bookkeeper$bookkeeper-server/ > > > >> > > > > > >> > > ``` > > > >> > > *Started 3 hr 29 min ago* > > > >> > > *Took 57 min > > > >> > > < > > > >> > https://builds.apache.org/job/bookkeeper_precommit_pullreque > > > >> st_java8/org.apache.bookkeeper$bookkeeper-server/buildTimeTrend> > > > >> > on H23 > > > >> > > <https://builds.apache.org/computer/H23>* > > > >> > > > > > >> > > ``` > > > >> > > > > > >> > > Looking into the detailed build time, following 4 packages used > > most > > > >> of > > > >> > > the time. > > > >> > > > > > >> > > https://builds.apache.org/job/bookkeeper_precommit_ > > > >> > > > > > >> > pullrequest_java8/1178/org.apache.bookkeeper$bookkeeper-serv > > > >> er/testReport/ > > > >> > > > > > >> > > - `org.apache.bookkeeper.client` : 19min > > > >> > > - `org.apache.bookkeeper.bookie` : 7min > > > >> > > - `org.apache.bookkeeper.replication`: 7min > > > >> > > - `org.apache.bookkeeper.tls`: 7min > > > >> > > - all the remaining tests: 13min > > > >> > > > > > >> > > > > > >> > > In order to improve the building time, we can split the build > job > > > into > > > >> > > multiple smaller jobs that only run a subset of tests. so we can > > > >> reduce > > > >> > the > > > >> > > build time to 1/3 or even 1/4. > > > >> > > > > > >> > > > > > >> > > Here is one quick prototype using travis: > https://travis-ci.org/ > > > >> > > sijie/bookkeeper/builds/388083586 > > > >> > > > > > >> > > using `mvn -pl bookkeeper-server test -Dtest=${BK_TEST_GROUP}` > to > > > >> test a > > > >> > > group of tests. Those test groups can be: > > > >> > > > > > >> > > - client: "org.apache.bookkeeper.client.**" > > > >> > > - bookie: "org.apache.bookkeeper.bookie.**" > > > >> > > - replication: "org.apache.bookkeeper.replication.**" > > > >> > > - tls: "org.apache.bookkeeper.tls.**" > > > >> > > - all others: "!org.apache.bookkeeper.client.**,!org. > > > >> > > apache.bookkeeper.bookie.**,!org.apache.bookkeeper. > > replication.**,! > > > >> > > org.apache.bookkeeper.tls.**" > > > >> > > > > > >> > > > > > >> > > Thoughts? If this looks good, I will submit a PR for this, but > do > > it > > > >> in > > > >> > > apache jenkins. > > > >> > > > > > >> > > - Sijie > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > -- -- Enrico Olivelli
