On 18 May 2013 04:06, Kevin Hilman <khil...@linaro.org> wrote: > Fathi Boudra <fathi.bou...@linaro.org> writes: > > [...] > >>> Is there a way for us (linaro folks) to see more of the Jenkins setup >>> for these jobs (including the scripts.) There appears to be some useful >>> add-ons being used. Read-only access to the detailed configuration of >>> the jenkins jobs would be very useful. >> >> You've got access and can even view/modify the set up. >> I used a multi-configuration project for these jobs. >> If you have more questions, just ask. > > Now that I have permission on linaro jenkins, I started experimenting > with trying to get something useful for ARM maintainers, and I created a > basic job[1] for building all ARM defconfigs (120+) in linux-next. > > This is a simple build-only job. No downloading toolchains, ubuntu > packaging, lava testing, etc. etc like the other Linaro CI jobs.
It will be great if we can keep using latest Linaro GCC. We copy a tarball from the master to slave and extract it only once, it's taking less than a minute on the full build time. We probably want to keep the boot testing part optional, there's several options to implement, without impacting the main build testing job. IMO, to investigate. > Just kernel builds, and output that should make sense to kernel developers. > IMO, this is the starting point to having some basic sanity testing for > maintainers. > > I have several questions (and suggestions) now on how to speed this up > based on configuration of the slaves, as well as several questions > around best practice for using the slaves, how workspaces/tools/scripts > are (or aren't) shared between slaves, etc. > > The first suggestion is to speed up the git clones/fetches. Even a > shallow git clone (--depth=1) is taking > 3 minutes on the slaves. What > I do on my home jenkins box is to have a local repo (periodically > updated via cron), and then use the advanced options under jenkins git > SCM config to point to it using the "Path of the reference repo to use > during clone" option. That makes the git clones/fetches very fast since > they're (almost) always from a local repo. The difference between the slaves and your home box is that the slaves are ephemeral instances (terminated after a defined timeout) while your box is always up. We'll need to move to a persistent slave (stopped instead of terminated), opening the door to proper caching (local mirror). We have such set up for OpenEmbedded builds and other tricks in our toolbox for Android builds. > Another suggestion is to have ccache installed on the slaves. Since > this job is building much of the same code over and over (120+ > defconfigs), ccache would dramatically speed this up, and probably make > it more sensible to run all the builds sequentially on the same slave. ccache is already installed but due to the ephemeral property of current instances, it isn't exploited. Once again, moving to persistent slave will resolve the issue. Also, the EC2 instances aren't I/O optimized. On some jobs, I create a tmpfs directory where I build. It reduces dramatically build time. > Kevin > > P.S. I hope I'm not overloading the slaves too much[2] nope :) > P.P.S. Can someone install the Jenkins Warnings Plugin[3]. This can > automatically post-process gcc warnings/errors and plot the history of > them across multiple jobs. It's a really useful plugin. Done. Note, we have log parser plugin doing similar post-processing on gcc warnings/errors. > [1] https://ci.linaro.org/jenkins/job/khilman-kernel-arm-next/ > [2] https://ci.linaro.org/jenkins/label/kernel_cloud/load-statistics > [3] https://wiki.jenkins-ci.org/display/JENKINS/Warnings+Plugin Cheers, -- Fathi Boudra Builds and Baselines Manager | Release Manager Linaro.org | Open source software for ARM SoCs _______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
