On 3/30/21 1:55 PM, Thomas Huth wrote: > On 30/03/2021 13.19, Daniel P. Berrangé wrote: >> On Mon, Mar 29, 2021 at 03:10:36PM +0100, Stefan Hajnoczi wrote:
>>> Traditionally ccache (https://ccache.dev/) was used to detect >>> recompilation of the same compiler input files. This is trickier to do >>> in GitLab CI since it would be necessary to share and update a cache, >>> potentially between untrusted users. Unfortunately this shifts the >>> bottleneck from CPU to network in a CI-as-a-Service environment since >>> the cached build output needs to be accessed by the linker on the CI >>> runner but is stored remotely. >> >> Our docker containers install ccache already and I could have sworn >> that we use that in gitlab, but now I'm not so sure. We're only >> saving the "build/" directory as an artifact between jobs, and I'm >> not sure that directory holds the ccache cache. > > AFAIK we never really enabled ccache in the gitlab-CI, only in Travis. Back then the Travis setup was simpler, and it took me 2 to 3 weeks to get it right (probably spending 3 to 4h a day on it). >>> This is as far as I've gotten with thinking about CI efficiency. Do you >>> think these optimizations are worth investigating or should we keep it >>> simple and just disable many builds by default? >> >> ccache is a no-brainer and assuming it isn't already working with >> our gitlab jobs, we must fix that asap. > > I've found some nice instructions here: > > https://gould.cx/ted/blog/2017/06/10/ccache-for-Gitlab-CI/ > > ... and just kicked off a build with these modifications, let's see how > it goes... But we cross-build in Docker containers, so you need to mount the cache dir in the container and set the CCACHE_DIR env var, isn't it? Watch out about custom runners. If we do too many changes on the free-tier runners, we'll never have the custom runner series integrated. My 2 cents. Regards, Phil.
